Litert Runtime
Android (Google)OfficialCreated by Google LLCDistributed by Android (Google)
- Skill
- Tool integration
- Software development
- Apache-2.0
- No issues found
AutoGineer summary
Written by AutoGineer from the skill’s own files, not by the author
This skill provides a comprehensive guide for integrating LiteRT (.tflite) models into Android applications using the modern CompiledModel API. It covers project configuration, efficient inference implementation on CPU and GPU, memory management, and migration from legacy TensorFlow Lite Interpreter code. The skill emphasizes performance best practices, such as single-thread confinement, buffer reuse, and proper warm-up procedures to ensure smooth on-device execution.
What it does
- Configures Gradle dependencies for LiteRT bundled or Google Play services runtimes.
- Implements a thread-safe, AutoCloseable inference helper class.
- Provides patterns for GPU acceleration with automatic CPU fallback.
- Optimizes memory usage through TensorBuffer reuse and weight caching.
- Guides the migration process from legacy org.tensorflow.lite.Interpreter APIs.
- Includes preprocessing logic to convert Bitmaps into model-compatible input tensors.
When to use it
- When building new Android applications requiring on-device ML inference.
- When migrating existing apps from legacy TensorFlow Lite Interpreter to LiteRT.
- When optimizing model inference latency and memory footprint on Android.
- When needing to support GPU-accelerated inference with robust fallback mechanisms.
Advantages
- Uses the modern, performant CompiledModel API for efficient execution.
- Ensures thread safety by enforcing single-thread confinement for model operations.
- Provides clear migration paths for legacy TFLite implementations.
- Supports flexible runtime choices between bundled and Google Play services.
Usage examples
- “How do I set up my Android project to use LiteRT with GPU acceleration?”
- The skill provides the necessary Gradle dependencies for the bundled runtime and instructions for configuring CompiledModel.Options to enable the GPU accelerator.
- “I need to migrate my existing TFLite Interpreter code to LiteRT. Where do I start?”
- The skill outlines a seven-step migration process, including capturing golden outputs, swapping dependencies, and replacing legacy delegates with the CompiledModel API.
Install
Download the skill as a ZIP:
https://mp-staging.autogineer.com/agent-skills/android/litert-runtime/downloadUnzip it into the cross-client skills folder of your project:
mkdir -p .agents/skills && unzip litert-runtime.zip -d .agents/skills
Documentation by Google LLC
This skill provides step-by-step guidance for building an Android app that runs
a .tflite model with the LiteRT CompiledModel API
(com.google.ai.edge.litert:litert 2.x; source repository) on the CPU or
the GPU, with the runtime bundled in the APK or from Google Play services. Don't
use the deprecated org.tensorflow.lite.Interpreter API. For
open text language models using .litertlm files, suggest the litert-lm
skill.
Prerequisites
A Kotlin Android project (
minSdkmust be24or higher, as declared by the LiteRT 2.3.0 AARs).The runtime that Step 1 picks: the Google Play services runtime (
com.google.android.gms:play-services-litert, the default; see LiteRT in Google Play services), or the bundled runtime (implementation("com.google.ai.edge.litert:litert:2.3.0")andimplementation("com.google.ai.edge.litert:litert-gpu:2.3.0")from Google Maven). For the bundled runtime, from 2.3.0 the GPU accelerator is the second artifact: withlitertalone the app builds, andCompiledModel.createwithCompiledModel.Options(Accelerator.GPU)throwsLiteRtExceptionbecause the APK has no GPU accelerator library. WithCompiledModel.Options(Accelerator.GPU, Accelerator.CPU)it doesn't throw: the model runs on the CPU while the app reports the GPU, and only Logcat shows it (GPU accelerator could not be loaded and registered). No manifest entry is needed:litert-api, whichlitertdepends on, declares the OpenCL driver libraries (uses-native-library) in its own manifest.A verified
.tflitemodel and its input contract (input dimensions, mean, standard deviation, channel order, and tensor layout). For models with a verified recipe, inspect the LiteRT model recipes.
Critical mandates
Single-thread confinement : One single-thread executor or confined coroutine dispatcher must own the
CompiledModelinstance. Creation, inference, and closing must execute on that same background thread, never on the main thread.Buffer lifecycle : Allocate
TensorBufferinstances once usingcreateInputBuffersandcreateOutputBuffers, reuse them across inferences, and close all input and output buffers before closing theCompiledModel.Synchronous timing :
model.runenqueues work on the GPU and returns before GPU execution completes. Always measuremodel.runand output buffer readback (outputs[0].readFloat) together.Warm-up and exception safety : Execute one warm-up inference inside the helper constructor's
initblock so first-run GPU shader compilation doesn't block the first user interaction, and callcloseincatchbefore rethrowing so neitherCompiledModelnorTensorBufferhandles leak if buffer allocation or warm-up throws.Runtime choice : Before Step 1, you must read LiteRT in Google Play services and choose the runtime. Use the Google Play services runtime unless the app needs the NPU, custom operators, or a Support library class that this skill can't replace (such as
MetadataExtractor), or also targets devices without Google Play services. Never put both runtimes in the same build variant.Migration : When the app already uses
org.tensorflow.litecode, you must read the "Migrating from legacy TFLite Interpreter" section in full before you edit any file. Migrated code must not import anyorg.tensorflow.liteclass (includingDataType), every legacy GPU choice (GpuDelegate, or the Support library'sModel.Device.GPU) becomesCompiledModel.Options(Accelerator.GPU, Accelerator.CPU), and an NNAPI choice maps to that same GPU option.
Detailed steps
Step 1: Configure Gradle and place the model file
Search the project for libs.versions.toml, gradle.properties,
app/build.gradle.kts, and any existing org.tensorflow.lite imports before
editing build files. Then pick the runtime:
Google Play services runtime (default) : Use it unless one of the bundled runtime cases applies, also when the request doesn't name a runtime. You must read LiteRT in Google Play services now: its dependencies replace the ones in this step, and its
Environmentreplaces theenvironmentproperty of Step 2. Use version17.0.0exactly, and run./gradlew assembleDebug(not--dry-run) after you add it. Switch to the bundled runtime only when that build reportsCould not find com.google.android.gms:play-services-litert:17.0.0, and tell the user, also when the user asked for Google Play services. Never leave the build broken.Bundled runtime : Use it only when the app needs the NPU, custom operators, or a Support library class that this skill can't replace (such as
MetadataExtractor), when the app also targets devices without Google Play services, or when the user asks for it.FileUtil,ImageProcessor,TensorImage, andTensorLabeldon't count: replace them as LiteRT in Google Play services describes. For the bundled runtime, configureminSdkand add the LiteRT runtime and GPU accelerator dependencies inapp/build.gradle.kts:android { defaultConfig { minSdk = 24 } }
dependencies { implementation("com.google.ai.edge.litert:litert:2.3.0") implementation("com.google.ai.edge.litert:litert-gpu:2.3.0") }
Place bundled models at app/src/main/assets/model.tflite (AGP stores .tflite
files uncompressed by default). Deliver a model that exceeds the base APK size
limit through Play for On-Device AI as an AI pack, or download it into
context.filesDir (bundletool --local-testing installs AI packs locally
without Google Play). Read an install-time AI pack through AssetManager. Load
a fast-follow or on-demand AI pack or a downloaded file from its on-disk
filePath using CompiledModel.create(filePath, CompiledModel.Options(accelerator), environment) in place of the asset call in
Step 2. Obtain the pack's directory from the AI Delivery library
(com.google.android.play:ai-delivery) using
AiPackManagerFactory.getInstance(context).getPackLocation(name)?.assetsPath(),
which returns null until the pack has finished downloading.
With 2.3.0, nothing is added to gradle.properties: litert 2.3.0 declares
the namespace com.google.ai.edge.litert.impl and its dependency litert-api
2.3.0 declares com.google.ai.edge.litert, so AGP 9.x builds the project. With
2.2.0 or 2.1.6, both libraries declare com.google.ai.edge.litert, and AGP
9.x stops at processDebugMainManifest (AGP 8.x reports it as a warning and
builds); for those two versions, add android.uniquePackageNames=false to
gradle.properties, and remove it when moving to 2.3.0 because it also hides
the same clash between any other two libraries. Excluding litert-api instead
doesn't work: CompiledModel and Accelerator are in litert-api, and the
build then fails at compileDebugKotlin.
Step 2: Implement the inference helper class
Isolate CompiledModel and its TensorBuffer instances in a dedicated
AutoCloseable helper class free of Android UI types:
import android.content.Context
import com.google.ai.edge.litert.Accelerator
import com.google.ai.edge.litert.CompiledModel
import com.google.ai.edge.litert.Environment
import com.google.ai.edge.litert.TensorBuffer
const val INPUT_SIZE = 224 * 224 * 3
private val environment by lazy { Environment.create() }
class Classifier(context: Context, accelerator: Accelerator) : AutoCloseable {
private val model = CompiledModel.create(
context.assets,
"model.tflite",
CompiledModel.Options(accelerator),
environment
)
private val inputs = mutableListOf<TensorBuffer>()
private val outputs = mutableListOf<TensorBuffer>()
init {
try {
inputs += model.createInputBuffers()
outputs += model.createOutputBuffers()
infer(FloatArray(INPUT_SIZE))
} catch (e: Exception) {
close()
throw e
}
}
fun infer(input: FloatArray): FloatArray {
inputs[0].writeFloat(input)
model.run(inputs, outputs)
return outputs[0].readFloat()
}
override fun close() {
(inputs + outputs).forEach { it.close() }
model.close()
}
}Accelerator.CPU requires nothing from the device. Accelerator.GPU compiles
the graph for the GPU; when the GPU can't execute the model, the constructor
throws LiteRtException (from create for an operator the GPU doesn't support,
and on the Android emulator from create or from buffer allocation). A missing
asset or a file that isn't a valid model also throws LiteRtException from
create.
The buffers are created once with the model, reused for every inference, and
closed before the model. One Environment serves every model in the process and
stays open, so LiteRT loads the GPU accelerator library once rather than on
every create call. The constructor ends with one warm-up inference (the first
GPU run includes shader compilation) and closes the model and its buffers if any
step in init throws. INPUT_SIZE is the model's input element count:
writeFloat throws on a longer array and writes a shorter one without an error,
so a wrong size produces an incorrect result. For int8 or uint8 tensors, use
writeInt8 and readInt8 on TensorBuffer instead of writeFloat and
readFloat.
When memory matters, use the following options:
Load from a path :
create(context.assets, ...)keeps a copy of the asset's bytes in the native heap while the model is open. Copy the asset once intocontext.noBackupFilesDir(a downloaded or AI pack file is already on disk) and callcreate(filePath, ...)so the file is mapped and reclaimable. Version the staged filename, bump the version when the bundled model changes, and delete the previous version's file. Copy to a temporary file and rename it so a copy interrupted by process death is never loaded:import android.content.Context import com.google.ai.edge.litert.Accelerator import com.google.ai.edge.litert.CompiledModel import java.io.File
fun stageModelAsset( context: Context, assetName: String = "model.tflite", version: String = "v1" ): File = File(context.noBackupFilesDir, "$version-$assetName").also { file -> if (!file.isFile) { context.noBackupFilesDir.listFiles()?.forEach { if (it.name.endsWith("-$assetName")) it.delete() } val tmp = File("${file.path}.tmp") context.assets.open(assetName).use { input -> tmp.outputStream().use { output -> input.copyTo(output) } } check(tmp.renameTo(file)) } }
fun memoryOptions( context: Context, accelerator: Accelerator = Accelerator.CPU, assetName: String = "model.tflite", version: String = "v1" ): CompiledModel.Options = CompiledModel.Options(accelerator).apply { val prefix = "$assetName-" val cache = File(context.codeCacheDir, "$prefix$version.xnnpack_cache") context.codeCacheDir.listFiles()?.forEach { if (it.name.startsWith(prefix) && it.name.endsWith(".xnnpack_cache") && it != cache) { it.delete() } } cpuOptions = CompiledModel.CpuOptions( numThreads = 4, xnnPackFlags = 0x23, xnnPackWeightCachePath = cache.path ) }
Weight cache (
xnnPackWeightCachePath): Written on the first load (~1x an FP32 or ~2x an FP16 model's file size) and mapped on later loads, moving packed FP32 weights off the heap. Version the cache filename and delete outdated caches.Transient indirection buffer (
xnnPackFlags = 0x23) : Default0x3plus flag0x20, replacing per-operator convolution pointer tables with one shared buffer (byte-identical outputs, ~1% latency overhead at 4 threads, ~3% single-threaded). Because the value replaces the default,0x20alone leavesint8anduint8operators to fallback kernels.cpuOptionsis ignored on the GPU.Close after a task: Closing frees the runtime workspace; recreating with a warm cache takes ~20 ms, whereas the first inference after any load pays a full warm-up run, so close when the next use isn't imminent.
Step 3: Wire the ViewModel and Compose screen
Drive the helper from a single-thread executor inside an AndroidViewModel, and
fall back to Accelerator.CPU when GPU compilation fails:
import android.app.Application
import androidx.lifecycle.AndroidViewModel
import com.google.ai.edge.litert.Accelerator
import com.google.ai.edge.litert.LiteRtException
import java.util.concurrent.Executors
import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.SupervisorJob
import kotlinx.coroutines.asCoroutineDispatcher
import kotlinx.coroutines.cancel
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.flow.asStateFlow
import kotlinx.coroutines.flow.update
import kotlinx.coroutines.launch
data class UiState(
val ready: Boolean = false,
val accelerator: Accelerator = Accelerator.CPU,
val result: List<Float>? = null,
val error: String? = null
)
class MainViewModel(app: Application) : AndroidViewModel(app) {
private val executor = Executors.newSingleThreadExecutor()
private val scope = CoroutineScope(
SupervisorJob() + executor.asCoroutineDispatcher()
)
private var classifier: Classifier? = null
private val _uiState = MutableStateFlow(UiState())
val uiState: StateFlow<UiState> = _uiState.asStateFlow()
fun load(preferred: Accelerator = Accelerator.GPU) {
scope.launch {
if (classifier != null) return@launch
val appContext = getApplication<Application>()
val selected = try {
classifier = Classifier(appContext, preferred)
preferred
} catch (e: LiteRtException) {
if (preferred != Accelerator.CPU) {
try {
classifier = Classifier(appContext, Accelerator.CPU)
Accelerator.CPU
} catch (cpuError: LiteRtException) {
_uiState.value = UiState(error = cpuError.message)
return@launch
}
} else {
_uiState.value = UiState(error = e.message)
return@launch
}
}
_uiState.value = UiState(ready = true, accelerator = selected)
}
}
fun classify(input: FloatArray) {
scope.launch {
try {
classifier?.infer(input)?.let { result ->
_uiState.update {
it.copy(result = result.toList(), error = null)
}
}
} catch (e: LiteRtException) {
_uiState.update { it.copy(error = e.message) }
}
}
}
override fun onCleared() {
scope.launch { classifier?.close() }.invokeOnCompletion {
scope.cancel()
}
executor.shutdown()
}
}One single-thread executor owns the model: creation, inference, and closing
execute only on it, never on the main thread. The ViewModel keeps its own
scope because viewModelScope is canceled before onCleared runs, so a close
launched there wouldn't run; onCleared cancels scope once close completes.
load is a no-op when a model is already loaded across configuration changes; a
failed load leaves classifier null. classify reports any
LiteRtException (in package com.google.ai.edge.litert) in error and keeps
the model.
The following Compose screen takes the picture (bitmap) from the photo picker
or CameraX and runs preprocess in the click handler (a few milliseconds for
224x224; move preprocess onto the executor for images exceeding 224x224).
viewModel and collectAsStateWithLifecycle come from
androidx.lifecycle:lifecycle-viewmodel-compose and
androidx.lifecycle:lifecycle-runtime-compose:
import android.graphics.Bitmap
import androidx.compose.foundation.layout.Column
import androidx.compose.material3.Button
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.runtime.LaunchedEffect
import androidx.compose.runtime.getValue
import androidx.lifecycle.compose.collectAsStateWithLifecycle
import androidx.lifecycle.viewmodel.compose.viewModel
import com.google.ai.edge.litert.Accelerator
@Composable
fun MainScreen(
bitmap: Bitmap,
viewModel: MainViewModel = viewModel()
) {
val state by viewModel.uiState.collectAsStateWithLifecycle()
LaunchedEffect(Unit) { viewModel.load(Accelerator.GPU) }
Column {
state.error?.let { Text(it) }
Button(
onClick = { viewModel.classify(preprocess(bitmap)) },
enabled = state.ready
) {
Text("Run (${state.accelerator})")
}
state.result?.let { scores ->
Text("Top class: ${scores.indices.maxBy { i -> scores[i] }}")
}
}
}Step 4: Preprocess inputs to match the model contract
Convert the Bitmap into the model's input with preprocess. Match the
model's exact input dimensions, normalization mean and standard deviation,
channel order, and memory layout (this example uses NHWC, RGB, scaled to
-1.0..1.0). Mismatched mean or std values produce corrupted predictions
that look like a broken model:
import android.graphics.Bitmap
fun preprocess(
bitmap: Bitmap,
size: Int = 224,
mean: Float = 127.5f,
std: Float = 127.5f
): FloatArray {
val scaled = checkNotNull(
Bitmap.createScaledBitmap(bitmap, size, size, true)
.copy(Bitmap.Config.ARGB_8888, false)
)
val pixels = IntArray(size * size).also {
scaled.getPixels(it, 0, size, 0, 0, size, size)
}
val out = FloatArray(size * size * 3)
pixels.forEachIndexed { i, p ->
out[i * 3] = ((p shr 16 and 0xFF) - mean) / std
out[i * 3 + 1] = ((p shr 8 and 0xFF) - mean) / std
out[i * 3 + 2] = ((p and 0xFF) - mean) / std
}
return out
}The scaled bitmap is copied to Bitmap.Config.ARGB_8888 because ImageDecoder
returns Bitmap.Config.HARDWARE bitmaps by default on Android 9 and higher
(including from a photo picker Uri), scaling preserves that config, and
getPixels can't read HARDWARE bitmaps; the copy is of the 224x224 scaled
bitmap rather than the full image. Scaling doesn't preserve the aspect ratio;
crop first if the model requires center cropping. An audio or embedding model
replaces preprocess with its own input encoding and keeps the inference class.
Step 5: Verify output on a physical device
Run the app on a physical device that runs Android from Android Studio. Compare
the app's output with the model's reference output for one fixed input on
Accelerator.CPU first and then on Accelerator.GPU; when verifying model
outputs on a physical device, you must follow device verification.
Migrating from legacy TFLite Interpreter
An existing app on org.tensorflow.lite.Interpreter (from
org.tensorflow:tensorflow-lite* or from com.google.ai.edge.litert:litert
1.x) moves to the CompiledModel code of Steps 1 to 5 in seven steps, in this
order. Before you change any file of an
existing app, you must read migration, which gives the commands and code
for each step. An app on the Interpreter API in Google Play services
(play-services-tflite-java) follows LiteRT in Google Play services
instead:
Find the runtime: Read the build files and the resolved dependencies, and note the delegate, the thread count, and the Support library classes that the app uses.
Capture a golden output first : While the legacy runtime is still in the project, record the app's output for one fixed input on the CPU. This step needs a device or an emulator. Run the capture test with
am instrument, notconnectedDebugAndroidTest, which uninstalls the app and deletes the file.Swap the dependencies : Delete every legacy artifact line (never leave it commented out) and add the dependencies of the runtime that Step 1 picks: by default those of LiteRT in Google Play services, with the same fallback to the bundled runtime when Gradle can't find
play-services-litert. On the Play services runtime, replace the Support library classes as LiteRT in Google Play services describes. On the bundled runtime, an app keeps them withcom.google.ai.edge.litert:litert-support:2.3.0, with the packageorg.tensorflow.lite.supportrenamed tocom.google.ai.edge.litert.support(LiteRT Support migration guide).Replace the delegates :
GpuDelegatebecomesCompiledModel.Options(Accelerator.GPU, Accelerator.CPU), so the CPU runs the operators that the GPU doesn't support. An NNAPI choice in the app maps to the same GPU option, not toAccelerator.CPUalone or to an NPU accelerator.setNumThreads(n)becomes new CPU options withcpuOptions = CompiledModel.CpuOptions(numThreads = n).Rewrite the inference code : Replace
InterpreterandFileUtil.loadMappedFilewith theClassifierhelper of Step 2, created and run only on the single-thread executor of Step 3, never in a constructor that the main thread calls. When no caller retries on the CPU as in Step 3, create the helper's GPU option with the CPU as a second accelerator (CompiledModel.Options(Accelerator.GPU, Accelerator.CPU)). Aninterpreter.runSignature(inputs, outputs, key)call becomes the helper's buffers andruncall with the signature key, as follows. This form runs on both runtimes, also after the app changes runtime or upgrades LiteRT; when the signature has more than one input or output, you must follow the signature rules in migration:inputs += model.createInputBuffers("serving_default") outputs += model.createOutputBuffers("serving_default") model.run(inputs, outputs, "serving_default")Verify in two stages: The build passes and the searches for legacy references print nothing; then the migrated output matches the golden output on a device.
Report what changed and what didn't migrate : When the app used NNAPI, tell the user that it now runs on the GPU with the CPU as a second accelerator, which changes the speed and the numerical results of a quantized model that ran on a DSP or an NPU before; point them to NPU acceleration with LiteRT for the NPU. Tell the user about the TensorFlow Lite Task Library, C++ modules, and Select TensorFlow operators, which this skill doesn't migrate.
Migration mandates that apply to every migrated app:
No
org.tensorflow.liteimports : Migrated code must not import anyorg.tensorflow.liteclass, includingorg.tensorflow.lite.DataType; the LiteRT artifacts keep these classes only for backward compatibility. Replace Support calls that needDataType(TensorImage(DataType.FLOAT32),TensorBuffer.createFixedSize, andTensorLabel) withpreprocessof Step 4, theCompiledModelbuffers, and labels paired with the output scores. Remove every existingorg.tensorflow.liteimport line, not only the ones that you add.
Antipatterns
Never create, run, or close
CompiledModelon the Android main thread.Never allocate new
TensorBufferinstances on every inference call; failing to close per-frameTensorBufferobjects leaks C++ heap memory.Never benchmark
model.runwithout including output buffer readback (outputs[0].readFloat).Never mix
org.tensorflow:tensorflow-lite*artifacts (such asorg.tensorflow:tensorflow-lite-support) withcom.google.ai.edge.litert:*artifacts; both ship theorg.tensorflow.lite.*classes and cause duplicate class build failures.Never use
Accelerator.GPUon LiteRT 2.3.0 without adding thecom.google.ai.edge.litert:litert-gpu:2.3.0dependency alongsidecom.google.ai.edge.litert:litert:2.3.0; with the CPU as a second accelerator, the model silently runs on the CPU.Never set
xnnPackFlags = 0x20on its own to get the transient indirection buffer; the value replaces the runtime's default0x3, so pass0x23.
Checklist
[ ] Is
minSdkset to24or higher and are bothcom.google.ai.edge.litert:litert:2.3.0andcom.google.ai.edge.litert:litert-gpu:2.3.0added inapp/build.gradle.kts?[ ] Are all legacy
org.tensorflow:tensorflow-lite*dependencies andorg.tensorflow.lite.*imports removed?[ ] Do
CompiledModel.create,model.run, andcloseall execute on the same single-thread background dispatcher?[ ] Are
TensorBufferinstances allocated once, reused across calls, and closed beforemodel.close, including wheninitor warm-up throws?[ ] Does
loadexecute one warm-up inference and fall back toAccelerator.CPUifAccelerator.GPUthrowsLiteRtException?[ ] Does the project compile with
./gradlew assembleDebug?[ ] If memory matters, is the model created from a path with a versioned weight cache and
xnnPackFlags = 0x23on the CPU, and closed after a task when the next use isn't imminent?
Files
- ATTRIBUTION.mdAdded by AutoGineer973 B
- LICENSELicence file added by AutoGineer11.1 KiB
- SKILL.md27.0 KiB
- references/android/google/play/on-device-ai.md36.8 KiB
- references/migration.md18.9 KiB
- references/play-services.md10.8 KiB
- references/verify.md2.4 KiB
references/android/google/play/on-device-ai.md
Introduction
Play for On-device AI brings the benefits of Android App Bundles and Google Play delivery to custom ML model distribution so you can improve model performance with less device ecosystem complexity at no additional cost. It lets you publish a single artifact to Play containing your code, assets, and ML models and to choose from a number of delivery modes and targeting options.
Benefits
Upload a single publishing artifact to Google Play and delegate hosting, delivery, updates, and targeting to Play at no additional cost.
Deliver your ML models at install-time, fast-follow, or on-demand.
Install-time delivery can guarantee that a very large model is present when your app is opened. Your model will be installed as an APK.
Fast-follow delivery occurs automatically in the background after your app has been installed. Users may open your app before your model has been fully downloaded. Your model will be downloaded to your app's internal storage space.
On-demand delivery lets you request the model at runtime, which is useful if the model is only required for certain user-flows. Your model will be downloaded to your app's internal storage space.
Deliver variants of your ML models that are targeted to specific devices based on device model, system properties, or RAM.
Keep app updates small and optimized with Play's automatic patching, which means only the differences in files need to be downloaded.
Considerations
By using Play for On-device AI you agree to the terms in the Google Play Developer Distribution Agreement and the Play Core Software Development Kit Terms of Service.
Models downloaded by Play for On-device AI should only be used by your apps. Models shouldn't be offered to other apps.
Individual AI packs can be up to 1.5GB, based on their compressed download sizes. The maximum cumulative app size of any version of your app generated from your app bundle is 4GB.
Apps over 1GB in size must set min SDK Level to 21 or higher.
How to use Play for On-device AI
Play for On-device AI uses AI packs. You package custom models that are ready for distribution in AI packs in your app bundle. You can choose whether the AI pack should be delivered at install-time, fast-follow, or on-demand.
By packaging AI packs with your app bundle, you can use all of Play's existing testing and release tools, such as test tracks and staged rollouts to manage your app's distribution with your custom models.
AI packs are updated together with the app binary. If your new app release doesn't make changes to an AI pack, then Play's automatic patching process will ensure the user doesn't have to re-download it. Play will just download what's changed when it updates the app.
AI packs only contain models. Java/Kotlin and native libraries are not allowed. If you need to ship libraries or code to run your ML model, move it into the base module or a feature module. You can configure your feature module so that it has the same download and targeting settings as the AI pack.
[!NOTE] Note: AI packs don't contain Java/Kotlin or native libraries. You can ship libraries or code to run your ML models in feature modules.
Use LiteRT and MediaPipe with AI packs
You can use LiteRT and MediaPipe with AI packs. Package your model in an AI pack and then access it using the instructions for install-time packs or for fast-follow and on-demand packs.
Further reading:
The sample app shows how you can package a LiteRT model in an AI pack and load it at runtime.
There are many pretrained LiteRT models that you can use in AI packs to get started.
Getting started with MediaPipe
For fast-follow and on-demand packs, you can use AssetCache.java to load your assets (eg.
.binarypbfiles) by their file paths.For install-time packs, you can use AndroidAssetUtil.java.
Get started with AI packs
At a high level, here's how you can start using Play for On-device AI:
Package your models into AI packs into your Android App Bundle and specify how the AI packs should be delivered.
[Optional] If you want to deliver different models to different devices, you can configure device targeting for your AI packs. For example, you could deliver AI pack A to a specific device model, AI pack B to devices with at least 6GB of RAM, and all other devices could receive no model.
[Optional] If you're using on-demand or fast-follow delivery, integrate the Play AI Delivery Library into your app to download your AI packs as needed.
Test and release your app bundle to Google Play.
Check Android Gradle Plugin version
To use AI packs, ensure that your Android Gradle Plugin (AGP) version is at least 8.8. This version is packaged with Android Studio Ladybug 2.
Extract your model into an AI pack
Android Studio is not required for the following steps.
In the top-level directory of your project, create a directory for the AI pack. This directory name is used as the AI pack name. AI pack names must start with a letter and can only contain letters, numbers, and underscores.
In the AI pack directory, create a
build.gradlefile and add the following code. Make sure to specify the name of the AI pack and only one delivery type:// In the AI pack's build.gradle file: plugins { id 'com.android.ai-pack' } aiPack { packName = "ai-pack-name" // Directory name for the AI pack dynamicDelivery { deliveryType = "[ install-time | fast-follow | on-demand ]" } }In the project's app
build.gradlefile, add the name of every AI pack in your project as shown below:// In the app build.gradle file: android { ... assetPacks = [":ai-pack-name", ":ai-pack2-name"] }In the project's
settings.gradlefile, include all AI packs in your project as shown below:// In the settings.gradle file: include ':app' include ':ai-pack-name' include ':ai-pack2-name'Inside your AI pack, create a
src/main/assets/directory.Place your models in the
src/main/assetsdirectory. You can create subdirectories in here as well. The directory structure for your app should now look like the following:build.gradlesettings.gradleapp/ai-pack-name/build.gradleai-pack-name/src/main/assets/your-model-directories
Add code to load and run your models. How you do this will depend on the delivery mode of your AI packs. See instructions for install-time and fast-follow/on-demand below.
[Optional] Configure device targeting to deliver different models to different devices.
Build the Android App Bundle with Gradle. In the generated app bundle, the root-level directory now includes the following:
ai-pack-name/manifest/AndroidManifest.xml: Configures the AI pack's identifier and delivery modeai-pack-name/assets/your-model-directories: Directory that contains all assets delivered as part of the AI pack
Gradle generates the manifest for each AI pack and outputs the
assets/directory for you.
Configure install-time delivery
AI packs configured as install-time are immediately available at app launch. Use the Java AssetManager API to access AI packs served in this mode:
import android.content.res.AssetManager;
...
Context context = createPackageContext("com.example.app", 0);
AssetManager assetManager = context.getAssets();
InputStream is = assetManager.open("model-name");Configure fast-follow and on-demand delivery
To download AI packs with fast-follow or on-demand delivery, use the Play AI Delivery Library.
Declare dependency on the Play AI Delivery Library
In your app's build.gradle file, declare a dependency on the Play AI Delivery
Library:
dependencies {
...
implementation "com.google.android.play:ai-delivery:0.1.1-alpha01"
}Check status
Each AI pack is stored in a separate folder in the app's internal storage. Use
the
getPackLocation()
method to determine the root folder of an AI pack. This method returns the
following values:
| Return value | Status |
|---|---|
A valid AiPackLocation object | AI pack root folder is ready for immediate access at assetsPath() |
null | Unknown AI pack or AI packs are not available |
[!NOTE] Note: Don't rely on cached AI pack locations between app launches. The app should always check for the existence of AI packs at every launch. AI packs may become invalid due to app updates or if the user clears the app data.
Get download information about AI packs
Use the
getPackStates()
method to determine the size of the download and whether the pack is already
downloading.
Task<AiPackStates> getPackStates(List<String> packNames)getPackStates() is an asynchronous method that returns a Task<AiPackStates>.
The
packStates()
method of an AiPackStates object returns a Map<String, AiPackState>. This
map contains the state of each requested AI pack, keyed by its name:
Map<String, AiPackState> AiPackStates#packStates()The final request is shown by the following:
final String aiPackName = "myAiPackName";
aiPackManager
.getPackStates(Collections.singletonList(aiPackName))
.addOnCompleteListener(new OnCompleteListener<AiPackStates>() {
@Override
public void onComplete(Task<AiPackStates> task) {
AiPackStates aiPackStates;
try {
aiPackStates = task.getResult();
AiPackState aiPackState =
aiPackStates.packStates().get(aiPackName);
} catch (RuntimeExecutionException e) {
Log.d("MainActivity", e.getMessage());
return;
});The following
AiPackState
methods provide the size of the AI pack, the downloaded amount so far (if
requested), and the amount already transferred to the app:
To get the status of an AI pack, use the
status()
method, which returns the status as an integer that corresponds to a constant
field in the
AiPackStatus
class. An AI pack that's not installed yet has the status
AiPackStatus.NOT_INSTALLED.
If a request fails, use the
errorCode()
method, whose return value corresponds to a constant field in the
AiPackErrorCode
class.
Install
Use the
fetch()
method to download an AI pack for the first time or call for the update of an AI
pack to complete:
Task<AiPackStates> fetch(List<String> packNames)This method returns an
AiPackStates
object containing a list of packs and their initial download states and sizes.
If an AI pack requested via fetch() is already downloading, the download
status is returned and no additional download is started.
[!NOTE] Note: In most cases, you implement a
listenerto track the download and installation process as covered in the next section.
Monitor download states
You should implement an
AiPackStateUpdateListener
to track the installation progress of AI packs. The status updates are broken
down per pack to support tracking the status of individual AI packs. You can
start using available AI packs before all other downloads for your request have
completed.
void registerListener(AiPackStateUpdateListener listener)
void unregisterListener(AiPackStateUpdateListener listener)[!NOTE] Note: The Play Store automatically triggers the download of any
fast-followpacks after the user installs or updates the app. However, these packs may not be ready to use immediately. You must check the status of thefast-followpacks at every app launch. If the download is in progress, monitor it with a listener. If the download is cancelled or paused, you can resume it by using thefetch()method, as covered in the Install section.
Large downloads
If the download is larger than 200 MB and the user is not on Wi-Fi, the download
does not start until the user explicitly gives their consent to proceed with the
download using a mobile data connection. Similarly, if the download is large and
the user loses Wi-Fi, the download is paused and explicit consent is required to
proceed using a mobile data connection. A paused pack has state
WAITING_FOR_WIFI. To trigger the UI flow to prompt the user for consent, use
the
showConfirmationDialog()
method.
Note that if the app does not call this method, the download is paused and will resume automatically only when the user is back on a Wi-Fi connection.
Required user confirmation
If a pack has the REQUIRES_USER_CONFIRMATION status, the download won't
proceed until the user accepts the dialog that is shown with
showConfirmationDialog().
This status can occur when the app is not recognized by Play---for example, if the
app was sideloaded. Note that calling
showConfirmationDialog()
in this case will cause the app to be updated. After the update, you will need
to request the AI packs again.
The following is an example implementation of a listener:
AiPackStateUpdateListener aiPackStateUpdateListener = new AiPackStateUpdateListener() {
private final ActivityResultLauncher<IntentSenderRequest> activityResultLauncher =
registerForActivityResult(
new ActivityResultContracts.StartIntentSenderForResult(),
new ActivityResultCallback<ActivityResult>() {
@Override
public void onActivityResult(ActivityResult result) {
if (result.getResultCode() == RESULT_OK) {
Log.d(TAG, "Confirmation dialog has been accepted.");
} else if (result.getResultCode() == RESULT_CANCELED) {
Log.d(TAG, "Confirmation dialog has been denied by the user.");
}
}
});
@Override
public void onStateUpdate(AiPackState aiPackState) {
switch (aiPackState.status()) {
case AiPackStatus.PENDING:
Log.i(TAG, "Pending");
break;
case AiPackStatus.DOWNLOADING:
long downloaded = aiPackState.bytesDownloaded();
long totalSize = aiPackState.totalBytesToDownload();
double percent = 100.0 * downloaded / totalSize;
Log.i(TAG, "PercentDone=" + String.format("%.2f", percent));
break;
case AiPackStatus.TRANSFERRING:
// 100% downloaded and assets are being transferred.
// Notify user to wait until transfer is complete.
break;
case AiPackStatus.COMPLETED:
// AI pack is ready to use. Run the model.
break;
case AiPackStatus.FAILED:
// Request failed. Notify user.
Log.e(TAG, aiPackState.errorCode());
break;
case AiPackStatus.CANCELED:
// Request canceled. Notify user.
break;
case AiPackStatus.WAITING_FOR_WIFI:
case AiPackStatus.REQUIRES_USER_CONFIRMATION:
if (!confirmationDialogShown) {
aiPackManager.showConfirmationDialog(activityResultLauncher);
confirmationDialogShown = true;
}
break;
case AiPackStatus.NOT_INSTALLED:
// AI pack is not downloaded yet.
break;
case AiPackStatus.UNKNOWN:
Log.wtf(TAG, "AI pack status unknown")
break;
}
}
}Alternatively, you can use the
getPackStates()
method to get the status of current downloads.
AiPackStates
contains the download progress, download status, and any failure error codes.
Access AI packs
You can access an AI pack using file system calls after the download request
reaches the
COMPLETED
state. Use the
getPackLocation()
method to get the root folder of the AI pack.
AI packs are stored in the assets directory within the AI pack root directory.
You can get the path to the assets directory by using the convenience method
assetsPath().
Use the following method to get the path to a specific asset:
private String getAbsoluteAiAssetPath(String aiPack, String relativeAiAssetPath) {
AiPackLocation aiPackPath = aiPackManager.getPackLocation(aiPack);
if (aiPackPath == null) {
// AI pack is not ready
return null;
}
String aiAssetsFolderPath = aiPackPath.assetsPath();
// equivalent to: FilenameUtils.concat(aiPackPath.path(), "assets");
String aiAssetPath = FilenameUtils.concat(aiAssetsFolderPath, relativeAiAssetPath);
return aiAssetPath;
}Configure device targeting
You can follow the device targeting instructions to specify devices or groups of devices that should receive your AI packs.
Other Play AI Delivery API methods
The following are some additional API methods you may want to use in your app.
Cancel request
Use
cancel()
to cancel an active AI pack request. Note that this request is a best-effort
operation.
Remove an AI pack
Use
removePack()
to schedule the removal of an AI pack.
Get locations of multiple AI packs
Use
getPackLocations()
to query the status of multiple AI packs in bulk, which returns a map of AI
packs and their locations. The map returned by getPackLocations() contains an
entry for each pack that is currently downloaded and up-to-date.
Device targeting
Device targeting gives you finer control over which parts of your app bundle are delivered to specific devices. For example, you could ensure that a large model is only delivered to devices with high RAM or you could deliver different versions of a model to different devices.
You can target device properties such as:
System on chip
Device model
Device RAM
Overview of required steps
The following steps are required to enable device targeting:
Define your device groups in an XML file.
Specify which parts of your bundle should go to which device groups.
[Optional] Test your configuration locally.
Upload your bundle (containing the XML file) to Google Play.
Check Android Gradle Plugin version
To use device targeting, ensure that your Android Gradle Plugin (AGP) version is at least 8.10.0. This is packaged with Android Studio (Meerkat 2 and later). Download the latest stable version of Android Studio.
Enable this feature in Android Gradle Plugin
Device targeting must be enabled explicitly in your gradle.properties file:
android.experimental.enableDeviceTargetingConfigApi=trueCreate a device targeting configuration XML file
The device targeting configuration file is an XML file in which you define your
custom device groups. For example, you could define a device group called
qti_v79 that contains all devices with the Qualcomm SM8750 system on chip:
<config:device-targeting-config
xmlns:config="http://schemas.android.com/apk/config">
<config:device-group name="qti_v79">
<config:device-selector>
<config:system-on-chip manufacturer="QTI" model="SM8750"/>
</config:device-selector>
</config:device-group>
</config:device-targeting-config>A device group is composed of up to 5 device selectors. A device is included in a device group if it satisfies any of its device selectors.
A device selector can have one or more device properties. A device is selected if it matches all of the selector's device properties.
If a device matches multiple groups, it will be served the content for the group that is defined first in the XML file. The order you define groups in the XML file is your priority order.
If a device matches no groups, it will receive the default "other" group. This group is automatically generated and shouldn't be defined explicitly.
Available device properties
device_ram : Device RAM requirements
min_bytes (inclusive): Minimum required RAM (in bytes)
max_bytes (exclusive): Maximum required RAM (in bytes)
included_device_ids : Device models to be included in this selector (max of 10000 device_ids per group) . This property is satisfied if the device matches any device_id in the list.
build_brand: Device manufacturer
build_device: Device model code
excluded_device_ids : Device models to be excluded in this selector (max of 10000 device_ids per group) . This property is satisfied if the device matches no device_id in the list.
build_brand: Device manufacturer
build_device: Device model code
required_system_features : Features that a device needs to have to be included by this selector (max of 100 features per group). A device needs to have all system features in this list to satisfy this property.
System feature reference
name: A system feature
forbidden_system_features : Features that a device mustn't have to be included by this selector (max of 100 features per group). If a device has any of the system features in this list it doesn't satisfy this property.
System feature reference
name: A system feature
system-on-chip: System on chips to be included in this selector. A device needs to have any chip in this list to satisfy this property.
manufacturer : System on chip manufacturer
model : System on chip model
[!TIP] Tip: Including multiple properties in a single selector creates a logical AND, for example:
<config:device-selector ram-min-bytes="7000000000"> <config:included-device-id brand="google" device="flame"/> </config:device-selector>will create the condition for all devices with > 7GB of RAM AND it is a Pixel 4 - also written as follows:
If you want an OR condition, create separate selectors in a single device group, for example:
<config:device-selector ram-min-bytes="7000000000"/> <config:device-selector> <config:included-device-id brand="google" device="flame"/> </config:device-selector>will create the condition for all devices with > 7GB of RAM OR it is a Pixel 4 - also written as follows:
Here is an example showing all possible device properties:
<config:device-targeting-config
xmlns:config="http://schemas.android.com/apk/config">
<config:device-group name="myCustomGroup1">
<config:device-selector ram-min-bytes="8000000000">
<config:included-device-id brand="google" device="redfin"/>
<config:included-device-id brand="google" device="sailfish"/>
<config:included-device-id brand="good-brand"/>
<config:excluded-device-id brand="google" device="caiman"/>
<config:system-on-chip manufacturer="Sinclair" model="ZX80"/>
<config:system-on-chip manufacturer="Commodore" model="C64"/>
</config:device-selector>
<config:device-selector ram-min-bytes="16000000000"/>
</config:device-group>
<config:device-group name="myCustomGroup2">
<config:device-selector ram-min-bytes="4000000000" ram-max-bytes="8000000000">
<config:required-system-feature name="android.hardware.bluetooth"/>
<config:required-system-feature name="android.hardware.location"/>
<config:forbidden-system-feature name="android.hardware.camera"/>
<config:forbidden-system-feature name="mindcontrol.laser"/>
</config:device-selector>
</config:device-group>
</config:device-targeting-config>Official device manufacturer and device model codes
You can find the correct formatting for the device manufacturer and model code by using the Device Catalog on the Google Play Console, by either:
Inspecting individual devices using the Device Catalog, and finding the manufacturer and model code in the locations as shown in the example below (For a Google Pixel 4a, the manufacturer is "Google" and the model code is "sunfish")'
Downloading a CSV of supported devices, and using the Manufacturer and Model Code for the build_brand and build_device fields, respectively.
Include your device targeting configuration file in your app bundle
Add the following to your main module's build.gradle file:
android {
...
bundle {
deviceTargetingConfig = file('device_targeting_config.xml')
deviceGroup {
enableSplit = true // split bundle by #group
defaultGroup = "other" // group used for standalone APKs
}
}
...
}device_targeting_config.xml is the path of your configuration file relative to
the main module. This ensures that your configuration file is packaged with your
app bundle.
The deviceGroup clause ensures that the APKs generated from your bundle are
split by device groups.
Use device targeting for your AI packs
You can keep size optimized on devices by only delivering your large models to devices that can run them.
Subdivide your AI packs by device groups by taking the existing AI pack directories created in the last step, and post-fixing the appropriate folders (as described below) with #group_myCustomGroup1, #group_myCustomGroup2, etc. When using the AI packs in your app, you won't need to address folders by postfix (in other words, the postfix is automatically stripped during the build process).
After the previous step, this might look like:
...
.../ai-pack-name/src/main/assets/image-classifier#group_myCustomGroup1/
.../ai-pack-name/src/main/assets/image-classifier#group_myCustomGroup2/
...In this example, you would reference ai-pack-name/assets/image-classifier/
without any postfixes.
Devices in myCustomGroup1 will receive all the assets under
image-classifier#group_myCustomGroup1/, while devices in myCustomGroup2 will
receive all the assets under image-classifier#group_myCustomGroup2/.
Devices that don't belong to either myCustomGroup1 or myCustomGroup2 will
receive an empty ai-pack-name pack.
This is because devices that don't match any device group will receive the
default variant of your AI pack. This includes anything that is not inside a
directory with a #group_suffix.
Once you have downloaded the AI pack, you can check whether your model is present by using the AssetManager for install-time packs or the AiPackManager for fast-follow and on-demand packs. Examples for doing this are shown for all delivery modes in the sample app.
[!IMPORTANT] Important: It's not possible to prevent any variant of your AI pack being delivered to certain devices. Non-targeted devices will always receive the default variant.
Use device targeting for your feature modules
You can also use device targeting for feature modules. Instead of subdividing feature modules by device group, you specify whether the entire module should be delivered based on device group membership.
To deliver a feature module to devices that belong to either myCustomGroup1 or
myCustomGroup2, modify its AndroidManifest.xml:
<manifest ...>
...
<dist:module dist:title="...">
<dist:delivery>
<dist:install-time>
<dist:conditions>
<dist:device-groups>
<dist:device-group dist:name="myCustomGroup1"/>
<dist:device-group dist:name="myCustomGroup2"/>
</dist:device-groups>
...
</dist:conditions>
</dist:install-time>
</dist:delivery>
</dist:module>
...
</manifest>[!NOTE] Note: Devices that are not targeted won't receive the feature module at all. This is different from targeting for AI packs, where devices that are not targeted receive a default, empty variant of the AI pack.
Test locally
Before creating a release for your new bundle, you can test locally with either Internal App Sharing or Bundletool.
Internal App Sharing
Internal App Sharing lets you use an app bundle to quickly generate a URL that you can tap on a local device to install exactly what Google Play would install for that device if that version of the app was live in a test or prod track.
Take a look at the internal app sharing instructions.
Bundletool
Alternatively, you can generate APKs using
bundletool (1.18.0 or above) and sideload
them onto your device. Follow these steps to test your app locally using
bundletool:
Build your app bundle with Android Studio or bundletool.
Generate APKs with the
--local-testingflag:java -jar bundletool-all.jar build-apks --bundle=path/to/your/bundle.aab \ --output=output.apks --local-testingConnect a device and run
bundletoolto sideload the APKs:# Example without Device Targeting Configuration java -jar bundletool.jar install-apks --apks=output.apks# Example with Device Targeting Configuration (you must specify which groups the connected device belongs to) java -jar bundletool.jar install-apks --apks=output.apks --device-groups=myCustomGroup1,myCustomGroup2
Limitations of local testing with bundletool
The following are limitations of local testing with bundletool:
fast-followpacks behave ason-demandpacks. That is, they won't be automatically fetched when the app is sideloaded. Developers need to request them manually when the app starts; this does not require any code changes in your app.Packs fetch from external storage instead of Play, so you cannot test how your code behaves in the case of network errors.
Local testing does not cover the wait-for-Wi-Fi scenario.
Updates are not supported. Before installing a new version of your build, manually uninstall the previous version.
Verify that the correct APKs are being installed
Use the following method to ensure only the correct APKs are installed on the device
adb shell pm path {packageName}You should see something like:
package:{...}/base.apk
package:{...}/split_config.en.apk
package:{...}/split_config.xxhdpi.apk
package:{...}/split_main_ai-pack-name.apk
package:{...}/split_main_ai-pack-name.config.group_myCustomGroup1.apkNote that you will only see APKs in this list, which are made from feature modules and install-time AI packs. On-demand and fast-follow AI packs are not installed as APKs.
Test and release on Google Play
We recommend that you test your app end to end on Google Play with an internal test track.
Once you've done this, you can incrementally release your app update to production with staged roll-outs.
Sample app using Play for On-device AI
Download the sample app.
It demonstrates how to use each of the delivery modes as well as the device targeting configuration. See the local testing section to get started.
Related content
Learn more about Android App Bundles and read the references for the AI Delivery SDK.
Requirements and compatibility
Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta
“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.
Security
No issues found by our static scan· No issues found
A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.
Attribution and licence
- Created by
- Google LLC
- Publisher
- Android (Google) (Official)
- Distributed by
- Android (Google)
- Source
- android/skills · device-ai/litert/litert-runtime · commit bf68e2d
- Licence
- Apache License 2.0 (redistribution allowed)
- LICENSE (added by AutoGineer from LICENSE.txt)
- ATTRIBUTION.md (written by AutoGineer)
- Synced
AutoGineer is not affiliated with or endorsed by Google LLC or Android (Google).
Download ZIP
The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download litert-runtime.zip