ML Kit for On-Device AI Intermediate

ML Kit brings Google's machine learning capabilities to mobile devices. It runs AI models directly on the device - no network connection needed. This means instant results, offline support, and no API costs for inference.

ML Kit Capabilities

Feature Platform Use Case
Text Recognition (OCR) iOS, Android Extract text from images, scan documents, read signs
Barcode Scanning iOS, Android Scan QR codes, UPC codes, and 2D barcodes
Face Detection iOS, Android Detect faces, landmarks, expressions, and contours
Image Labeling iOS, Android Identify objects, places, and activities in images
Object Detection iOS, Android Detect and track objects in real time from camera feed
Custom Models iOS, Android Deploy your own TFLite models via Firebase

Text Recognition Example (Android/Kotlin)

Kotlin
import com.google.mlkit.vision.text.TextRecognition
import com.google.mlkit.vision.text.latin.TextRecognizerOptions
import com.google.mlkit.vision.common.InputImage

// Create a text recognizer
val recognizer = TextRecognition.getClient(TextRecognizerOptions.DEFAULT_OPTIONS)

// Process an image
val image = InputImage.fromBitmap(bitmap, 0)
recognizer.process(image)
    .addOnSuccessListener { visionText ->
        // Extract text from the result
        for (block in visionText.textBlocks) {
            for (line in block.lines) {
                Log.d("OCR", line.text)
            }
        }
    }
    .addOnFailureListener { e ->
        Log.e("OCR", "Text recognition failed", e)
    }

Image Labeling (Web)

For web apps, you can use the ML Kit Web APIs (currently in beta):

TypeScript
// For web, use the Vertex AI Gemini model for vision tasks
import { getVertexAI, getGenerativeModel } from 'firebase/vertexai';

const model = getGenerativeModel(vertexAI, { model: 'gemini-2.0-flash' });

// Label an image using Gemini's vision capabilities
const result = await model.generateContent([
  'List all objects you can identify in this image as a JSON array.',
  { inlineData: { mimeType: 'image/jpeg', data: base64Image } },
]);

const labels = JSON.parse(result.response.text());

Custom TFLite Models

Deploy your own trained models via Firebase:

Kotlin
import com.google.firebase.ml.modeldownloader.FirebaseModelDownloader
import com.google.firebase.ml.modeldownloader.CustomModelDownloadConditions

// Download a custom model from Firebase
val conditions = CustomModelDownloadConditions.Builder()
    .requireWifi()
    .build()

FirebaseModelDownloader.getInstance()
    .getModel("my-custom-model", DownloadType.LOCAL_MODEL, conditions)
    .addOnSuccessListener { model ->
        val modelFile = model.file
        // Use with TFLite Interpreter
    }
ML Kit vs Gemini: Use ML Kit for specialized on-device tasks (OCR, barcode, face detection) that need to work offline and return instant results. Use Gemini for general-purpose AI tasks that require reasoning, conversation, or multimodal understanding.

ML Kit Integrated!

You now have on-device AI capabilities. In the next lesson, learn to build server-side AI with Cloud Functions.

Next: Cloud Functions →

Ready to Go Deeper?

Live instructor-led courses from our partners. Affiliate disclosure.