Vertex AI Extension Intermediate

The Firebase Vertex AI SDK lets you call Gemini models directly from your client app - web, iOS, or Android. Firebase handles authentication, rate limiting, and API key management, so you can focus on building AI features.

Text Generation

TypeScript
import { getVertexAI, getGenerativeModel } from 'firebase/vertexai';

const vertexAI = getVertexAI(app);
const model = getGenerativeModel(vertexAI, { model: 'gemini-2.0-flash' });

// Simple text generation
const result = await model.generateContent('Explain quantum computing in simple terms');
console.log(result.response.text());

Multimodal Inputs

Gemini supports text, images, video, and audio inputs:

TypeScript
// Analyze an image
const result = await model.generateContent([
  'What do you see in this image?',
  {
    inlineData: {
      mimeType: 'image/jpeg',
      data: base64ImageData,  // Base64-encoded image
    },
  },
]);

console.log(result.response.text());
// "The image shows a golden retriever playing in a park..."

Chat Conversations

TypeScript
// Start a chat session
const chat = model.startChat({
  history: [
    { role: 'user', parts: [{ text: 'You are a helpful cooking assistant.' }] },
    { role: 'model', parts: [{ text: 'I would be happy to help with cooking!' }] },
  ],
});

// Send messages in the conversation
const result1 = await chat.sendMessage('How do I make pasta carbonara?');
console.log(result1.response.text());

// Follow-up (context is maintained)
const result2 = await chat.sendMessage('Can I use bacon instead of guanciale?');
console.log(result2.response.text());

Streaming Responses

TypeScript
// Stream the response token by token
const result = await model.generateContentStream('Write a short story about AI');

for await (const chunk of result.stream) {
  const text = chunk.text();
  process.stdout.write(text); // Display progressively
}

// Get the final aggregated response
const finalResponse = await result.response;
console.log('\n\nFull response:', finalResponse.text());

Safety Settings

TypeScript
import { HarmCategory, HarmBlockThreshold } from 'firebase/vertexai';

const model = getGenerativeModel(vertexAI, {
  model: 'gemini-2.0-flash',
  safetySettings: [
    {
      category: HarmCategory.HARM_CATEGORY_HARASSMENT,
      threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
    },
    {
      category: HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,
      threshold: HarmBlockThreshold.BLOCK_ONLY_HIGH,
    },
  ],
});
App Check: Enable Firebase App Check to protect your Vertex AI endpoints from abuse. App Check verifies that requests come from your genuine app, not from unauthorized clients.

Vertex AI Integrated!

You can now call Gemini from your client app. In the next lesson, learn about ML Kit for on-device AI.

Next: ML Kit →

Ready to Go Deeper?

Live instructor-led courses from our partners. Affiliate disclosure.