Google Vertex AI LLM

GoogleVertexLLMService extends GoogleLLMService with Vertex AI authentication for running Gemini models on Google Cloud.

Overview

GoogleVertexLLMService provides access to Google’s language models through Vertex AI. It extends GoogleLLMService with Vertex AI authentication, connecting to Google’s enterprise AI services with enhanced security and compliance.

Vertex AI LLM API Reference

Pipecat’s API methods for Google Vertex AI integration

Example Implementation

Browse examples using Vertex AI models

Vertex AI Documentation

Official Google Vertex AI documentation

Google Cloud Console

Access Vertex AI and manage credentials

Installation

To use Google Vertex AI services, install the required dependencies:

uv add "pipecat-ai[google]"

Prerequisites

Google Cloud Setup

Before using Google Vertex AI LLM services, you need:

1. Google Cloud Account: Sign up at Google Cloud Console
2. Project Setup: Create a project and enable the Vertex AI API
3. Service Account: Create a service account with Vertex AI permissions
4. Authentication: Set up credentials via service account key or Application Default Credentials

Required Environment Variables

• GOOGLE_APPLICATION_CREDENTIALS: Path to your service account key file (recommended)
• Or use Application Default Credentials for cloud deployments

Configuration

credentials · str · default: "None"

JSON string of Google service account credentials for authentication.

credentials_path · str · default: "None"

Path to the service account JSON key file. Alternative to providing credentials as a string.

project_id · str · required

Google Cloud project ID.

location · str · default: "us-east4"

GCP region for the Vertex AI endpoint (e.g., "us-east4", "us-central1").

model · str · default: "None" · deprecated

Deprecated in v0.0.105. Use settings=GoogleVertexLLMService.Settings(model=...) instead.

params · GoogleLLMService.InputParams · default: "None" · deprecated

Deprecated in v0.0.105. Use settings=GoogleVertexLLMService.Settings(...) instead.

settings · GoogleVertexLLMService.Settings · default: "None"

Runtime-configurable settings. See Google Gemini Settings for the full parameter reference.

system_instruction · str · default: "None" · deprecated

Deprecated in v0.0.105. Use settings=GoogleVertexLLMService.Settings(system_instruction=...) instead.

tools · list · default: "None"

List of available tools/functions for the model.

tool_config · dict · default: "None"

Configuration for tool usage behavior.

http_options · HttpOptions · default: "None"

HTTP options for the Google AI client.

stream_idle_timeout_secs · float | None · default: "20.0"

How long to wait for the next chunk of a streamed response before giving up on it. Bounds the wait when the API accepts a request and then stops producing without closing the stream. This is a gap between chunks, not a limit on how long a response may take overall. The first chunk is the slowest, since its wait spans the whole round trip including any thinking the model does before it emits anything; raise this for models configured to think at length. Set to None to wait indefinitely. Inherited from GoogleLLMService.

retry_timeout_secs · float | None · default: "5.0"

How long to wait for the first chunk before giving up on the request and re-issuing it, when retry_on_timeout is set. Like the first-chunk wait above, this window spans the whole round trip including any thinking, so it is only a good fit for models that start emitting quickly. Inherited from GoogleLLMService.

retry_on_timeout · bool | None · default: "False"

Whether to re-issue the request once if the first chunk doesn’t arrive within retry_timeout_secs. Only the first chunk is retried: once a chunk has been pushed downstream, re-issuing would duplicate the response. Inherited from GoogleLLMService.

Usage

Basic Setup

import os
from pipecat.services.google.vertex.llm import GoogleVertexLLMService

llm = GoogleVertexLLMService(
    credentials_path=os.getenv("GOOGLE_APPLICATION_CREDENTIALS"),
    project_id="my-gcp-project",
    location="us-east4",
    model="gemini-2.5-flash",
)

With Credentials JSON String

from pipecat.services.google.vertex.llm import GoogleVertexLLMService

llm = GoogleVertexLLMService(
    credentials=os.getenv("GOOGLE_CREDENTIALS_JSON"),
    project_id="my-gcp-project",
    location="us-central1",
    settings=GoogleVertexLLMService.Settings(
        model="gemini-2.5-flash",
        temperature=0.7,
        top_p=0.9,
    ),
)

With Application Default Credentials

from pipecat.services.google.vertex.llm import GoogleVertexLLMService

# Uses ADC when neither credentials nor credentials_path is provided
llm = GoogleVertexLLMService(
    project_id="my-gcp-project",
    location="us-east4",
    model="gemini-2.5-flash",
)

With Safety Settings

from google.genai.types import HarmBlockThreshold, HarmCategory, SafetySetting
from pipecat.services.google.vertex.llm import GoogleVertexLLMService

llm = GoogleVertexLLMService(
    credentials_path=os.getenv("GOOGLE_APPLICATION_CREDENTIALS"),
    project_id="my-gcp-project",
    location="us-east4",
    settings=GoogleVertexLLMService.Settings(
        model="gemini-2.5-flash",
        safety_settings=[
            SafetySetting(
                category=HarmCategory.HARM_CATEGORY_HATE_SPEECH,
                threshold=HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
            ),
        ],
    ),
)

Notes

• This service does not accept an api_key parameter. Use credentials, credentials_path, or Application Default Credentials instead.
• GoogleVertexLLMService extends GoogleLLMService and uses the Google AI Python SDK with Vertex AI authentication.
• Authentication supports three methods: direct JSON credentials string, path to a service account key file, or Application Default Credentials (ADC).
• The project_id parameter is required. If location is not provided, it defaults to "us-east4".

The InputParams / params= pattern is deprecated as of v0.0.105. Use Settings / settings= instead. See the Service Settings guide for migration details.