Google Gemini LLM

GoogleLLMService integrates Google’s Gemini models with streaming responses, function calling, and multimodal input support.

Overview

GoogleLLMService provides integration with Google’s Gemini models, supporting streaming responses, function calling, and multimodal inputs. It includes specialized context handling for Google’s message format while maintaining compatibility with OpenAI-style contexts.

Gemini LLM API Reference

Pipecat’s API methods for Google Gemini integration

Example Implementation

Complete example with function calling

Gemini Documentation

Official Google Gemini API documentation and features

Google AI Studio

Access Gemini models and manage API keys

Installation

To use Google Gemini services, install the required dependencies:

uv add "pipecat-ai[google]"

Prerequisites

Google Gemini Setup

Before using Google Gemini LLM services, you need:

1. Google Account: Sign up at Google AI Studio
2. API Key: Generate a Gemini API key from AI Studio
3. Model Selection: Choose from available Gemini models

Required Environment Variables

• GOOGLE_API_KEY: Your Google Gemini API key for authentication

Configuration

api_key · str · required

Google AI API key for authentication.

model · str · default: "None" · deprecated

Gemini model name to use. Deprecated in v0.0.105. Use settings=GoogleLLMService.Settings(...) instead.

settings · GoogleLLMService.Settings · default: "None"

Runtime-configurable model settings. See Settings below.

params · InputParams · default: "None" · deprecated

Runtime-configurable model settings. See Settings below. Deprecated in v0.0.105. Use settings=GoogleLLMService.Settings(...) instead.

system_instruction · str · default: "None" · deprecated

System instruction/prompt for the model. Sets the overall behavior and context. Deprecated in v0.0.105. Use settings=GoogleLLMService.Settings(system_instruction=...) instead.

tools · List[Dict[str, Any]] · default: "None"

List of available tools/functions for the model to call.

tool_config · Dict[str, Any] · default: "None"

Configuration for tool usage behavior.

http_options · HttpOptions · default: "None"

HTTP options for the Google API client.

stream_idle_timeout_secs · float | None · default: "20.0"

How long to wait for the next chunk of a streamed response before giving up on it. Bounds the wait when the API accepts a request and then stops producing without closing the stream. This is a gap between chunks, not a limit on how long a response may take overall. The first chunk is the slowest, since its wait spans the whole round trip including any thinking the model does before it emits anything; raise this for models configured to think at length. Set to None to wait indefinitely.

retry_timeout_secs · float | None · default: "5.0"

How long to wait for the first chunk before giving up on the request and re-issuing it, when retry_on_timeout is set. Like the first-chunk wait above, this window spans the whole round trip including any thinking, so it is only a good fit for models that start emitting quickly.

retry_on_timeout · bool | None · default: "False"

Whether to re-issue the request once if the first chunk doesn’t arrive within retry_timeout_secs. Only the first chunk is retried: once a chunk has been pushed downstream, re-issuing would duplicate the response.

Settings

Runtime-configurable settings passed via the settings constructor argument using GoogleLLMService.Settings(...). These can be updated mid-conversation with LLMUpdateSettingsFrame. See Service Settings for details.

Parameter	Type	Default	Description
model	str	gemini-3.6-flash	Gemini model identifier. (Inherited from base settings.)
system_instruction	str	None	System instruction/prompt for the model. (Inherited from base settings.)
max_tokens	int	NOT_GIVEN	Maximum number of tokens to generate.
temperature	float	NOT_GIVEN	Sampling temperature (0.0 to 2.0). Lower values are more focused, higher values are more creative.
top_k	int	NOT_GIVEN	Top-k sampling parameter. Limits tokens to the top k most likely.
top_p	float	NOT_GIVEN	Top-p (nucleus) sampling (0.0 to 1.0). Controls diversity of output.
thinking	GoogleThinkingConfig	NOT_GIVEN	Thinking configuration. See GoogleThinkingConfig below.
safety_settings	list[SafetySetting]	NOT_GIVEN	Content safety filters. Each entry pairs a harm category with a blocking threshold.

NOT_GIVEN values are omitted from the API request, letting the Gemini API use its own defaults. If thinking is not provided, Pipecat applies low-latency thinking defaults for Flash models: Gemini 2.5 Flash uses thinking_budget=0 (disables thinking), while Gemini 3+ Flash uses thinking_level="minimal".

GoogleThinkingConfig

Configuration for controlling the model’s internal thinking process. Gemini 2.5 and 3 series models support this feature.

Parameter	Type	Default	Description
thinking_budget	int	None	Token budget for thinking (Gemini 2.5 series). -1 for dynamic, 0 to disable, or a specific count (e.g., 128-32768).
thinking_level	str	None	Thinking level for Gemini 3 models. "low", "high" for 3 Pro; "minimal", "low", "medium", "high" for 3 Flash.
include_thoughts	bool	None	Whether to include thought summaries in the response.

Gemini 2.5 series models use thinking_budget, while Gemini 3 models use thinking_level. Do not mix these parameters across model generations.

Usage

Basic Setup

from pipecat.services.google import GoogleLLMService

llm = GoogleLLMService(
    api_key=os.getenv("GOOGLE_API_KEY"),
    model="gemini-3.5-flash",
)

With Custom Settings

from pipecat.services.google import GoogleLLMService

llm = GoogleLLMService(
    api_key=os.getenv("GOOGLE_API_KEY"),
    settings=GoogleLLMService.Settings(
        model="gemini-2.5-pro",
        system_instruction="You are a helpful assistant.",
        temperature=0.7,
        max_tokens=2048,
        top_p=0.9,
    ),
)

With Thinking Configuration

# Gemini 2.5 series (using thinking_budget)
llm = GoogleLLMService(
    api_key=os.getenv("GOOGLE_API_KEY"),
    settings=GoogleLLMService.Settings(
        model="gemini-2.5-pro",
        max_tokens=8192,
        thinking=GoogleLLMService.GoogleThinkingConfig(
            thinking_budget=4096,
            include_thoughts=True,
        ),
    ),
)

# Gemini 3 series (using thinking_level)
llm = GoogleLLMService(
    api_key=os.getenv("GOOGLE_API_KEY"),
    settings=GoogleLLMService.Settings(
        model="gemini-3.5-flash",
        max_tokens=8192,
        thinking=GoogleLLMService.GoogleThinkingConfig(
            thinking_level="high",
            include_thoughts=True,
        ),
    ),
)

With Safety Settings

from google.genai.types import HarmBlockThreshold, HarmCategory, SafetySetting

llm = GoogleLLMService(
    api_key=os.getenv("GOOGLE_API_KEY"),
    settings=GoogleLLMService.Settings(
        model="gemini-3.5-flash",
        safety_settings=[
            SafetySetting(
                category=HarmCategory.HARM_CATEGORY_HATE_SPEECH,
                threshold=HarmBlockThreshold.BLOCK_LOW_AND_ABOVE,
            ),
            SafetySetting(
                category=HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,
                threshold=HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE,
            ),
        ],
    ),
)

Updating Settings at Runtime

Model settings can be changed mid-conversation using LLMUpdateSettingsFrame:

from pipecat.frames.frames import LLMUpdateSettingsFrame
from pipecat.services.google.llm import GoogleLLMSettings

await worker.queue_frame(
    LLMUpdateSettingsFrame(
        delta=GoogleLLMSettings(
            temperature=0.3,
            max_tokens=1024,
        )
    )
)

# Update safety settings (accepts SafetySetting objects or plain dicts)
await worker.queue_frame(
    LLMUpdateSettingsFrame(
        delta=GoogleLLMSettings(
            safety_settings=[
                {
                    "category": "HARM_CATEGORY_HATE_SPEECH",
                    "threshold": "BLOCK_LOW_AND_ABOVE",
                }
            ]
        )
    )
)

The InputParams / params= pattern is deprecated as of v0.0.105. Use Settings / settings= instead. See the Service Settings guide for migration details.

Notes

• System instruction priority: The system_instruction set via the constructor or GoogleLLMSettings takes priority over any system message in the context. If both are set, a warning is logged and the constructor/settings value is used.
• Thinking defaults: By default, Pipecat applies low-latency thinking defaults for Flash models to reduce latency. Gemini 2.5 Flash uses thinking_budget=0 (disables thinking), while Gemini 3+ Flash uses thinking_level="minimal". To override this behavior, explicitly pass a GoogleThinkingConfig via settings.
• Safety settings: Categories left unspecified in safety_settings keep the Gemini API defaults. The setting accepts SafetySetting objects or plain dicts, and is runtime-updatable via LLMUpdateSettingsFrame.
• Multimodal support: Gemini models natively support image and audio inputs through Google’s Content/Part format. Images and audio are automatically converted from OpenAI-style contexts.
• Grounding with Google Search: When grounding metadata is present in the response (e.g., from Google Search tool), the service emits LLMSearchResponseFrame with search results and source attributions.
• Context format: The service automatically converts between OpenAI-style message formats and Google’s native Content/Part format, so you can use either.

Event Handlers

GoogleLLMService supports the following event handlers, inherited from LLMService:

Event	Description
on_completion_timeout	Called when an LLM completion request times out (Google DeadlineExceeded or stream idle timeout reached)
on_function_calls_started	Called when function calls are received and execution is about to start

@llm.event_handler("on_completion_timeout")
async def on_completion_timeout(service):
    print("LLM completion timed out")