---
title: "Stream generate content (LiteLLM - Gemini format)"
method: POST
path: "/litellm/genai/v1beta/models/{model}:streamGenerateContent"
tags: ["LiteLLM Integration"]
---

# Stream generate content (LiteLLM - Gemini format)

`POST /litellm/genai/v1beta/models/{model}:streamGenerateContent`

Streams content generation using Google Gemini-compatible format via LiteLLM.

## Path parameters

- `model` string, required

## Request body

- GeminiGenerationRequest
  - `model` string — Model field for explicit model specification
  - `contents` GeminiContent[] — Content for the model to process
    - `role` 'user' | 'model' — The producer of the content. Must be either 'user' or 'model'
    - `parts` GeminiPart[] — List of parts that constitute a single message
      - `text` string — Text part (can be code)
      - `thought` boolean — Indicates if the part is thought from the model
      - `thoughtSignature` string, byte — Opaque signature for thought that can be reused in subsequent requests
      - `inlineData` object
        - `mimeType` string — The IANA standard MIME type of the source data
        - `data` string, byte — Base64-encoded raw bytes
        - `displayName` string — Display name of the blob (not currently used in GenerateContent calls)
      - `fileData` object
        - `mimeType` string — The IANA standard MIME type of the source data
        - `fileUri` string — URI of the file
        - `displayName` string — Display name of the file data
      - `functionCall` object
        - `id` string — Unique ID of the function call. If populated, client should return response with matching id
        - `name` string — The name of the function to call. Matches FunctionDeclaration.name
        - `args` object — Function parameters and values in JSON object format
      - `functionResponse` object
        - `id` string — ID of the function call this response is for. Matches FunctionCall.id
        - `name` string — The name of the function. Matches FunctionDeclaration.name and FunctionCall.name
        - `response` object — Function response in JSON object format. Use "output" key for output and "error" key for error details
        - `willContinue` boolean — Signals that function call continues (NON_BLOCKING only). If false, future responses will not be considered
        - `scheduling` string — How the response should be scheduled (NON_BLOCKING only). Defaults to WHEN_IDLE
      - `executableCode` object
        - `language` string — Programming language of the code
        - `code` string — The code to be executed
      - `codeExecutionResult` object
        - `outcome` 'OUTCOME_UNSPECIFIED' | 'OUTCOME_OK' | 'OUTCOME_FAILED' | 'OUTCOME_DEADLINE_EXCEEDED' — Outcome of the code execution
        - `output` string — Contains stdout when successful, stderr or other description otherwise
      - `videoMetadata` object
        - `fps` number — Frame rate of the video. Range is (0.0, 24.0]
        - `startOffset` string — Start offset of the video
        - `endOffset` string — End offset of the video
  - `systemInstruction` GeminiContent
    - `role` 'user' | 'model' — The producer of the content. Must be either 'user' or 'model'
    - `parts` GeminiPart[] — List of parts that constitute a single message
      - `text` string — Text part (can be code)
      - `thought` boolean — Indicates if the part is thought from the model
      - `thoughtSignature` string, byte — Opaque signature for thought that can be reused in subsequent requests
      - `inlineData` object
        - `mimeType` string — The IANA standard MIME type of the source data
        - `data` string, byte — Base64-encoded raw bytes
        - `displayName` string — Display name of the blob (not currently used in GenerateContent calls)
      - `fileData` object
        - `mimeType` string — The IANA standard MIME type of the source data
        - `fileUri` string — URI of the file
        - `displayName` string — Display name of the file data
      - `functionCall` object
        - `id` string — Unique ID of the function call. If populated, client should return response with matching id
        - `name` string — The name of the function to call. Matches FunctionDeclaration.name
        - `args` object — Function parameters and values in JSON object format
      - `functionResponse` object
        - `id` string — ID of the function call this response is for. Matches FunctionCall.id
        - `name` string — The name of the function. Matches FunctionDeclaration.name and FunctionCall.name
        - `response` object — Function response in JSON object format. Use "output" key for output and "error" key for error details
        - `willContinue` boolean — Signals that function call continues (NON_BLOCKING only). If false, future responses will not be considered
        - `scheduling` string — How the response should be scheduled (NON_BLOCKING only). Defaults to WHEN_IDLE
      - `executableCode` object
        - `language` string — Programming language of the code
        - `code` string — The code to be executed
      - `codeExecutionResult` object
        - `outcome` 'OUTCOME_UNSPECIFIED' | 'OUTCOME_OK' | 'OUTCOME_FAILED' | 'OUTCOME_DEADLINE_EXCEEDED' — Outcome of the code execution
        - `output` string — Contains stdout when successful, stderr or other description otherwise
      - `videoMetadata` object
        - `fps` number — Frame rate of the video. Range is (0.0, 24.0]
        - `startOffset` string — Start offset of the video
        - `endOffset` string — End offset of the video
  - `generationConfig` object
    - `temperature` number — Controls the randomness of predictions
    - `topP` number — Nucleus sampling parameter
    - `topK` integer — Top-k sampling parameter
    - `candidateCount` integer — Number of candidates to generate. Defaults to 1
    - `maxOutputTokens` integer — Maximum number of output tokens to generate per message
    - `stopSequences` string[] — Stop sequences
    - `responseMimeType` string — Output response mimetype (text/plain, application/json)
    - `responseSchema` GeminiSchema — Schema object for defining input/output data types (OpenAPI 3.0 subset)
      - `type` 'TYPE_UNSPECIFIED' | 'STRING' | 'NUMBER' | 'INTEGER' | 'BOOLEAN' | 'ARRAY' | 'OBJECT' | 'NULL' — The type of the data
      - `format` string — Format of the data (e.g., float, double, int32, int64, email, byte)
      - `title` string — The title of the Schema
      - `description` string — The description of the data
      - `nullable` boolean — Indicates if the value may be null
      - `enum` string[] — Possible values for primitive types with enum format
      - `properties` object — Properties of Type.OBJECT
      - `required` string[] — Required properties of Type.OBJECT
      - `items` GeminiSchema — recursive
      - `minItems` integer — Minimum number of elements for Type.ARRAY
      - `maxItems` integer — Maximum number of elements for Type.ARRAY
      - `minLength` integer — Minimum length of Type.STRING
      - `maxLength` integer — Maximum length of Type.STRING
      - `minimum` number — Minimum value of Type.INTEGER and Type.NUMBER
      - `maximum` number — Maximum value of Type.INTEGER and Type.NUMBER
      - `pattern` string — Pattern to restrict a string to a regular expression
      - `default` unknown
      - `example` unknown
      - `anyOf` GeminiSchema[] — Value should be validated against any of the subschemas
      - `propertyOrdering` string[] — Order of the properties (not standard OpenAPI)
      - `minProperties` integer — Minimum number of properties for Type.OBJECT
      - `maxProperties` integer — Maximum number of properties for Type.OBJECT
    - `responseJsonSchema` object — Alternative to responseSchema using JSON Schema format
    - `responseModalities` string[] — The modalities of the response
    - `speechConfig` object
      - `voiceConfig` object
        - `prebuiltVoiceConfig` object
          - `voiceName` string — The name of the prebuilt voice to use
      - `multiSpeakerVoiceConfig` object
        - `speakerVoiceConfigs` object[]
          - `speaker` string — The name of the speaker (should match prompt)
          - `voiceConfig` object
            - `prebuiltVoiceConfig` object
              - …
      - `languageCode` string — Language code (ISO 639) for speech synthesis. Only available for Live API
    - `thinkingConfig` object
      - `includeThoughts` boolean — Whether to include thoughts in the response
      - `thinkingBudget` integer — Thinking budget in tokens
      - `thinkingLevel` 'THINKING_LEVEL_UNSPECIFIED' | 'LOW' | 'HIGH' — Thinking level preset
    - `frequencyPenalty` number — Frequency penalty for token generation
    - `presencePenalty` number — Presence penalty for token generation
    - `seed` integer — Seed for deterministic generation
    - `logprobs` integer — Number of log probabilities to return
    - `responseLogprobs` boolean — If true, export logprobs results in response
    - `audioTimestamp` boolean — If enabled, audio timestamp will be included in request
    - `mediaResolution` string — Media resolution specification
    - `routingConfig` object
      - `autoMode` object
        - `modelRoutingPreference` string — Model routing preference
      - `manualMode` object
        - `modelName` string — Model name to use
    - `modelSelectionConfig` object
      - `featureSelectionPreference` string — Options for feature selection preference
    - `enableAffectiveDialog` boolean — If enabled, model will detect emotions and adapt responses
  - `safetySettings` object[]
    - `category` string — Harm category
    - `threshold` string — The harm block threshold
    - `method` string — Determines if harm block uses probability or probability and severity scores
  - `tools` object[]
    - `functionDeclarations` object[]
      - `name` string — Function name. Must start with letter/underscore, a-z, A-Z, 0-9, underscores, dots, dashes. Max 64 chars
      - `description` string — Description and purpose of the function
      - `parameters` GeminiSchema — Schema object for defining input/output data types (OpenAPI 3.0 subset)
        - `type` 'TYPE_UNSPECIFIED' | 'STRING' | 'NUMBER' | 'INTEGER' | 'BOOLEAN' | 'ARRAY' | 'OBJECT' | 'NULL' — The type of the data
        - `format` string — Format of the data (e.g., float, double, int32, int64, email, byte)
        - `title` string — The title of the Schema
        - `description` string — The description of the data
        - `nullable` boolean — Indicates if the value may be null
        - `enum` string[] — Possible values for primitive types with enum format
        - `properties` object — Properties of Type.OBJECT
        - `required` string[] — Required properties of Type.OBJECT
        - `items` GeminiSchema — recursive
        - `minItems` integer — Minimum number of elements for Type.ARRAY
        - `maxItems` integer — Maximum number of elements for Type.ARRAY
        - `minLength` integer — Minimum length of Type.STRING
        - `maxLength` integer — Maximum length of Type.STRING
        - `minimum` number — Minimum value of Type.INTEGER and Type.NUMBER
        - `maximum` number — Maximum value of Type.INTEGER and Type.NUMBER
        - `pattern` string — Pattern to restrict a string to a regular expression
        - `default` unknown
        - `example` unknown
        - `anyOf` GeminiSchema[] — Value should be validated against any of the subschemas
        - `propertyOrdering` string[] — Order of the properties (not standard OpenAPI)
        - `minProperties` integer — Minimum number of properties for Type.OBJECT
        - `maxProperties` integer — Maximum number of properties for Type.OBJECT
      - `parametersJsonSchema` object — Alternative to parameters using JSON Schema format
      - `response` GeminiSchema — Schema object for defining input/output data types (OpenAPI 3.0 subset)
        - `type` 'TYPE_UNSPECIFIED' | 'STRING' | 'NUMBER' | 'INTEGER' | 'BOOLEAN' | 'ARRAY' | 'OBJECT' | 'NULL' — The type of the data
        - `format` string — Format of the data (e.g., float, double, int32, int64, email, byte)
        - `title` string — The title of the Schema
        - `description` string — The description of the data
        - `nullable` boolean — Indicates if the value may be null
        - `enum` string[] — Possible values for primitive types with enum format
        - `properties` object — Properties of Type.OBJECT
        - `required` string[] — Required properties of Type.OBJECT
        - `items` GeminiSchema — recursive
        - `minItems` integer — Minimum number of elements for Type.ARRAY
        - `maxItems` integer — Maximum number of elements for Type.ARRAY
        - `minLength` integer — Minimum length of Type.STRING
        - `maxLength` integer — Maximum length of Type.STRING
        - `minimum` number — Minimum value of Type.INTEGER and Type.NUMBER
        - `maximum` number — Maximum value of Type.INTEGER and Type.NUMBER
        - `pattern` string — Pattern to restrict a string to a regular expression
        - `default` unknown
        - `example` unknown
        - `anyOf` GeminiSchema[] — Value should be validated against any of the subschemas
        - `propertyOrdering` string[] — Order of the properties (not standard OpenAPI)
        - `minProperties` integer — Minimum number of properties for Type.OBJECT
        - `maxProperties` integer — Maximum number of properties for Type.OBJECT
      - `responseJsonSchema` object — Alternative to response using JSON Schema format
      - `behavior` 'UNSPECIFIED' | 'BLOCKING' | 'NON_BLOCKING' — Function behavior mode. BLOCKING waits for response, NON_BLOCKING continues conversation
    - `googleSearch` object
      - `timeRangeFilter` object
        - `startTime` string, date-time
        - `endTime` string, date-time
      - `excludeDomains` string[] — List of domains to exclude from search results (max 2000)
    - `googleSearchRetrieval` object
      - `dynamicRetrievalConfig` object
        - `mode` string — The mode of the predictor for dynamic retrieval
        - `dynamicThreshold` number — Threshold for dynamic retrieval
    - `retrieval` object
      - `disableAttribution` boolean — Deprecated. This option is no longer supported
      - `externalApi` object
        - `endpoint` string — The endpoint of the external API
        - `apiSpec` 'API_SPEC_UNSPECIFIED' | 'SIMPLE_SEARCH' | 'ELASTIC_SEARCH'
        - `authConfig` object
          - `authType` 'AUTH_TYPE_UNSPECIFIED' | 'NO_AUTH' | 'API_KEY_AUTH' | 'HTTP_BASIC_AUTH' | 'GOOGLE_SERVICE_ACCOUNT_AUTH' | 'OAUTH' | 'OIDC_AUTH'
          - `apiKeyConfig` object
            - `apiKeyString` string
          - `googleServiceAccountConfig` object
            - `serviceAccount` string
          - `httpBasicAuthConfig` object
            - `credentialSecret` string
          - `oauthConfig` object
            - `accessToken` string
            - `serviceAccount` string
          - `oidcConfig` object
            - `idToken` string
            - `serviceAccount` string
        - `elasticSearchParams` object
          - `index` string — The ElasticSearch index to use
          - `numHits` integer — Number of hits (chunks) to request
          - `searchTemplate` string — The ElasticSearch search template to use
      - `vertexAiSearch` object
        - `datastore` string — Fully-qualified Vertex AI Search data store resource ID
        - `engine` string — Fully-qualified Vertex AI Search engine resource ID
        - `filter` string — Filter strings to be passed to the search API
        - `maxResults` integer — Number of search results to return (max 10, default 10)
        - `dataStoreSpecs` object[]
          - `dataStore` string — Full resource name of DataStore
          - `filter` string — Filter specification for documents in the data store
      - `vertexRagStore` object
        - `ragCorpora` string[] — Deprecated. Use ragResources instead
        - `ragResources` object[]
          - `ragCorpus` string — RAGCorpora resource name
          - `ragFileIds` string[] — rag_file_id. Files should be in the same rag_corpus
        - `ragRetrievalConfig` object
          - `topK` integer — The number of contexts to retrieve
          - `filter` object
            - `metadataFilter` string — String for metadata filtering
            - `vectorDistanceThreshold` number — Only returns contexts with vector distance smaller than threshold
            - `vectorSimilarityThreshold` number — Only returns contexts with vector similarity larger than threshold
          - `hybridSearch` object
            - `alpha` number — Weight between dense and sparse vector search (0-1). 0 = sparse only, 1 = dense only
          - `ranking` object
            - `llmRanker` object
              - …
            - `rankService` object
              - …
        - `similarityTopK` integer — Number of top k results to return from selected corpora
        - `storeContext` boolean — For Gemini Multimodal Live API - memorize interactions
        - `vectorDistanceThreshold` number — Only return results with vector distance smaller than threshold
    - `codeExecution` object — Enables code execution by the model
    - `enterpriseWebSearch` object
      - `excludeDomains` string[] — List of domains to exclude (max 2000)
    - `googleMaps` object
      - `authConfig` object
        - `authType` 'AUTH_TYPE_UNSPECIFIED' | 'NO_AUTH' | 'API_KEY_AUTH' | 'HTTP_BASIC_AUTH' | 'GOOGLE_SERVICE_ACCOUNT_AUTH' | 'OAUTH' | 'OIDC_AUTH'
        - `apiKeyConfig` object
          - `apiKeyString` string
        - `googleServiceAccountConfig` object
          - `serviceAccount` string
        - `httpBasicAuthConfig` object
          - `credentialSecret` string
        - `oauthConfig` object
          - `accessToken` string
          - `serviceAccount` string
        - `oidcConfig` object
          - `idToken` string
          - `serviceAccount` string
    - `urlContext` object — Tool to support URL context retrieval
    - `computerUse` object
      - `environment` 'ENVIRONMENT_UNSPECIFIED' | 'ENVIRONMENT_BROWSER' — The environment being operated
  - `toolConfig` object
    - `functionCallingConfig` object
      - `mode` 'MODE_UNSPECIFIED' | 'AUTO' | 'ANY' | 'NONE' | 'VALIDATED' — Function calling mode
      - `allowedFunctionNames` string[] — Function names to call when mode is ANY
    - `retrievalConfig` object
      - `latLng` object
        - `latitude` number — Latitude in degrees [-90.0, +90.0]
        - `longitude` number — Longitude in degrees [-180.0, +180.0]
      - `languageCode` string
  - `cachedContent` string — Cached content resource name
  - `labels` object — Labels for the request
  - `requests` GeminiEmbeddingRequest[] — Batch embedding requests
    - `model` string
    - `content` GeminiContent
      - `role` 'user' | 'model' — The producer of the content. Must be either 'user' or 'model'
      - `parts` GeminiPart[] — List of parts that constitute a single message
        - `text` string — Text part (can be code)
        - `thought` boolean — Indicates if the part is thought from the model
        - `thoughtSignature` string, byte — Opaque signature for thought that can be reused in subsequent requests
        - `inlineData` object
          - `mimeType` string — The IANA standard MIME type of the source data
          - `data` string, byte — Base64-encoded raw bytes
          - `displayName` string — Display name of the blob (not currently used in GenerateContent calls)
        - `fileData` object
          - `mimeType` string — The IANA standard MIME type of the source data
          - `fileUri` string — URI of the file
          - `displayName` string — Display name of the file data
        - `functionCall` object
          - `id` string — Unique ID of the function call. If populated, client should return response with matching id
          - `name` string — The name of the function to call. Matches FunctionDeclaration.name
          - `args` object — Function parameters and values in JSON object format
        - `functionResponse` object
          - `id` string — ID of the function call this response is for. Matches FunctionCall.id
          - `name` string — The name of the function. Matches FunctionDeclaration.name and FunctionCall.name
          - `response` object — Function response in JSON object format. Use "output" key for output and "error" key for error details
          - `willContinue` boolean — Signals that function call continues (NON_BLOCKING only). If false, future responses will not be considered
          - `scheduling` string — How the response should be scheduled (NON_BLOCKING only). Defaults to WHEN_IDLE
        - `executableCode` object
          - `language` string — Programming language of the code
          - `code` string — The code to be executed
        - `codeExecutionResult` object
          - `outcome` 'OUTCOME_UNSPECIFIED' | 'OUTCOME_OK' | 'OUTCOME_FAILED' | 'OUTCOME_DEADLINE_EXCEEDED' — Outcome of the code execution
          - `output` string — Contains stdout when successful, stderr or other description otherwise
        - `videoMetadata` object
          - `fps` number — Frame rate of the video. Range is (0.0, 24.0]
          - `startOffset` string — Start offset of the video
          - `endOffset` string — End offset of the video
    - `taskType` string
    - `title` string
    - `outputDimensionality` integer
  - `fallbacks` string[]

## Response `200`

Successful streaming response

## Other responses

- `400` — Bad request
- `500` — Internal server error

---

[API](https://skmtc.net/getbifrost/apis/bifrost-api.md) · [All operations](https://skmtc.net/getbifrost/apis/bifrost-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/getbifrost/bifrost-api/versions/f8cab88f64ea/schema)