---
title: "Unified Contextual Memory Search (Text + Multimodal)"
method: POST
path: "/v1/modal/query"
tags: ["v1 modals"]
---

# Unified Contextual Memory Search (Text + Multimodal)

`POST /v1/modal/query`

Retrieves relevant memories based on text conversation context and/or multimodal inputs.
    
    You can provide **text messages**, **images**, **video**, **audio**, or any combination.
    The system finds memories semantically relevant to the provided inputs.
    
    **Universal Base Params:**
    - user_id (str, required): Restrict search to this user
    - project_id (str, required): Restrict search to this project
    - persona_id (str, optional): Use persona's context if provided
    
    **Input — at least one required:**
    - messages (array, optional): Text conversation context
    - video_base64 (str, optional): Base64 encoded video
    - image_base64 (str, optional): Base64 encoded image
    - audio_base64 (str, optional): Base64 encoded audio
    
    **Search Config:**
    - include_modalities (array, optional): Filter results by type: ["text", "image", "video"]

    **Response:**
    - new_prompt (str): Enhanced prompt with retrieved memory context
    - raw_results (dict): Structured memory data from retrieval
    - entity_images (dict, optional): Reference images for matched entities
    - success (bool): True if query succeeded

    Returns 200 OK with memory data. Requires authentication.

## Request body

- QueryRequest — Unified request model for contextual memory search (text + multimodal)
  - `user_id` string, required — Restrict search to this user (required)
  - `project_id` string, nullable — Restrict search to this project (required)
  - `persona_id` string, nullable — Optional persona ID. If provided, query uses persona's context
  - `text_input` string, nullable — Text input to search against. The system finds memories *relevant* to this text.
  - `video_base64` string, nullable — Base64 encoded video content
  - `image_base64` string, nullable — Base64 encoded image content
  - `audio_base64` string, nullable — Base64 encoded audio content (supports webm, wav, mp3, mp4, and other formats)
  - `include_modalities` string[], nullable — Filter results by type: ['text', 'image', 'video']
  - `session_id` string, nullable — Optional session identifier for conversation context

## Response `200`

Successful Response

- QueryResponse — Unified response model for memory query (text + multimodal)
  - `new_prompt` string, required — Enhanced prompt with retrieved memory context
  - `raw_results` object — Raw results from the retrieval process
  - `entity_images` object, nullable — Reference images for matched entities (entity_name -> base64 image)
  - `success` boolean — Whether the query was processed successfully

## Other responses

- `400` — Bad Request
- `401` — Unauthorized
- `404` — User/Persona/Project Not Found
- `422` — Validation Error
- `500` — Internal Server Error

---

[API](https://skmtc.net/elicitlabs/apis/elicit-labs-api.md) · [All operations](https://skmtc.net/elicitlabs/apis/elicit-labs-api/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/elicitlabs/elicit-labs-api/revisions/27701839a070/schema)