Update Speaker Face Match

Manually match detected faces to identified speakers with bidirectional name synchronization.

This endpoint enables client-driven multimodal person identification by allowing you to manually specify which detected faces (from video analysis) correspond to which identified speakers (from audio analysis). Each mapping synchronizes identity information across video and audio modalities while managing personality analysis data.

Features

• Face-to-Speaker Mapping: Links face IDs (from video) to speaker IDs (from audio)
• Bidirectional Name Sync: Automatically synchronizes names between faces and speakers
• Duplicate Prevention: Validates that each face and speaker is mapped only once
• Personality Data Cleanup: Clears existing personality analysis when mappings change
• Batch Operations: Process multiple mappings in a single request

Name Synchronization

The name field in each mapping supports bidirectional synchronization:

• Empty/null name ("" or null): Fetches the speaker's name from the audio document and applies it to the face
• Provided name (e.g., "Alice"): Updates both the face name AND the speaker name in their respective documents

This ensures consistency across audio and video data while giving you control over whether to use existing speaker names or provide new ones.

Request Body

{
  "mappings": [
    {
      "face_id": 1,
      "speaker_id": 0,
      "name": "Alice Johnson"
    },
    {
      "face_id": 2,
      "speaker_id": 1,
      "name": ""
    }
  ]
}

Each mapping object contains:

• face_id (int, required): ID of detected face from video analysis
• speaker_id (int, required): ID of identified speaker from audio analysis
• name (string, optional): Person's name for synchronization

Response

Returns a confirmation response:

{
  "status": "completed",
  "message": "Speaker-face matching completed successfully"
}

Prerequisites

• Video must be in "completed" status
• Video must have associated audio document (multimodal analysis)
• Face IDs must exist in the video's face detections
• Speaker IDs must exist in the audio's speaker identifications

Use Cases

1. Manual Initial Matching: Manually map speakers to faces after video processing completes
2. Name Updates: Update names while preserving analysis data
3. Correction of Auto-Matching: Override or correct automatic face-to-speaker associations
4. Custom Mappings: Specify mappings based on client-side knowledge or user input
5. Prepare for Personality Analysis: Define mappings before triggering personality analysis
6. Batch Processing: Map multiple face-speaker pairs in a single API call

Validation Rules

• ✅ Each face_id must appear only once in the request (no duplicate faces)
• ✅ Each speaker_id must appear only once in the request (no duplicate speakers)
• ✅ All face_id values must exist in the video document
• ✅ All speaker_id values must exist in the audio document
• ✅ User must own the video (admin users can access any video)

Important Notes

• Manual Operation: This endpoint requires the client to explicitly provide all face-to-speaker mappings. For automatic AI-driven matching, use the POST endpoint.
• Personality Data: When mappings are created or updated, any existing personality analysis data for those faces is cleared and must be re-analyzed
• Audio Document Updates: If you provide names, the audio document's speaker names are automatically updated to match
• Idempotent: Calling this endpoint multiple times with the same mappings will update the entities (not create duplicates)

Example Scenarios

Scenario 1: Create New Mappings

PUT /v2/videos/abc123/speaker-face-match
{
  "mappings": [
    {"face_id": 1, "speaker_id": 0, "name": "Alice"},
    {"face_id": 2, "speaker_id": 1, "name": "Bob"}
  ]
}

Scenario 2: Use Existing Speaker Names

PUT /v2/videos/abc123/speaker-face-match
{
  "mappings": [
    {"face_id": 1, "speaker_id": 0, "name": ""},
    {"face_id": 2, "speaker_id": 1, "name": ""}
  ]
}

Names will be fetched from the audio document's speaker records.

Scenario 3: Update Names

PUT /v2/videos/abc123/speaker-face-match
{
  "mappings": [
    {"face_id": 1, "speaker_id": 0, "name": "Alice Johnson"}
  ]
}

Updates both the face name and speaker name to "Alice Johnson".

Error Responses

• 400 Bad Request: Empty mappings, duplicate face/speaker IDs, or missing audio ID
• 401 Unauthorized: Missing authentication
• 403 Forbidden: User doesn't own the video
• 404 Not Found: Face ID or speaker ID not found in respective documents
• 422 Unprocessable Entity: Video processing not completed