A list of multimodal inputs to be vectorized.<br> <br> A single input in the list is a dictionary containing a single key "content", whose value represents a sequence of text, images, and videos. <ul>
<li> The value of <code>"content"</code> is a list of dictionaries, each representing a single piece of text or image. The dictionaries have four possible keys:
<ol class="nested-ordered-list">
<li> <b>type</b>: Specifies the type of the piece of the content. Allowed values are <code>text</code>, <code>image_url</code>, <code>image_base64</code>, <code>video_url</code>, or <code>video_base64</code>.</li>
<li> <b>text</b>: Only present when <code>type</code> is <code>text</code>. The value should be a text string.</li>
<li> <b>image_base64</b>: Only present when <code>type</code> is <code>image_base64</code>. The value should be a Base64-encoded image in the <a href="https://developer.mozilla.org/en-US/docs/Web/URI/Schemes/data" target="_blank">data URL</a> format <code>data:[<mediatype>];base64,<data></code>. Currently supported <code>mediatypes</code> are: <code>image/png</code>, <code>image/jpeg</code>, <code>image/webp</code>, and <code>image/gif</code>.</li>
<li> <b>image_url</b>: Only present when <code>type</code> is <code>image_url</code>. The value should be a URL linking to the image. We support PNG, JPEG, WEBP, and GIF images. The following constraints apply to the URL:
<ul>
<li> Limit the number of redirects. </li>
<li> Require that responses include a content-length header. </li>
<li> Respect robots.txt to prevent unauthorized scraping. </li>
</ul>
</li>
<li> <b>video_base64</b>: Only present when <code>type</code> is <code>video_base64</code>. The value should be a Base64-encoded video in the <a href="https://developer.mozilla.org/en-US/docs/Web/URI/Schemes/data" target="_blank">data URL</a> format <code>data:[<mediatype>];base64,<data></code>. Currently supported <code>mediatypes</code> are: <code>video/mp4</code>.</li>
<li> <b>video_url</b>: Only present when <code>type</code> is <code>video_url</code>. The value should be a URL linking to the video. We support MP4 videos. The following constraints apply to the URL:
<ul>
<li> Limit the number of redirects. </li>
<li> Require that responses include a content-length header. </li>
<li> Respect robots.txt to prevent unauthorized scraping. </li>
</ul>
</li>
</ol>
</li>
<li> <b>Note</b>: Only one of the keys, <code>base64</code> or <code>url</code>, should be present in each dictionary for image and video data. Consistency is required within a request, meaning each request should use either <code>image_base64</code>/<code>video_base64</code> or <code>image_url</code>/<code>video_url</code> exclusively, not both.<br>
<br>
<details> <summary> Example payload where <code>inputs</code> contains an image as a URL </summary>
<br>
The <code>inputs</code> list contains a single input, which consists of a piece of text and an image (which is provided via a URL).
<pre><code>
{
"inputs": [
{
"content": [
{
"type": "text",
"text": "This is a banana."
},
{
"type": "image_url",
"image_url": "https://raw.githubusercontent.com/voyage-ai/voyage-multimodal-3/refs/heads/main/images/banana.jpg"
},
{
"type": "video_url",
"video_url": "https://test-videos.co.uk/vids/bigbuckbunny/mp4/h264/360/Big_Buck_Bunny_360_10s_1MB.mp4"
}
]
}
],
"model": "voyage-multimodal-3.5"
}
</code></pre>
</details>
<details> <summary> Example payload where <code>inputs</code> contains a Base64 image </summary>
<br>
Below is an equivalent example to the one above where the image content is a Base64 image instead of a URL. (Base64 images can be lengthy, so the example only shows a shortened version.)
<pre><code>
{
"inputs": [
{
"content": [
{
"type": "text",
"text": "This is a banana."
},
{
"type": "image_base64",
"image_base64": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAA..."
},
{
"type": "video_base64",
"video_base64": "data:video/mp4;base64,AAAAIGZ0eXBpc29tAAACA..."
}
]
}
],
"model": "voyage-multimodal-3.5"
}
</code></pre>
</details>
</li>
</ul>
<span style="font-size: 13px;">The following constraints apply to the <code>inputs</code> list:</span> <ul>
<li> The list must not contain more than 1,000 inputs. </li>
<li> Each image must not contain more than 16 million pixels or be larger than 20 MB in size. </li>
<li> Each video must not be larger than 20 MB in size. </li>
<li> With every 560 pixels of an image and every 1120 pixels of a video being counted as a token, each input in the list must not exceed 32,000 tokens, and the total number of tokens across all inputs must not exceed 320,000. </li>
</ul>