v21

latestOpenAPI 3.1.0raw.githubusercontent.com2025-05-22122299.2 KB

Token Chunker

Splits text into chunks based on token count, ensuring each chunk stays within specified token limits.

post/v1/chunk/token

Successful Response: A list of Chunk objects.

textstring

The actual text content of the chunk.

start_indexinteger

The starting character index of the chunk within the original input text.

end_indexinteger

The ending character index (exclusive) of the chunk within the original input text.

token_countinteger

The number of tokens in this specific chunk, according to the tokenizer used.

contextstring nullable

Chunk context.