---
title: "Sample"
method: POST
path: "/rl/training-sessions/{session_id}/operations/sample"
tags: ["RL"]
---

# Sample

`POST /rl/training-sessions/{session_id}/operations/sample`

Submits a sample operation that will asynchronously generate text completions with logprobs.

## Path parameters

- `session_id` string, required — Training session ID

## Request body

- RLSampleBody
  - `model_inputs` RLModelInput[], required — Model inputs to sample from
    - `chunks` RLModelInputChunk[], required — Input chunks for the model
      - `encoded_text` RLEncodedTextChunk, required — Pre-tokenized text content for a model input chunk.
        - `tokens` union[], required — Pre-tokenized text input
          - union
            - string
            - integer
  - `sampling_params` RLSamplingParams
    - `max_tokens` integer — Maximum number of tokens to generate per completion
    - `temperature` number — Sampling temperature
    - `top_p` number — Nucleus sampling probability threshold
    - `top_k` integer — Top-k sampling limit
    - `stop` string[] — Generation stops when any of these strings is produced
    - `seed` union — Random seed for reproducible sampling for the same prompt and model state. Per-completion seeds remain stable if the request is split across generator replicas.
      - string
      - integer
  - `num_samples` integer — Number of completions to generate per prompt
  - `prompt_logprobs` boolean — When true, also compute teacher-forced log-probabilities for the model input tokens and return them in `SampleResult.prompt_logprobs`.
  - `topk_prompt_logprobs` integer — Number of most likely alternative tokens to return per model input token in `SampleResult.topk_prompt_logprobs`. 0 disables top-k prompt log-probabilities. Maximum 20.
  - `return_routed_experts` boolean — When true, capture the mixture-of-experts routing decisions made while generating and return them in `SampledSequence.routed_experts`, so training can reuse the same expert selection. Only available on mixture-of-experts models; ignored otherwise. The captured buffer scales with sequence length, so leave it off unless you replay routing during training.

## Response `200`

- RLSampleOperation — Async sample operation
  - `id` string, required — Operation ID
  - `status` 'TRAINING_OPERATION_STATUS_UNSPECIFIED' | 'TRAINING_OPERATION_STATUS_PENDING' | 'TRAINING_OPERATION_STATUS_RUNNING' | 'TRAINING_OPERATION_STATUS_COMPLETED' | 'TRAINING_OPERATION_STATUS_FAILED', required
  - `output` RLSampleBatchResult — Batched result of a sample operation
    - `results` RLSampleResult[], required — One result per model input
      - `sequences` RLSampledSequence[], required — Generated completions
        - `tokens` union[], required — Generated token IDs
          - union
            - string
            - integer
        - `logprobs` number[] — Log probabilities for each generated token
        - `stop_reason` 'STOP_REASON_LENGTH' | 'STOP_REASON_STOP', required — Reason generation stopped.
        - `prompt_cache_hit_tokens` integer, required — Number of model input tokens served from the prefix cache while generating this sequence.
        - `routed_experts` RLRoutedExperts — Mixture-of-experts routing decisions captured while generating, so training can reuse the same expert selection. A contiguous uint16 buffer of selected expert indices, reshaped by `shape`, which is always `[num_tokens, num_layers, topk]`.
          - `data` string, byte, required — Base64-encoded contiguous uint16 buffer of selected expert indices, row-major over (token, layer, k).
          - `shape` union[], required — Buffer shape as `[num_tokens, num_layers, topk]`.
            - union
              - …
      - `prompt_logprobs` number[] — Teacher-forced log-probability of each model input token. Full prompt length; entry i corresponds to prompt token i. Entry 0 is always 0 as a placeholder: the first prompt token has no conditioning context, so it has no log-probability. Present only when prompt_logprobs was set on the request.
      - `topk_prompt_logprobs` RLPromptTopLogprobs[] — The most likely alternative tokens at each model input token, up to `topk_prompt_logprobs` per position. Full prompt length; entry i corresponds to prompt token i, and entry 0 is empty. Present only when topk_prompt_logprobs was set on the request.
        - `token_ids` integer[] — Token IDs of the alternatives, ordered by descending log-probability.
        - `logprobs` number[] — Log-probability of each alternative in `token_ids`, at the same index.
      - `policy_segments` RLPolicyVersionSegment[], required — Policy versions that produced these completions
        - `version` integer, required — Model version under which this segment of tokens was generated
        - `start_token` integer, required — Index of the first token of this segment within the sampled sequence. Always 0 for the first segment.
  - `error` RLTrainingOperationError — Error details for a failed training operation
    - `code` 'TRAINING_OPERATION_ERROR_CODE_UNSPECIFIED' | 'TRAINING_OPERATION_ERROR_CODE_RESOURCE_EXHAUSTED' | 'TRAINING_OPERATION_ERROR_CODE_TIMEOUT' | 'TRAINING_OPERATION_ERROR_CODE_INTERNAL_ERROR' | 'TRAINING_OPERATION_ERROR_CODE_SESSION_NOT_ACTIVE' | 'TRAINING_OPERATION_ERROR_CODE_INVALID_INPUT' | 'TRAINING_OPERATION_ERROR_CODE_NON_FINITE_LOSS' — Application error code for a failed training operation
    - `message` string — Human-readable error message

## Other responses

- `default` — An unexpected error response.

---

[API](https://skmtc.net/together/apis/together-apis.md) · [All operations](https://skmtc.net/together/apis/together-apis/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/together/together-apis/revisions/1fab430e87d6/schema)