--- title: "Forward-backward pass" method: POST path: "/rl/training-sessions/{session_id}/operations/forward-backward" tags: ["RL"] --- # Forward-backward pass `POST /rl/training-sessions/{session_id}/operations/forward-backward` Submits a forward-backward pass operation that will asynchronously compute gradients via backpropagation. ## Path parameters - `session_id` string, required — Training session ID ## Request body - RLForwardBackwardBody - `samples` RLTrainingSample[], required — Batch of training samples to process - `model_input` RLModelInput, required - `chunks` RLModelInputChunk[], required — Input chunks for the model - `encoded_text` RLEncodedTextChunk, required — Pre-tokenized text content for a model input chunk. - `tokens` union[], required — Pre-tokenized text input - union - … - `loss_inputs` RLLossInputs, required — Token-level inputs used to compute the loss for one training sample. - `target_tokens` RLLossTargetTokens, required - `data` union[], required — Integer array of target tokens - union - string - integer - `dtype` 'D_TYPE_UNSPECIFIED' | 'D_TYPE_INT64' | 'D_TYPE_FLOAT32' | 'D_TYPE_BFLOAT16' - `weights` RLWeights - `data` number[], required — Per-token loss weights, one non-negative weight per target token. A weight of 0 excludes the token from loss; fractional weights are honored only by cross-entropy. - `dtype` 'D_TYPE_UNSPECIFIED' | 'D_TYPE_INT64' | 'D_TYPE_FLOAT32' | 'D_TYPE_BFLOAT16' - `grpo_inputs` RLGRPOLossInputs - `advantages` RLLossAdvantages, required - `data` number[], required — Float array of per-token advantages - `dtype` 'D_TYPE_UNSPECIFIED' | 'D_TYPE_INT64' | 'D_TYPE_FLOAT32' | 'D_TYPE_BFLOAT16' - `logprobs` RLLossLogprobs, required - `data` number[], required — Float array of per-token log probabilities - `dtype` 'D_TYPE_UNSPECIFIED' | 'D_TYPE_INT64' | 'D_TYPE_FLOAT32' | 'D_TYPE_BFLOAT16' - `reference_logprobs` RLLossLogprobs - `data` number[], required — Float array of per-token log probabilities - `dtype` 'D_TYPE_UNSPECIFIED' | 'D_TYPE_INT64' | 'D_TYPE_FLOAT32' | 'D_TYPE_BFLOAT16' - `importance_sampling_inputs` RLImportanceSamplingLossInputs — Loss inputs for unclipped importance-sampling policy-gradient updates. - `advantages` RLLossAdvantages, required - `data` number[], required — Float array of per-token advantages - `dtype` 'D_TYPE_UNSPECIFIED' | 'D_TYPE_INT64' | 'D_TYPE_FLOAT32' | 'D_TYPE_BFLOAT16' - `logprobs` RLLossLogprobs, required - `data` number[], required — Float array of per-token log probabilities - `dtype` 'D_TYPE_UNSPECIFIED' | 'D_TYPE_INT64' | 'D_TYPE_FLOAT32' | 'D_TYPE_BFLOAT16' - `ppo_inputs` RLPPOLossInputs - `advantages` RLLossAdvantages, required - `data` number[], required — Float array of per-token advantages - `dtype` 'D_TYPE_UNSPECIFIED' | 'D_TYPE_INT64' | 'D_TYPE_FLOAT32' | 'D_TYPE_BFLOAT16' - `logprobs` RLLossLogprobs, required - `data` number[], required — Float array of per-token log probabilities - `dtype` 'D_TYPE_UNSPECIFIED' | 'D_TYPE_INT64' | 'D_TYPE_FLOAT32' | 'D_TYPE_BFLOAT16' - `cispo_inputs` RLCISPOLossInputs - `advantages` RLLossAdvantages, required - `data` number[], required — Float array of per-token advantages - `dtype` 'D_TYPE_UNSPECIFIED' | 'D_TYPE_INT64' | 'D_TYPE_FLOAT32' | 'D_TYPE_BFLOAT16' - `logprobs` RLLossLogprobs, required - `data` number[], required — Float array of per-token log probabilities - `dtype` 'D_TYPE_UNSPECIFIED' | 'D_TYPE_INT64' | 'D_TYPE_FLOAT32' | 'D_TYPE_BFLOAT16' - `dro_inputs` RLDROLossInputs - `advantages` RLLossAdvantages, required - `data` number[], required — Float array of per-token advantages - `dtype` 'D_TYPE_UNSPECIFIED' | 'D_TYPE_INT64' | 'D_TYPE_FLOAT32' | 'D_TYPE_BFLOAT16' - `logprobs` RLLossLogprobs, required - `data` number[], required — Float array of per-token log probabilities - `dtype` 'D_TYPE_UNSPECIFIED' | 'D_TYPE_INT64' | 'D_TYPE_FLOAT32' | 'D_TYPE_BFLOAT16' - `routed_experts` RLRoutedExperts — Mixture-of-experts routing decisions captured while generating, so training can reuse the same expert selection. A contiguous uint16 buffer of selected expert indices, reshaped by `shape`, which is always `[num_tokens, num_layers, topk]`. - `data` string, byte, required — Base64-encoded contiguous uint16 buffer of selected expert indices, row-major over (token, layer, k). - `shape` union[], required — Buffer shape as `[num_tokens, num_layers, topk]`. - union - string - integer - `loss` RLLossConfig, required - `type` 'LOSS_TYPE_UNSPECIFIED' | 'LOSS_TYPE_CROSS_ENTROPY' | 'LOSS_TYPE_GRPO' | 'LOSS_TYPE_IMPORTANCE_SAMPLING' | 'LOSS_TYPE_PPO' | 'LOSS_TYPE_CISPO' | 'LOSS_TYPE_DRO', required — Type of loss function used for RL training. - `cross_entropy_params` RLCrossEntropyLossParams — Cross-entropy loss parameters (currently empty). - `grpo_params` RLGRPOLossParams - `clip_low_threshold` number, float — Lower clip threshold for the importance-sampling ratio. The ratio is clamped to this bound; tighter clipping makes policy updates more conservative. Must be <= 1. - `clip_high_threshold` number, float — Upper clip threshold for the importance-sampling ratio. The ratio is clamped to this bound; tighter clipping makes policy updates more conservative. Must be >= 1. - `beta` number — KL penalty coefficient - `agg_type` 'GRPO_LOSS_AGGREGATION_TYPE_UNSPECIFIED' | 'GRPO_LOSS_AGGREGATION_TYPE_FIXED_HORIZON' | 'GRPO_LOSS_AGGREGATION_TYPE_TOKEN_MEAN' | 'GRPO_LOSS_AGGREGATION_TYPE_SEQUENCE_MEAN' - `ratio_type` 'GRPO_LOSS_RATIO_TYPE_TOKEN' | 'GRPO_LOSS_RATIO_TYPE_SEQUENCE' — Controls whether GRPO loss uses token-level or sequence-level importance ratios. - `ppo_params` RLPPOLossParams - `clip_low_threshold` number, float — Lower absolute bound for the importance ratio in the clipped surrogate. Must be <= 1. - `clip_high_threshold` number, float — Upper absolute bound for the importance ratio in the clipped surrogate. Must be >= 1. - `cispo_params` RLCISPOLossParams - `clip_low_threshold` number, float — Lower absolute bound for the importance ratio; the clipped ratio is applied as a detached coefficient - `clip_high_threshold` number, float — Upper absolute bound for the importance ratio; the clipped ratio is applied as a detached coefficient - `dro_params` RLDROLossParams - `beta` number, float, required — Coefficient on the quadratic log-ratio penalty. Required; there is no default. ## Response `200` - RLForwardBackwardOperation — Async forward-backward pass operation - `id` string, required — Operation ID - `status` 'TRAINING_OPERATION_STATUS_UNSPECIFIED' | 'TRAINING_OPERATION_STATUS_PENDING' | 'TRAINING_OPERATION_STATUS_RUNNING' | 'TRAINING_OPERATION_STATUS_COMPLETED' | 'TRAINING_OPERATION_STATUS_FAILED', required - `output` RLForwardBackwardResult — Result of a forward-backward pass operation - `loss` number, required — Loss value - `metrics` object — Loss-specific metrics (e.g., KL divergence, clip fraction for GRPO) - `error` RLTrainingOperationError — Error details for a failed training operation - `code` 'TRAINING_OPERATION_ERROR_CODE_UNSPECIFIED' | 'TRAINING_OPERATION_ERROR_CODE_RESOURCE_EXHAUSTED' | 'TRAINING_OPERATION_ERROR_CODE_TIMEOUT' | 'TRAINING_OPERATION_ERROR_CODE_INTERNAL_ERROR' | 'TRAINING_OPERATION_ERROR_CODE_SESSION_NOT_ACTIVE' | 'TRAINING_OPERATION_ERROR_CODE_INVALID_INPUT' | 'TRAINING_OPERATION_ERROR_CODE_NON_FINITE_LOSS' — Application error code for a failed training operation - `message` string — Human-readable error message ## Other responses - `default` — An unexpected error response. --- [API](https://skmtc.net/together/apis/together-apis.md) · [All operations](https://skmtc.net/together/apis/together-apis/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/together/together-apis/revisions/1fab430e87d6/schema)