--- title: "Validate a grader." method: POST path: "/fine_tuning/alpha/graders/validate" tags: ["Fine-tuning"] --- # Validate a grader. `POST /fine_tuning/alpha/graders/validate` ## Request body - ValidateGraderRequest - `grader` union, required — The grader used for the fine-tuning job. - object — A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - `type` 'string_check', required — The object type, which is always `string_check`. - `name` string, required — The name of the grader. - `input` string, required — The input text. This may include template strings. - `reference` string, required — The reference text. This may include template strings. - `operation` 'eq' | 'ne' | 'like' | 'ilike', required — The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. - object — A TextSimilarityGrader object which grades text based on similarity metrics. - `type` 'text_similarity', required — The type of grader. - `name` string, required — The name of the grader. - `input` string, required — The text being graded. - `reference` string, required — The text being graded against. - `evaluation_metric` 'cosine' | 'fuzzy_match' | 'bleu' | 'gleu' | 'meteor' | 'rouge_1' | 'rouge_2' | 'rouge_3' | 'rouge_4' | 'rouge_5' | 'rouge_l', required — The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`. - object — A PythonGrader object that runs a python script on the input. - `type` 'python', required — The object type, which is always `python`. - `name` string, required — The name of the grader. - `source` string, required — The source code of the python script. - `image_tag` string — The image tag to use for the python script. - object — A ScoreModelGrader object that uses a model to assign a score to the input. - `type` 'score_model', required — The object type, which is always `score_model`. - `name` string, required — The name of the grader. - `model` string, required — The model to use for the evaluation. - `sampling_params` object — The sampling parameters for the model. - `seed` integer, nullable — A seed value to initialize the randomness, during sampling. - `top_p` number, nullable — An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - `temperature` number, nullable — A higher temperature increases randomness in the outputs. - `max_completions_tokens` integer, nullable — The maximum number of tokens the grader model may generate in its response. - `reasoning_effort` 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh', nullable — Constrains effort on reasoning for [reasoning models](https://platform.openai.com/docs/guides/reasoning). Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - `xhigh` is supported for all models after `gpt-5.1-codex-max`. - `input` EvalItem[], required — The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. - `role` 'user' | 'assistant' | 'system' | 'developer', required — The role of the message input. One of `user`, `assistant`, `system`, or `developer`. - `content` union, required — Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - union — A single content item: input text, output text, input image, or input audio. - string — A text input to the model. - InputTextContent — A text input to the model. - … - EvalItemContentOutputText — A text output from the model. - … - EvalItemInputImage — An image input block used within EvalItem content arrays. - … - InputAudio — An audio input to the model. - … - EvalItemContentItem[] — A list of inputs, each of which may be either an input text, output text, input image, or input audio object. - union — A single content item: input text, output text, input image, or input audio. - … - `type` 'message' — The type of the message input. Always `message`. - `range` number[] — The range of the score. Defaults to `[0, 1]`. - object — A MultiGrader object combines the output of multiple graders to produce a single score. - `type` 'multi', required — The object type, which is always `multi`. - `name` string, required — The name of the grader. - `graders` union, required - GraderStringCheck — A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - `type` 'string_check', required — The object type, which is always `string_check`. - `name` string, required — The name of the grader. - `input` string, required — The input text. This may include template strings. - `reference` string, required — The reference text. This may include template strings. - `operation` 'eq' | 'ne' | 'like' | 'ilike', required — The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. - GraderTextSimilarity — A TextSimilarityGrader object which grades text based on similarity metrics. - `type` 'text_similarity', required — The type of grader. - `name` string, required — The name of the grader. - `input` string, required — The text being graded. - `reference` string, required — The text being graded against. - `evaluation_metric` 'cosine' | 'fuzzy_match' | 'bleu' | 'gleu' | 'meteor' | 'rouge_1' | 'rouge_2' | 'rouge_3' | 'rouge_4' | 'rouge_5' | 'rouge_l', required — The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`. - GraderPython — A PythonGrader object that runs a python script on the input. - `type` 'python', required — The object type, which is always `python`. - `name` string, required — The name of the grader. - `source` string, required — The source code of the python script. - `image_tag` string — The image tag to use for the python script. - GraderScoreModel — A ScoreModelGrader object that uses a model to assign a score to the input. - `type` 'score_model', required — The object type, which is always `score_model`. - `name` string, required — The name of the grader. - `model` string, required — The model to use for the evaluation. - `sampling_params` object — The sampling parameters for the model. - `seed` integer, nullable — A seed value to initialize the randomness, during sampling. - `top_p` number, nullable — An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - `temperature` number, nullable — A higher temperature increases randomness in the outputs. - `max_completions_tokens` integer, nullable — The maximum number of tokens the grader model may generate in its response. - `reasoning_effort` 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh', nullable — Constrains effort on reasoning for [reasoning models](https://platform.openai.com/docs/guides/reasoning). Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - `xhigh` is supported for all models after `gpt-5.1-codex-max`. - `input` EvalItem[], required — The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. - `role` 'user' | 'assistant' | 'system' | 'developer', required — The role of the message input. One of `user`, `assistant`, `system`, or `developer`. - `content` union, required — Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - … - `type` 'message' — The type of the message input. Always `message`. - `range` number[] — The range of the score. Defaults to `[0, 1]`. - GraderLabelModel — A LabelModelGrader object which uses a model to assign labels to each item in the evaluation. - `type` 'label_model', required — The object type, which is always `label_model`. - `name` string, required — The name of the grader. - `model` string, required — The model to use for the evaluation. Must support structured outputs. - `input` EvalItem[], required - `role` 'user' | 'assistant' | 'system' | 'developer', required — The role of the message input. One of `user`, `assistant`, `system`, or `developer`. - `content` union, required — Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - … - `type` 'message' — The type of the message input. Always `message`. - `labels` string[], required — The labels to assign to each item in the evaluation. - `passing_labels` string[], required — The labels that indicate a passing result. Must be a subset of labels. - `calculate_output` string, required — A formula to calculate the output based on grader results. ## Response `200` OK - ValidateGraderResponse - `grader` union — The grader used for the fine-tuning job. - object — A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - `type` 'string_check', required — The object type, which is always `string_check`. - `name` string, required — The name of the grader. - `input` string, required — The input text. This may include template strings. - `reference` string, required — The reference text. This may include template strings. - `operation` 'eq' | 'ne' | 'like' | 'ilike', required — The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. - object — A TextSimilarityGrader object which grades text based on similarity metrics. - `type` 'text_similarity', required — The type of grader. - `name` string, required — The name of the grader. - `input` string, required — The text being graded. - `reference` string, required — The text being graded against. - `evaluation_metric` 'cosine' | 'fuzzy_match' | 'bleu' | 'gleu' | 'meteor' | 'rouge_1' | 'rouge_2' | 'rouge_3' | 'rouge_4' | 'rouge_5' | 'rouge_l', required — The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`. - object — A PythonGrader object that runs a python script on the input. - `type` 'python', required — The object type, which is always `python`. - `name` string, required — The name of the grader. - `source` string, required — The source code of the python script. - `image_tag` string — The image tag to use for the python script. - object — A ScoreModelGrader object that uses a model to assign a score to the input. - `type` 'score_model', required — The object type, which is always `score_model`. - `name` string, required — The name of the grader. - `model` string, required — The model to use for the evaluation. - `sampling_params` object — The sampling parameters for the model. - `seed` integer, nullable — A seed value to initialize the randomness, during sampling. - `top_p` number, nullable — An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - `temperature` number, nullable — A higher temperature increases randomness in the outputs. - `max_completions_tokens` integer, nullable — The maximum number of tokens the grader model may generate in its response. - `reasoning_effort` 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh', nullable — Constrains effort on reasoning for [reasoning models](https://platform.openai.com/docs/guides/reasoning). Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - `xhigh` is supported for all models after `gpt-5.1-codex-max`. - `input` EvalItem[], required — The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. - `role` 'user' | 'assistant' | 'system' | 'developer', required — The role of the message input. One of `user`, `assistant`, `system`, or `developer`. - `content` union, required — Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - union — A single content item: input text, output text, input image, or input audio. - string — A text input to the model. - InputTextContent — A text input to the model. - … - EvalItemContentOutputText — A text output from the model. - … - EvalItemInputImage — An image input block used within EvalItem content arrays. - … - InputAudio — An audio input to the model. - … - EvalItemContentItem[] — A list of inputs, each of which may be either an input text, output text, input image, or input audio object. - union — A single content item: input text, output text, input image, or input audio. - … - `type` 'message' — The type of the message input. Always `message`. - `range` number[] — The range of the score. Defaults to `[0, 1]`. - object — A MultiGrader object combines the output of multiple graders to produce a single score. - `type` 'multi', required — The object type, which is always `multi`. - `name` string, required — The name of the grader. - `graders` union, required - GraderStringCheck — A StringCheckGrader object that performs a string comparison between input and reference using a specified operation. - `type` 'string_check', required — The object type, which is always `string_check`. - `name` string, required — The name of the grader. - `input` string, required — The input text. This may include template strings. - `reference` string, required — The reference text. This may include template strings. - `operation` 'eq' | 'ne' | 'like' | 'ilike', required — The string check operation to perform. One of `eq`, `ne`, `like`, or `ilike`. - GraderTextSimilarity — A TextSimilarityGrader object which grades text based on similarity metrics. - `type` 'text_similarity', required — The type of grader. - `name` string, required — The name of the grader. - `input` string, required — The text being graded. - `reference` string, required — The text being graded against. - `evaluation_metric` 'cosine' | 'fuzzy_match' | 'bleu' | 'gleu' | 'meteor' | 'rouge_1' | 'rouge_2' | 'rouge_3' | 'rouge_4' | 'rouge_5' | 'rouge_l', required — The evaluation metric to use. One of `cosine`, `fuzzy_match`, `bleu`, `gleu`, `meteor`, `rouge_1`, `rouge_2`, `rouge_3`, `rouge_4`, `rouge_5`, or `rouge_l`. - GraderPython — A PythonGrader object that runs a python script on the input. - `type` 'python', required — The object type, which is always `python`. - `name` string, required — The name of the grader. - `source` string, required — The source code of the python script. - `image_tag` string — The image tag to use for the python script. - GraderScoreModel — A ScoreModelGrader object that uses a model to assign a score to the input. - `type` 'score_model', required — The object type, which is always `score_model`. - `name` string, required — The name of the grader. - `model` string, required — The model to use for the evaluation. - `sampling_params` object — The sampling parameters for the model. - `seed` integer, nullable — A seed value to initialize the randomness, during sampling. - `top_p` number, nullable — An alternative to temperature for nucleus sampling; 1.0 includes all tokens. - `temperature` number, nullable — A higher temperature increases randomness in the outputs. - `max_completions_tokens` integer, nullable — The maximum number of tokens the grader model may generate in its response. - `reasoning_effort` 'none' | 'minimal' | 'low' | 'medium' | 'high' | 'xhigh', nullable — Constrains effort on reasoning for [reasoning models](https://platform.openai.com/docs/guides/reasoning). Currently supported values are `none`, `minimal`, `low`, `medium`, `high`, and `xhigh`. Reducing reasoning effort can result in faster responses and fewer tokens used on reasoning in a response. - `gpt-5.1` defaults to `none`, which does not perform reasoning. The supported reasoning values for `gpt-5.1` are `none`, `low`, `medium`, and `high`. Tool calls are supported for all reasoning values in gpt-5.1. - All models before `gpt-5.1` default to `medium` reasoning effort, and do not support `none`. - The `gpt-5-pro` model defaults to (and only supports) `high` reasoning effort. - `xhigh` is supported for all models after `gpt-5.1-codex-max`. - `input` EvalItem[], required — The input messages evaluated by the grader. Supports text, output text, input image, and input audio content blocks, and may include template strings. - `role` 'user' | 'assistant' | 'system' | 'developer', required — The role of the message input. One of `user`, `assistant`, `system`, or `developer`. - `content` union, required — Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - … - `type` 'message' — The type of the message input. Always `message`. - `range` number[] — The range of the score. Defaults to `[0, 1]`. - GraderLabelModel — A LabelModelGrader object which uses a model to assign labels to each item in the evaluation. - `type` 'label_model', required — The object type, which is always `label_model`. - `name` string, required — The name of the grader. - `model` string, required — The model to use for the evaluation. Must support structured outputs. - `input` EvalItem[], required - `role` 'user' | 'assistant' | 'system' | 'developer', required — The role of the message input. One of `user`, `assistant`, `system`, or `developer`. - `content` union, required — Inputs to the model - can contain template strings. Supports text, output text, input images, and input audio, either as a single item or an array of items. - … - `type` 'message' — The type of the message input. Always `message`. - `labels` string[], required — The labels to assign to each item in the evaluation. - `passing_labels` string[], required — The labels that indicate a passing result. Must be a subset of labels. - `calculate_output` string, required — A formula to calculate the output based on grader results. --- [API](https://skmtc.net/openai/apis/openai-api-3.md) · [All operations](https://skmtc.net/openai/apis/openai-api-3/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/openai/openai-api-3/versions/74cbcf73838f/schema)