--- title: "GET /evaluation-jobs/{jobIdentifier}" method: GET path: "/evaluation-jobs/{jobIdentifier}" --- # GET /evaluation-jobs/{jobIdentifier} `GET /evaluation-jobs/{jobIdentifier}` Gets information about an evaluation job, such as the status of the job. ## Path parameters - `jobIdentifier` string, password, required ## Response `200` Success - GetEvaluationJobResponse - `jobName` string, required — The name for the evaluation job. - `status` 'InProgress' | 'Completed' | 'Failed' | 'Stopping' | 'Stopped' | 'Deleting', required — The current status of the evaluation job. - `jobArn` string, required — The Amazon Resource Name (ARN) of the evaluation job. - `jobDescription` string, password — The description of the evaluation job. - `roleArn` string, required — The Amazon Resource Name (ARN) of the IAM service role used in the evaluation job. - `customerEncryptionKeyId` string — The Amazon Resource Name (ARN) of the customer managed encryption key specified when the evaluation job was created. - `jobType` 'Human' | 'Automated', required — Specifies whether the evaluation job is automated or human-based. - `applicationType` 'ModelEvaluation' | 'RagEvaluation' — Specifies whether the evaluation job is for evaluating a model or evaluating a knowledge base (retrieval and response generation). - `evaluationConfig` object, required — Contains the configuration details of either an automated or human-based evaluation job. - `automated` object — Contains the configuration details of an automated evaluation job that computes metrics. - `datasetMetricConfigs` EvaluationDatasetMetricConfig[], required — Configuration details of the prompt datasets and metrics you want to use for your evaluation job. - `taskType` 'Summarization' | 'Classification' | 'QuestionAndAnswer' | 'Generation' | 'Custom', required — The the type of task you want to evaluate for your evaluation job. This applies only to model evaluation jobs and is ignored for knowledge base evaluation jobs. - `dataset` object, required — Specifies the prompt dataset. - `name` string, password, required — Used to specify supported built-in prompt datasets. Valid values are Builtin.Bold, Builtin.BoolQ, Builtin.NaturalQuestions, Builtin.Gigaword, Builtin.RealToxicityPrompts, Builtin.TriviaQA, Builtin.T-Rex, Builtin.WomensEcommerceClothingReviews and Builtin.Wikitext2. - `datasetLocation` object — For custom prompt datasets, you must specify the location in Amazon S3 where the prompt dataset is saved. - `s3Uri` string — The S3 URI of the S3 bucket specified in the job. - `metricNames` EvaluationMetricName[], required —

The names of the metrics you want to use for your evaluation job.

For knowledge base evaluation jobs that evaluate retrieval only, valid values are "Builtin.ContextRelevance", "Builtin.ContextCoverage".

For knowledge base evaluation jobs that evaluate retrieval with response generation, valid values are "Builtin.Correctness", "Builtin.Completeness", "Builtin.Helpfulness", "Builtin.LogicalCoherence", "Builtin.Faithfulness", "Builtin.Harmfulness", "Builtin.Stereotyping", "Builtin.Refusal".

For automated model evaluation jobs, valid values are "Builtin.Accuracy", "Builtin.Robustness", and "Builtin.Toxicity". In model evaluation jobs that use a LLM as judge you can specify "Builtin.Correctness", "Builtin.Completeness", "Builtin.Faithfulness", "Builtin.Helpfulness", "Builtin.Coherence", "Builtin.Relevance", "Builtin.FollowingInstructions", "Builtin.ProfessionalStyleAndTone", You can also specify the following responsible AI related metrics only for model evaluation job that use a LLM as judge "Builtin.Harmfulness", "Builtin.Stereotyping", and "Builtin.Refusal".

For human-based model evaluation jobs, the list of strings must match the name parameter specified in HumanEvaluationCustomMetric.

- `evaluatorModelConfig` object — Contains the evaluator model configuration details. EvaluatorModelConfig is required for evaluation jobs that use a knowledge base or in model evaluation job that use a model as judge. This model computes all evaluation related metrics. - `bedrockEvaluatorModels` BedrockEvaluatorModel[] — The evaluator model used in knowledge base evaluation job or in model evaluation job that use a model as judge. This model computes all evaluation related metrics. - `modelIdentifier` string, required — The Amazon Resource Name (ARN) of the evaluator model used used in knowledge base evaluation job or in model evaluation job that use a model as judge. - `customMetricConfig` object — Defines the configuration of custom metrics to be used in an evaluation job. - `customMetrics` AutomatedEvaluationCustomMetricSource[], required — Defines a list of custom metrics to be used in an Amazon Bedrock evaluation job. - `customMetricDefinition` object — The definition of a custom metric for use in an Amazon Bedrock evaluation job. - `name` string, password, required — The name for a custom metric. Names must be unique in your Amazon Web Services region. - `instructions` string, required — The prompt for a custom metric that instructs the evaluator model how to rate the model or RAG source under evaluation. - `ratingScale` RatingScaleItem[] — Defines the rating scale to be used for a custom metric. We recommend that you always define a ratings scale when creating a custom metric. If you don't define a scale, Amazon Bedrock won't be able to visually display the results of the evaluation in the console or calculate average values of numerical scores. For more information on specifying a rating scale, see Specifying an output schema (rating scale). - … - `evaluatorModelConfig` object, required — Configuration of the evaluator model you want to use to evaluate custom metrics in an Amazon Bedrock evaluation job. - `bedrockEvaluatorModels` CustomMetricBedrockEvaluatorModel[], required — Defines the model you want to evaluate custom metrics in an Amazon Bedrock evaluation job. - `modelIdentifier` string, required — The Amazon Resource Name (ARN) of the evaluator model for custom metrics. For a list of supported evaluator models, see Evaluate model performance using another LLM as a judge and Evaluate the performance of RAG sources using Amazon Bedrock evaluations. - `human` object — Contains the configuration details of an evaluation job that uses human workers. - `humanWorkflowConfig` object — The parameters of the human workflow. - `flowDefinitionArn` string, required — The Amazon Resource Number (ARN) for the flow definition - `instructions` string, password — Instructions for the flow definition - `customMetrics` HumanEvaluationCustomMetric[] — A HumanEvaluationCustomMetric object. It contains the names the metrics, how the metrics are to be evaluated, an optional description. - `name` string, password, required — The name of the metric. Your human evaluators will see this name in the evaluation UI. - `description` string, password — An optional description of the metric. Use this parameter to provide more details about the metric. - `ratingMethod` string, required — Choose how you want your human workers to evaluation your model. Valid values for rating methods are ThumbsUpDown, IndividualLikertScale,ComparisonLikertScale, ComparisonChoice, and ComparisonRank - `datasetMetricConfigs` EvaluationDatasetMetricConfig[], required — Use to specify the metrics, task, and prompt dataset to be used in your model evaluation job. - `taskType` 'Summarization' | 'Classification' | 'QuestionAndAnswer' | 'Generation' | 'Custom', required — The the type of task you want to evaluate for your evaluation job. This applies only to model evaluation jobs and is ignored for knowledge base evaluation jobs. - `dataset` object, required — Specifies the prompt dataset. - `name` string, password, required — Used to specify supported built-in prompt datasets. Valid values are Builtin.Bold, Builtin.BoolQ, Builtin.NaturalQuestions, Builtin.Gigaword, Builtin.RealToxicityPrompts, Builtin.TriviaQA, Builtin.T-Rex, Builtin.WomensEcommerceClothingReviews and Builtin.Wikitext2. - `datasetLocation` object — For custom prompt datasets, you must specify the location in Amazon S3 where the prompt dataset is saved. - `s3Uri` string — The S3 URI of the S3 bucket specified in the job. - `metricNames` EvaluationMetricName[], required —

The names of the metrics you want to use for your evaluation job.

For knowledge base evaluation jobs that evaluate retrieval only, valid values are "Builtin.ContextRelevance", "Builtin.ContextCoverage".

For knowledge base evaluation jobs that evaluate retrieval with response generation, valid values are "Builtin.Correctness", "Builtin.Completeness", "Builtin.Helpfulness", "Builtin.LogicalCoherence", "Builtin.Faithfulness", "Builtin.Harmfulness", "Builtin.Stereotyping", "Builtin.Refusal".

For automated model evaluation jobs, valid values are "Builtin.Accuracy", "Builtin.Robustness", and "Builtin.Toxicity". In model evaluation jobs that use a LLM as judge you can specify "Builtin.Correctness", "Builtin.Completeness", "Builtin.Faithfulness", "Builtin.Helpfulness", "Builtin.Coherence", "Builtin.Relevance", "Builtin.FollowingInstructions", "Builtin.ProfessionalStyleAndTone", You can also specify the following responsible AI related metrics only for model evaluation job that use a LLM as judge "Builtin.Harmfulness", "Builtin.Stereotyping", and "Builtin.Refusal".

For human-based model evaluation jobs, the list of strings must match the name parameter specified in HumanEvaluationCustomMetric.

- `inferenceConfig` object, required — Contains the configuration details of the inference model used for the evaluation job. - `models` EvaluationModelConfig[] — Specifies the inference models. - `bedrockModel` object — Defines the Amazon Bedrock model or inference profile and inference parameters you want used. - `modelIdentifier` string, required — The ARN of the Amazon Bedrock model or inference profile specified. - `inferenceParams` string, password — Each Amazon Bedrock support different inference parameters that change how the model behaves during inference. - `performanceConfig` object — Specifies performance settings for the model or inference profile. - `latency` 'standard' | 'optimized' — Specifies whether to use the latency-optimized or standard version of a model or inference profile. - `precomputedInferenceSource` object — Defines the model used to generate inference response data for a model evaluation job where you provide your own inference response data. - `inferenceSourceIdentifier` string, required — A label that identifies a model used in a model evaluation job where you provide your own inference response data. - `ragConfigs` RAGConfig[] — Contains the configuration details of the inference for a knowledge base evaluation job, including either the retrieval only configuration or the retrieval with response generation configuration. - `knowledgeBaseConfig` object — Contains configuration details for knowledge base retrieval and response generation. - `retrieveConfig` object — Contains configuration details for retrieving information from a knowledge base. - `knowledgeBaseId` string, required — The unique identifier of the knowledge base. - `knowledgeBaseRetrievalConfiguration` object, required — Contains configuration details for knowledge base retrieval. - `vectorSearchConfiguration` object, required — Contains configuration details for returning the results from the vector search. - … - `retrieveAndGenerateConfig` object — Contains configuration details for retrieving information from a knowledge base and generating responses. - `type` 'KNOWLEDGE_BASE' | 'EXTERNAL_SOURCES', required —

The type of resource that contains your data for retrieving information and generating responses.

If you choose to use EXTERNAL_SOURCES, then currently only Claude 3 Sonnet models for knowledge bases are supported.

- `knowledgeBaseConfiguration` object — Contains configuration details for the knowledge base retrieval and response generation. - `knowledgeBaseId` string, required — The unique identifier of the knowledge base. - `modelArn` string, required — The Amazon Resource Name (ARN) of the foundation model or inference profile used to generate responses. - `retrievalConfiguration` object — Contains configuration details for retrieving text chunks. - … - `generationConfiguration` object — Contains configurations details for response generation based on retrieved text chunks. - … - `orchestrationConfiguration` object — Contains configuration details for the model to process the prompt prior to retrieval and response generation. - … - `externalSourcesConfiguration` object — The configuration for the external source wrapper object in the retrieveAndGenerate function. - `modelArn` string, required — The Amazon Resource Name (ARN) of the foundation model or inference profile used to generate responses. - `sources` ExternalSource[], required — The document for the external source wrapper object in the retrieveAndGenerate function. - … - `generationConfiguration` object — Contains configurations details for response generation based on retrieved text chunks. - … - `precomputedRagSourceConfig` object — Contains configuration details about the RAG source used to generate inference response data for a Knowledge Base evaluation job. - `retrieveSourceConfig` object — A summary of a RAG source used for a retrieve-only Knowledge Base evaluation job where you provide your own inference response data. - `ragSourceIdentifier` string, required — A label that identifies the RAG source used for a retrieve-only Knowledge Base evaluation job where you provide your own inference response data. - `retrieveAndGenerateSourceConfig` object — A summary of a RAG source used for a retrieve-and-generate Knowledge Base evaluation job where you provide your own inference response data. - `ragSourceIdentifier` string, required — A label that identifies the RAG source used for a retrieve-and-generate Knowledge Base evaluation job where you provide your own inference response data. - `outputDataConfig` object, required — Contains the configuration details of the Amazon S3 bucket for storing the results of the evaluation job. - `s3Uri` string, required — The Amazon S3 URI where the results of the evaluation job are saved. - `creationTime` string, date-time, required — The time the evaluation job was created. - `lastModifiedTime` string, date-time — The time the evaluation job was last modified. - `failureMessages` ErrorMessage[] — A list of strings that specify why the evaluation job failed to create. ## Other responses - `480` — ResourceNotFoundException - `481` — AccessDeniedException - `482` — ValidationException - `483` — InternalServerException - `484` — ThrottlingException --- [API](https://skmtc.net/aws/apis/bedrock.md) · [All operations](https://skmtc.net/aws/apis/bedrock/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/aws/bedrock/versions/3bc874e43ecd/schema)