--- title: "POST /v1/{+endpoint}:deployModel" method: POST path: "/v1/{+endpoint}:deployModel" tags: ["projects"] --- # POST /v1/{+endpoint}:deployModel `POST /v1/{+endpoint}:deployModel` Deploys a Model into this Endpoint, creating a DeployedModel within it. ## Path parameters - `endpoint` string, required ## Request body - GoogleCloudAiplatformV1DeployModelRequest — Request message for EndpointService.DeployModel. - `deployedModel` GoogleCloudAiplatformV1DeployedModel — A deployment of a Model. Endpoints contain one or more DeployedModels. - `explanationSpec` GoogleCloudAiplatformV1ExplanationSpec — Specification of Model explanation. - `parameters` GoogleCloudAiplatformV1ExplanationParameters — Parameters to configure explaining for Model's predictions. - `outputIndices` unknown[] — If populated, only returns attributions that have output_index contained in output_indices. It must be an ndarray of integers, with the same shape of the output it's explaining. If not populated, returns attributions for top_k indices of outputs. If neither top_k nor output_indices is populated, returns the argmax index of the outputs. Only applicable to Models that predict multiple outputs (e,g, multi-class Models that predict multiple classes). - unknown - `xraiAttribution` GoogleCloudAiplatformV1XraiAttribution — An explanation method that redistributes Integrated Gradients attributions to segmented regions, taking advantage of the model's fully differentiable structure. Refer to this paper for more details: https://arxiv.org/abs/1906.02825 Supported only by image Models. - `stepCount` integer — Required. The number of steps for approximating the path integral. A good value to start is 50 and gradually increase until the sum to diff property is met within the desired error range. Valid range of its value is [1, 100], inclusively. - `blurBaselineConfig` GoogleCloudAiplatformV1BlurBaselineConfig — Config for blur baseline. When enabled, a linear path from the maximally blurred image to the input image is created. Using a blurred baseline instead of zero (black image) is motivated by the BlurIG approach explained here: https://arxiv.org/abs/2004.03383 - `maxBlurSigma` number, float — The standard deviation of the blur kernel for the blurred baseline. The same blurring parameter is used for both the height and the width dimension. If not set, the method defaults to the zero (i.e. black for images) baseline. - `smoothGradConfig` GoogleCloudAiplatformV1SmoothGradConfig — Config for SmoothGrad approximation of gradients. When enabled, the gradients are approximated by averaging the gradients from noisy samples in the vicinity of the inputs. Adding noise can help improve the computed gradients. Refer to this paper for more details: https://arxiv.org/pdf/1706.03825.pdf - `featureNoiseSigma` GoogleCloudAiplatformV1FeatureNoiseSigma — Noise sigma by features. Noise sigma represents the standard deviation of the gaussian kernel that will be used to add noise to interpolated inputs prior to computing gradients. - … - `noiseSigma` number, float — This is a single float value and will be used to add noise to all the features. Use this field when all features are normalized to have the same distribution: scale to range [0, 1], [-1, 1] or z-scoring, where features are normalized to have 0-mean and 1-variance. Learn more about [normalization](https://developers.google.com/machine-learning/data-prep/transform/normalization). For best results the recommended value is about 10% - 20% of the standard deviation of the input feature. Refer to section 3.2 of the SmoothGrad paper: https://arxiv.org/pdf/1706.03825.pdf. Defaults to 0.1. If the distribution is different per feature, set feature_noise_sigma instead for each feature. - `noisySampleCount` integer — The number of gradient samples to use for approximation. The higher this number, the more accurate the gradient is, but the runtime complexity increases by this factor as well. Valid range of its value is [1, 50]. Defaults to 3. - `examples` GoogleCloudAiplatformV1Examples — Example-based explainability that returns the nearest neighbors from the provided dataset. - `presets` GoogleCloudAiplatformV1Presets — Preset configuration for example-based explanations - `query` 'PRECISE' | 'FAST' — Preset option controlling parameters for speed-precision trade-off when querying for examples. If omitted, defaults to `PRECISE`. - `modality` 'MODALITY_UNSPECIFIED' | 'IMAGE' | 'TEXT' | 'TABULAR' — The modality of the uploaded model, which automatically configures the distance measurement and feature normalization for the underlying example index and queries. If your model does not precisely fit one of these types, it is okay to choose the closest type. - `neighborCount` integer — The number of neighbors to return when querying for examples. - `exampleGcsSource` GoogleCloudAiplatformV1ExamplesExampleGcsSource — The Cloud Storage input instances. - `gcsSource` GoogleCloudAiplatformV1GcsSource — The Google Cloud Storage location for the input content. - … - `dataFormat` 'DATA_FORMAT_UNSPECIFIED' | 'JSONL' — The format in which instances are given, if not specified, assume it's JSONL format. Currently only JSONL format is supported. - `nearestNeighborSearchConfig` unknown - `topK` integer — If populated, returns attributions for top K indices of outputs (defaults to 1). Only applies to Models that predicts more than one outputs (e,g, multi-class Models). When set to -1, returns explanations for all outputs. - `sampledShapleyAttribution` GoogleCloudAiplatformV1SampledShapleyAttribution — An attribution method that approximates Shapley values for features that contribute to the label being predicted. A sampling strategy is used to approximate the value rather than considering all subsets of features. - `pathCount` integer — Required. The number of feature permutations to consider when approximating the Shapley values. Valid range of its value is [1, 50], inclusively. - `integratedGradientsAttribution` GoogleCloudAiplatformV1IntegratedGradientsAttribution — An attribution method that computes the Aumann-Shapley value taking advantage of the model's fully differentiable structure. Refer to this paper for more details: https://arxiv.org/abs/1703.01365 - `smoothGradConfig` GoogleCloudAiplatformV1SmoothGradConfig — Config for SmoothGrad approximation of gradients. When enabled, the gradients are approximated by averaging the gradients from noisy samples in the vicinity of the inputs. Adding noise can help improve the computed gradients. Refer to this paper for more details: https://arxiv.org/pdf/1706.03825.pdf - `featureNoiseSigma` GoogleCloudAiplatformV1FeatureNoiseSigma — Noise sigma by features. Noise sigma represents the standard deviation of the gaussian kernel that will be used to add noise to interpolated inputs prior to computing gradients. - … - `noiseSigma` number, float — This is a single float value and will be used to add noise to all the features. Use this field when all features are normalized to have the same distribution: scale to range [0, 1], [-1, 1] or z-scoring, where features are normalized to have 0-mean and 1-variance. Learn more about [normalization](https://developers.google.com/machine-learning/data-prep/transform/normalization). For best results the recommended value is about 10% - 20% of the standard deviation of the input feature. Refer to section 3.2 of the SmoothGrad paper: https://arxiv.org/pdf/1706.03825.pdf. Defaults to 0.1. If the distribution is different per feature, set feature_noise_sigma instead for each feature. - `noisySampleCount` integer — The number of gradient samples to use for approximation. The higher this number, the more accurate the gradient is, but the runtime complexity increases by this factor as well. Valid range of its value is [1, 50]. Defaults to 3. - `stepCount` integer — Required. The number of steps for approximating the path integral. A good value to start is 50 and gradually increase until the sum to diff property is within the desired error range. Valid range of its value is [1, 100], inclusively. - `blurBaselineConfig` GoogleCloudAiplatformV1BlurBaselineConfig — Config for blur baseline. When enabled, a linear path from the maximally blurred image to the input image is created. Using a blurred baseline instead of zero (black image) is motivated by the BlurIG approach explained here: https://arxiv.org/abs/2004.03383 - `maxBlurSigma` number, float — The standard deviation of the blur kernel for the blurred baseline. The same blurring parameter is used for both the height and the width dimension. If not set, the method defaults to the zero (i.e. black for images) baseline. - `metadata` GoogleCloudAiplatformV1ExplanationMetadata — Metadata describing the Model's input and output for explanation. - `inputs` object — Required. Map from feature names to feature input metadata. Keys are the name of the features. Values are the specification of the feature. An empty InputMetadata is valid. It describes a text feature which has the name specified as the key in ExplanationMetadata.inputs. The baseline of the empty feature is chosen by Vertex AI. For Vertex AI-provided Tensorflow images, the key can be any friendly name of the feature. Once specified, featureAttributions are keyed by this key (if not grouped with another feature). For custom images, the key must match with the key in instance. - `featureAttributionsSchemaUri` string — Points to a YAML file stored on Google Cloud Storage describing the format of the feature attributions. The schema is defined as an OpenAPI 3.0.2 [Schema Object](https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.0.2.md#schemaObject). AutoML tabular Models always have this field populated by Vertex AI. Note: The URI given on output may be different, including the URI scheme, than the one given on input. The output URI will point to a location where the user only has a read access. - `latentSpaceSource` string — Name of the source to generate embeddings for example based explanations. - `outputs` object — Required. Map from output names to output metadata. For Vertex AI-provided Tensorflow images, keys can be any user defined string that consists of any UTF-8 characters. For custom images, keys are the name of the output field in the prediction to be explained. Currently only one key is allowed. - `privateEndpoints` GoogleCloudAiplatformV1PrivateEndpoints — PrivateEndpoints proto is used to provide paths for users to send requests privately. To send request via private service access, use predict_http_uri, explain_http_uri or health_http_uri. To send request via private service connect, use service_attachment. - `explainHttpUri` string — Output only. Http(s) path to send explain requests. - `healthHttpUri` string — Output only. Http(s) path to send health check requests. - `predictHttpUri` string — Output only. Http(s) path to send prediction requests. - `serviceAttachment` string — Output only. The name of the service attachment resource. Populated if private service connect is enabled. - `enableAccessLogging` boolean — If true, online prediction access logs are sent to Cloud Logging. These logs are like standard server access logs, containing information like timestamp and latency for each prediction request. Note that logs may incur a cost, especially if your project receives prediction requests at a high queries per second rate (QPS). Estimate your costs before enabling this option. - `modelVersionId` string — Output only. The version ID of the model that is deployed. - `dedicatedResources` GoogleCloudAiplatformV1DedicatedResources — A description of resources that are dedicated to a DeployedModel or DeployedIndex, and that need a higher degree of manual configuration. - `spot` boolean — Optional. If true, schedule the deployment workload on [spot VMs](https://cloud.google.com/kubernetes-engine/docs/concepts/spot-vms). - `machineSpec` GoogleCloudAiplatformV1MachineSpec — Specification of a single machine. - `reservationAffinity` GoogleCloudAiplatformV1ReservationAffinity — A ReservationAffinity can be used to configure a Vertex AI resource (e.g., a DeployedModel) to draw its Compute Engine resources from a Shared Reservation, or exclusively from on-demand capacity. - `key` string — Optional. Corresponds to the label key of a reservation resource. To target a SPECIFIC_RESERVATION by name, use `compute.googleapis.com/reservation-name` as the key and specify the name of your reservation as its value. - `reservationAffinityType` 'TYPE_UNSPECIFIED' | 'NO_RESERVATION' | 'ANY_RESERVATION' | 'SPECIFIC_RESERVATION' — Required. Specifies the reservation affinity type. - `values` string[] — Optional. Corresponds to the label values of a reservation resource. This must be the full resource name of the reservation or reservation block. - `gpuPartitionSize` string — Optional. Immutable. The Nvidia GPU partition size. When specified, the requested accelerators will be partitioned into smaller GPU partitions. For example, if the request is for 8 units of NVIDIA A100 GPUs, and gpu_partition_size="1g.10gb", the service will create 8 * 7 = 56 partitioned MIG instances. The partition size must be a value supported by the requested accelerator. Refer to [Nvidia GPU Partitioning](https://cloud.google.com/kubernetes-engine/docs/how-to/gpus-multi#multi-instance_gpu_partitions) for the available partition sizes. If set, the accelerator_count should be set to 1. - `tpuTopology` string — Immutable. The topology of the TPUs. Corresponds to the TPU topologies available from GKE. (Example: tpu_topology: "2x2x1"). - `machineType` string — Immutable. The type of the machine. See the [list of machine types supported for prediction](https://cloud.google.com/gemini-enterprise-agent-platform/machine-learning/predictions/configure-compute#machine-types) See the [list of machine types supported for custom training](https://cloud.google.com/gemini-enterprise-agent-platform/machine-learning/training/configure-compute#machine-types). For DeployedModel this field is optional, and the default value is `n1-standard-2`. For BatchPredictionJob or as part of WorkerPoolSpec this field is required. - `acceleratorCount` integer — The number of accelerators to attach to the machine. For [accelerator optimized machine types](https://cloud.google.com/compute/docs/accelerator-optimized-machines), One may set the accelerator_count from 1 to N for machine with N GPUs. If accelerator_count is less than or equal to N / 2, Agent Platform co-schedules the replicas of the model into the same VM to save cost. For example, if the machine type is a3-highgpu-8g, which has 8 H100 GPUs, one can set accelerator_count to 1 to 8. If accelerator_count is 1, 2, 3, or 4, Agent Platform co-schedules 8, 4, 2, or 2 replicas of the model into the same VM to save cost. When co-scheduling, CPU, memory and storage on the VM will be distributed to replicas on the VM. For example, one can expect a co-scheduled replica requesting 2 GPUs out of a 8-GPU VM will receive 25% of the CPU, memory and storage of the VM. Note that the feature is not compatible with multihost_gpu_node_count. When multihost_gpu_node_count is set, the co-scheduling will not be enabled. - `acceleratorType` 'ACCELERATOR_TYPE_UNSPECIFIED' | 'NVIDIA_TESLA_K80' | 'NVIDIA_TESLA_P100' | 'NVIDIA_TESLA_V100' | 'NVIDIA_TESLA_P4' | 'NVIDIA_TESLA_T4' | 'NVIDIA_TESLA_A100' | 'NVIDIA_A100_80GB' | 'NVIDIA_L4' | 'NVIDIA_H100_80GB' | 'NVIDIA_H100_MEGA_80GB' | 'NVIDIA_H200_141GB' | 'NVIDIA_B200' | 'NVIDIA_GB200' | 'NVIDIA_RTX_PRO_6000' | 'TPU_V2' | 'TPU_V3' | 'TPU_V4_POD' | 'TPU_V5_LITEPOD' — Immutable. The type of accelerator(s) that may be attached to the machine as per accelerator_count. - `autoscalingMetricSpecs` GoogleCloudAiplatformV1AutoscalingMetricSpec[] — Immutable. The metric specifications that overrides a resource utilization metric (CPU utilization, accelerator's duty cycle, and so on) target value (default to 60 if not set). At most one entry is allowed per metric. If machine_spec.accelerator_count is above 0, the autoscaling will be based on both CPU utilization and accelerator's duty cycle metrics and scale up when either metrics exceeds its target value while scale down if both metrics are under their target value. The default target value is 60 for both metrics. If machine_spec.accelerator_count is 0, the autoscaling will be based on CPU utilization metric only with default target value 60 if not explicitly set. For example, in the case of Online Prediction, if you want to override target CPU utilization to 80, you should set autoscaling_metric_specs.metric_name to `aiplatform.googleapis.com/prediction/online/cpu/utilization` and autoscaling_metric_specs.target to `80`. - `metricName` string — Required. The resource metric name. Supported metrics: * For Online Prediction: * `aiplatform.googleapis.com/prediction/online/accelerator/duty_cycle` * `aiplatform.googleapis.com/prediction/online/cpu/utilization` * `aiplatform.googleapis.com/prediction/online/request_count` * `pubsub.googleapis.com/subscription/num_undelivered_messages` * `prometheus.googleapis.com/vertex_dcgm_fi_dev_gpu_util` * `prometheus.googleapis.com/vertex_vllm_gpu_cache_usage_perc` * `prometheus.googleapis.com/vertex_vllm_num_requests_waiting` - `target` integer — The target resource utilization in percentage (1% - 100%) for the given metric; once the real usage deviates from the target by a certain percentage, the machine replicas change. The default value is 60 (representing 60%) if not provided. - `minReplicaCount` integer — Required. Immutable. The minimum number of machine replicas that will be always deployed on. This value must be greater than or equal to 1. If traffic increases, it may dynamically be deployed onto more replicas, and as traffic decreases, some of these extra replicas may be freed. - `maxReplicaCount` integer — Immutable. The maximum number of replicas that may be deployed on when the traffic against it increases. If the requested value is too large, the deployment will error, but if deployment succeeds then the ability to scale to that many replicas is guaranteed (barring service outages). If traffic increases beyond what its replicas at maximum may handle, a portion of the traffic will be dropped. If this value is not provided, will use min_replica_count as the default value. The value of this field impacts the charge against Agent Platform CPU and GPU quotas. Specifically, you will be charged for (max_replica_count * number of cores in the selected machine type) and (max_replica_count * number of GPUs per replica in the selected machine type). - `requiredReplicaCount` integer — Optional. Number of required available replicas for the deployment to succeed. This field is only needed when partial deployment/mutation is desired. If set, the deploy/mutate operation will succeed once available_replica_count reaches required_replica_count, and the rest of the replicas will be retried. If not set, the default required_replica_count will be min_replica_count. - `id` string — Immutable. The ID of the DeployedModel. If not provided upon deployment, Vertex AI will generate a value for this ID. This value should be 1-10 characters, and valid characters are `/[0-9]/`. - `status` GoogleCloudAiplatformV1DeployedModelStatus — Runtime status of the deployed model. - `availableReplicaCount` integer — Output only. The number of available replicas of the deployed model. - `message` string — Output only. The latest deployed model's status message (if any). - `lastUpdateTime` string, google-datetime — Output only. The time at which the status was last updated. - `gdcConnectedModel` string — GDC pretrained / Gemini model name. The model name is a plain model name, e.g. gemini-1.5-flash-002. - `displayName` string — The display name of the DeployedModel. If not provided upon creation, the Model's display_name is used. - `disableContainerLogging` boolean — For custom-trained Models and AutoML Tabular Models, the container of the DeployedModel instances will send `stderr` and `stdout` streams to Cloud Logging by default. Please note that the logs incur cost, which are subject to [Cloud Logging pricing](https://cloud.google.com/logging/pricing). User can disable container logging by setting this flag to true. - `serviceAccount` string — The service account that the DeployedModel's container runs as. Specify the email address of the service account. If this service account is not specified, the container runs as a service account that doesn't have access to the resource project. Users deploying the Model must have the `iam.serviceAccounts.actAs` permission on this service account. - `model` string — The resource name of the Model that this is the deployment of. Note that the Model may be in a different location than the DeployedModel's Endpoint. The resource name may contain version id or version alias to specify the version. Example: `projects/{project}/locations/{location}/models/{model}@2` or `projects/{project}/locations/{location}/models/{model}@golden` if no version is specified, the default version will be deployed. - `checkpointId` string — The checkpoint id of the model. - `systemLabels` object — System labels to apply to Model Garden deployments. System labels are managed by Google for internal use only. - `sharedResources` string — The resource name of the shared DeploymentResourcePool to deploy on. Format: `projects/{project}/locations/{location}/deploymentResourcePools/{deployment_resource_pool}` - `speculativeDecodingSpec` GoogleCloudAiplatformV1SpeculativeDecodingSpec — Configuration for Speculative Decoding. - `speculativeTokenCount` integer — The number of speculative tokens to generate at each step. - `ngramSpeculation` GoogleCloudAiplatformV1SpeculativeDecodingSpecNgramSpeculation — N-Gram speculation works by trying to find matching tokens in the previous prompt sequence and use those as speculation for generating new tokens. - `ngramSize` integer — The number of last N input tokens used as ngram to search/match against the previous prompt sequence. This is equal to the N in N-Gram. The default value is 3 if not specified. - `draftModelSpeculation` GoogleCloudAiplatformV1SpeculativeDecodingSpecDraftModelSpeculation — Draft model speculation works by using the smaller model to generate candidate tokens for speculative decoding. - `draftModel` string — Required. The resource name of the draft model. - `createTime` string, google-datetime — Output only. Timestamp when the DeployedModel was created. - `automaticResources` GoogleCloudAiplatformV1AutomaticResources — A description of resources that to large degree are decided by Agent Platform, and require only a modest additional configuration. Each Model supporting these resources documents its specific guidelines. - `minReplicaCount` integer — Immutable. The minimum number of replicas that will be always deployed on. If traffic against it increases, it may dynamically be deployed onto more replicas up to max_replica_count, and as traffic decreases, some of these extra replicas may be freed. If the requested value is too large, the deployment will error. - `maxReplicaCount` integer — Immutable. The maximum number of replicas that may be deployed on when the traffic against it increases. If the requested value is too large, the deployment will error, but if deployment succeeds then the ability to scale to that many replicas is guaranteed (barring service outages). If traffic increases beyond what its replicas at maximum may handle, a portion of the traffic will be dropped. If this value is not provided, a no upper bound for scaling under heavy traffic will be assume, though Agent Platform may be unable to scale beyond certain replica number. - `disableExplanations` boolean — If true, deploy the model without explainable feature, regardless the existence of Model.explanation_spec or explanation_spec. - `fasterDeploymentConfig` GoogleCloudAiplatformV1FasterDeploymentConfig — Configuration for faster model deployment. - `fastTryoutEnabled` boolean — If true, enable fast tryout feature for this deployed model. - `trafficSplit` object — A map from a DeployedModel's ID to the percentage of this Endpoint's traffic that should be forwarded to that DeployedModel. If this field is non-empty, then the Endpoint's traffic_split will be overwritten with it. To refer to the ID of the just being deployed Model, a "0" should be used, and the actual ID of the new DeployedModel will be filled in its place by this method. The traffic percentage values must add up to 100. If this field is empty, then the Endpoint's traffic_split is not updated. ## Response `200` Successful response --- [API](https://skmtc.net/google/apis/aiplatform.md) · [All operations](https://skmtc.net/google/apis/aiplatform/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/google/aiplatform/versions/b608d71b91f0/schema)