Sign in

v1

latestOpenAPI 3.0.22026-08-062794201004.1 KB
Models (AI Data Plane)

Create Model

Create a new model deployment.

In order to access this endpoint, the provided API key must have at least one of the following roles:

  • Organization Owner

To learn more, see Organization, Project, and Database Access Overview.

post/v4/organizations/{organizationId}/aiServices/models

Request body

namestring required

Name of the model.

catalogModelNamestring required

Name of the model deployed from the model catalog.

quantization'fp8' | 'fp16' | 'fullPrecision'

Quantization options for the model. Options include 8-bit, 16bit, and full-precision.

optimization'throughput' | 'latency'

Optimization profile option for the model.

dimensionsinteger

Dimensions specify the vector dimensions for the underlying embedding model.

guardrailsstring[]

List of guardrail categories as plain text strings. These will be formatted into a template and base64-encoded internally.

enableBatchingboolean

Option to enable batching.

keywordFilteringstring[]

Keywords in a comma-separated string to filter the input.

Example request

{
  "name": "my-new-model",
  "catalogModelName": "meta-llama/Llama-3.1-8B-Instruct",
  "cloudConfig": {
    "provider": "aws",
    "region": "us-east-1",
    "compute": {
      "cpu": 4,
      "gpuMemory": 48
    }
  },
  "quantization": "fullPrecision",
  "optimization": "latency",
  "guardrails": [
    "sexual crime",
    "hate speech"
  ],
  "jailbreak": {
    "scoreThreshold": 0.9
  },
  "caching": {
    "enableStandard": true,
    "semantic": {
      "embeddingModel": "my-embedding-model-id",
      "scoreThreshold": 0.75,
      "dimensions": 4096,
      "distanceMetric": "dot_product"
    },
    "defaultCache": "semantic",
    "expiryTTL": 1000
  },
  "enableBatching": true,
  "keywordFiltering": [
    "harassment",
    "murder"
  ]
}

Response

Model queued for deployment successfully.

idstring uuid

Unique identifier for the created language model.

Example response

{
  "id": "fffffffff-aaaa-1414-eeee-000000000000"
}