--- title: "POST /v1b3/projects/{projectId}/locations/{location}/WorkerMessages" method: POST path: "/v1b3/projects/{projectId}/locations/{location}/WorkerMessages" tags: ["projects"] --- # POST /v1b3/projects/{projectId}/locations/{location}/WorkerMessages `POST /v1b3/projects/{projectId}/locations/{location}/WorkerMessages` Send a worker_message to the service. ## Path parameters - `projectId` string, required - `location` string, required ## Request body - SendWorkerMessagesRequest — A request for sending worker messages to the service. - `workerMessages` WorkerMessage[] — The WorkerMessages to send. - `workerMetrics` ResourceUtilizationReport — Worker metrics exported from workers. This contains resource utilization metrics accumulated from a variety of sources. For more information, see go/df-resource-signals. - `gpuUsage` GPUUsage[] — Optional. GPU usage samples. - `timestamp` string, google-datetime — Required. Timestamp of the measurement. - `utilization` GPUUtilization — Utilization details about the GPU. - `rate` number, double — Required. GPU utilization rate of any kernel over the last sample period in the range of [0, 1]. - `cpuTime` CPUTime[] — CPU utilization samples. - `timestamp` string, google-datetime — Timestamp of the measurement. - `rate` number, double — Average CPU utilization rate (% non-idle cpu / second) since previous sample. - `totalMs` string, uint64 — Total active CPU time across all cores (ie., non-idle) in milliseconds since start-up. - `memoryInfo` MemInfo[] — Memory utilization samples. - `totalGbMs` string, uint64 — Total memory (RSS) usage since start up in GB * ms. - `currentOoms` string, int64 — Number of Out of Memory (OOM) events recorded since the previous measurement. - `currentLimitBytes` string, uint64 — Instantenous memory limit in bytes. - `timestamp` string, google-datetime — Timestamp of the measurement. - `currentRssBytes` string, uint64 — Instantenous memory (RSS) size in bytes. - `containers` object — Per container information. Key: container name. - `time` string, google-datetime — The timestamp of the worker_message. - `workerShutdownNotice` WorkerShutdownNotice — Shutdown notification from workers. This is to be sent by the shutdown script of the worker VM so that the backend knows that the VM is being shut down. - `reason` string — The reason for the worker shutdown. Current possible values are: "UNKNOWN": shutdown reason is unknown. "PREEMPTION": shutdown reason is preemption. Other possible reasons may be added in the future. - `perWorkerMetrics` PerWorkerMetrics — Per worker metrics. - `perStepNamespaceMetrics` PerStepNamespaceMetrics[] — Optional. Metrics for a particular unfused step and namespace. - `metricsNamespace` string — The namespace of these metrics on the worker. - `originalStep` string — The original system name of the unfused step that these metrics are reported from. - `metricValues` MetricValue[] — Optional. Metrics that are recorded for this namespace and unfused step. - `valueGauge64` DataflowGaugeValue — The gauge value of a metric. - `value` string, int64 — The value of the gauge. - `measuredTime` string, google-datetime — The timestamp when the gauge was recorded. - `metric` string — Base name for this metric. - `valueInt64` string, int64 — Integer value of this metric. - `valueHistogram` DataflowHistogramValue — Summary statistics for a population of values. HistogramValue contains a sequence of buckets and gives a count of values that fall into each bucket. Bucket boundares are defined by a formula and bucket widths are either fixed or exponentially increasing. - `bucketCounts` string[] — Optional. The number of values in each bucket of the histogram, as described in `bucket_options`. `bucket_counts` should contain N values, where N is the number of buckets specified in `bucket_options`. If `bucket_counts` has fewer than N values, the remaining values are assumed to be 0. - `bucketOptions` BucketOptions — `BucketOptions` describes the bucket boundaries used in the histogram. - … - `outlierStats` OutlierStats — Statistics for the underflow and overflow bucket. - … - `count` string, int64 — Number of values recorded in this histogram. - `metricLabels` object — Optional. Set of metric labels for this metric. - `streamingScalingReport` StreamingScalingReport — Contains per-user worker telemetry used in streaming autoscaling. - `activeThreadCount` integer — Current acive thread count. - `outstandingBytes` string, int64 — Current outstanding bytes. - `maximumBytesCount` integer - `outstandingBundleCount` integer — Current outstanding bundle count. - `maximumBytes` string, int64 — Maximum bytes. - `activeBundleCount` integer - `outstandingBytesCount` integer - `maximumThreadCount` integer — Maximum thread count limit. - `maximumBundleCount` integer — Maximum bundle count. - `workerHealthReport` WorkerHealthReport — WorkerHealthReport contains information about the health of a worker. The VM should be identified by the labels attached to the WorkerMessage that this health ping belongs to. - `vmIsBroken` boolean — Whether the VM is in a permanently broken state. Broken VMs should be abandoned or deleted ASAP to avoid assigning or completing any work. - `msg` string — Message describing any unusual health reports. - `reportInterval` string, google-duration — The interval at which the worker is sending health reports. The default value of 0 should be interpreted as the field is not being explicitly set by the worker. - `vmIsHealthy` boolean — Whether the VM is currently healthy. - `vmStartupTime` string, google-datetime — The time the VM was booted. - `vmBrokenCode` string — Code to describe a specific reason, if known, that a VM has reported broken state. - `pods` object[] — The pods running on the worker. See: http://kubernetes.io/v1.1/docs/api-reference/v1/definitions.html#_v1_pod This field is used by the worker to send the status of the indvidual containers running on each worker. - `workerMessageCode` WorkerMessageCode — A message code is used to report status and error messages to the service. The message codes are intended to be machine readable. The service will take care of translating these into user understandable messages if necessary. Example use cases: 1. Worker processes reporting successful startup. 2. Worker processes reporting specific errors (e.g. package staging failure). - `parameters` object — Parameters contains specific information about the code. This is a struct to allow parameters of different types. Examples: 1. For a "HARNESS_STARTED" message parameters might provide the name of the worker and additional data like timing information. 2. For a "GCS_DOWNLOAD_ERROR" parameters might contain fields listing the Cloud Storage objects being downloaded and fields containing errors. In general complex data structures should be avoided. If a worker needs to send a specific and complicated data structure then please consider defining a new proto and adding it to the data oneof in WorkerMessageResponse. Conventions: Parameters should only be used for information that isn't typically passed as a label. hostname and other worker identifiers should almost always be passed as labels since they will be included on most messages. - `code` string — The code is a string intended for consumption by a machine that identifies the type of message being sent. Examples: 1. "HARNESS_STARTED" might be used to indicate the worker harness has started. 2. "GCS_DOWNLOAD_ERROR" might be used to indicate an error downloading a Cloud Storage file as part of the boot process of one of the worker containers. This is a string and not an enum to make it easy to add new codes without waiting for an API change. - `workerLifecycleEvent` WorkerLifecycleEvent — A report of an event in a worker's lifecycle. The proto contains one event, because the worker is expected to asynchronously send each message immediately after the event. Due to this asynchrony, messages may arrive out of order (or missing), and it is up to the consumer to interpret. The timestamp of the event is in the enclosing WorkerMessage proto. - `event` 'UNKNOWN_EVENT' | 'OS_START' | 'CONTAINER_START' | 'NETWORK_UP' | 'STAGING_FILES_DOWNLOAD_START' | 'STAGING_FILES_DOWNLOAD_FINISH' | 'SDK_INSTALL_START' | 'SDK_INSTALL_FINISH' — The event being reported. - `containerStartTime` string, google-datetime — The start time of this container. All events will report this so that events can be grouped together across container/VM restarts. - `metadata` object — Other stats that can accompany an event. E.g. { "downloaded_bytes" : "123456" } - `workerThreadScalingReport` WorkerThreadScalingReport — Contains information about the thread scaling information of a worker. - `currentThreadCount` integer — Current number of active threads in a worker. - `dataSamplingReport` DataSamplingReport — Contains per-worker telemetry about the data sampling feature. - `elementsSampledCount` string, int64 — Optional. Delta of number of elements sampled from previous report. - `exceptionsSampledCount` string, int64 — Optional. Delta of number of samples taken from user code exceptions from previous report. - `pcollectionsSampledCount` string, int64 — Optional. Delta of number of PCollections sampled from previous report. - `bytesWrittenDelta` string, int64 — Optional. Delta of bytes written to file from previous report. - `elementsSampledBytes` string, int64 — Optional. Delta of bytes sampled from previous report. - `translationErrorsCount` string, int64 — Optional. Delta of errors counts from retrieving, or translating the samples from previous report. - `persistenceErrorsCount` string, int64 — Optional. Delta of errors counts from persisting the samples from previous report. - `labels` object — Labels are used to group WorkerMessages. For example, a worker_message about a particular container might have the labels: { "JOB_ID": "2015-04-22", "WORKER_ID": "wordcount-vm-2015…" "CONTAINER_TYPE": "worker", "CONTAINER_ID": "ac1234def"} Label tags typically correspond to Label enum values. However, for ease of development other strings can be used as tags. LABEL_UNSPECIFIED should not be used here. - `location` string — The [regional endpoint] (https://cloud.google.com/dataflow/docs/concepts/regional-endpoints) that contains the job. ## Response `200` Successful response --- [API](https://skmtc.net/google/apis/dataflow.md) · [All operations](https://skmtc.net/google/apis/dataflow/llms.txt) · [OpenAPI document](https://skmtc-service-staging.skmtc.workers.dev/v1/apis/google/dataflow/versions/f065de7a0dba/schema)