gcs_wait>: Wait for a file in Google Cloud Storage

Copy

• Copy for LLM

  Copy page as Markdown for LLMs
• View as Markdown

  Open this page as Markdown
• Open in ChatGPT

  Get insights from ChatGPT
• Open in Claude

  Get insights from Claude
• Connect to Cursor

  Install MCP server on Cursor
• Connect to VS Code

  Install MCP server on VS Code

gcs_wait> operator can be used to wait for file to appear in Google Cloud Storage.

+wait:
  gcs_wait>: my_bucket/some/file

+wait:
  gcs_wait>: gs://my_bucket/some/file

Secrets

When you don't know how to set secrets, please refer to Managing Workflow Secret

• gcp.credential: CREDENTIAL

  See gcp.credential.

Options

• gcs_wait>: URI | BUCKET/OBJECT

  Google Cloud Storage URI or path of the file to wait for.

  Examples:

  gcs_wait>: my-bucket/my-directory/my-data.gz

Examples:

  gcs_wait>: gs://my-bucket/my-directory/my-data.gz

• bucket: NAME

  The GCS bucket where the file is located. Can be used together with the object parameter instead of putting the path on the operator command line.

• object: PATH

  The GCS path of the file. Can be used together with the bucket parameter instead of putting the path on the operator command line.

Output parameters

• gcs_wait.last_object

  Information about the detected file.

{
  "metadata": {
    "bucket": "my_bucket",
    "contentType": "text/plain",
    "crc32c": "yV/Pdw==",
    "etag": "CKjJ6/H4988CEAE=",
    "generation": 1477466841081000,
    "id": "my_bucket/some/file",
    "kind": "storage#object",
    "md5Hash": "IT4zYwc3D23HpSGe3nZ85A==",
    "mediaLink": "https://www.googleapis.com/download/storage/v1/b/my_bucket/o/some%2Ffile?generation=1477466841081000&alt=media",
    "metageneration": 1,
    "name": "some/file",
    "selfLink": "https://www.googleapis.com/storage/v1/b/my_bucket/o/some%2Ffile",
    "size": 4711,
    "storageClass": "STANDARD",
    "timeCreated": {
      "value": 1477466841070,
      "dateOnly": false,
      "timeZoneShift": 0
    },
    "updated": {
      "value": 1477466841070,
      "dateOnly": false,
      "timeZoneShift": 0
    }
  }
}

Note: The gcs_wait> operator makes use of polling with exponential backoff. As such there might be some time interval between a file being created and the gcs_wait> operator detecting it.