Treasure Workflow Quickstart

Treasure Workflows is a workflow management system based on Digdag that allows you to orchestrate data processing tasks, queries, and integrations in the cloud.

Getting Started

Treasure Workflows enables you to define and execute complex data processing pipelines using YAML configuration files. You can schedule workflows, handle dependencies, and integrate with various services.

Operator References

• Workflow Control and Functional Operators - Control flow, loops, conditionals, and task management
• Scripting Operators - Execute Python code using custom scripts
• Treasure Data Operators - Query, load, and export data from Treasure Data
• Database Operators - Work with PostgreSQL
• Amazon Web Services Operators - Integrate with AWS
• Google Cloud Platform Operators - Integrate with Google Cloud Platform
• Network Operators - Send emails and make HTTP requests

Digdag vs Treasure Workflow

Treasure Workflow currently allows for most of the functionality that Digdag, the underlying open source project, allows. But, there are a few exceptions. Some Digdag operators and functionality are not yet enabled when you submit workflows to the Treasure Workflow cloud environment. The following options are not allowed because shared processing and local disk are used:

• embulk> : Use embulk> for running arbitrary embulk jobs. However, you can use td_load> to import bulk data into Treasure Data.
• download_file : Typically, you might use the download_file parameter with the td> and other operators for downloading files locally. Instead, you can use the normal Treasure Data result export functionality.

Next Steps

1. Review the Workflow Definition guide to understand workflow syntax
2. Explore specific operator documentation for your use case
3. Check out example workflows in the Treasure Data Console