Treasure Data TD-API Quickstart

The TD-API makes it easy to integrate with other tools to better manipulate your data. Through the TD API you can programatically access the functionality typically provided through the web based TD Console.

This functionality includes:

• Manage bulk imports of data
• List databases and tables
• Schedule jobs
• Manage connectors
• Manage users
• Access control policies

By leveraging the TD-API endpoints you can create an external application, tool, or CLI to automate common tasks. For example, you may want to integrate TD-API with your continous deployment service to run tests or setup complex automations.

Setup

Get Your API Key

Before you can use TD-API, you'll need to get your API key, which you'll use to authenticate with the API.

You can find your master or write-only key in your profile via TD Console or using TD Toolbelt.

Find your Endpoint

Confirm the appropriate endpoint for your region, for example The US is in.treasuredata.com.

Basic Use

List Databases

In this case, you want to return a list of databases. This example issues a call to the API to return a list of your databases.

Sample Request

curl -H "AUTHORIZATION: TD1 YOUR_API_KEY_HERE" \ 
          "https://api.treasuredata.com/v3/database/list"

Sample Response

{
  "databases": [
    {
      "name": "database1",
      "count": 5000,
      "created_at": "2013-11-01 16:48:41 -0700",
      "updated_at": "2013-11-01 16:48:41 -0700",
      "organization": null
    },
    {
      "name": "database2",
      "count": 5000,
      "created_at": "2013-11-08 17:47:22 -0800",
      "updated_at": "2013-11-08 17:47:22 -0800",
      "organization": null
    }
  ]
}

Issue a Query

You might want to run a query to count how many visitors have come to your website.

URI: https://api.treasuredata.com/v3/job/issue/presto/td_audit_log

Sample Request

curl --location --request POST          
     'https://api.treasuredata.com/v3/job/issue/presto/td_audit_log' \
     --header 'AUTHORIZATION: TD1 YOUR_API_KEY_HERE' \
     --header 'Content-Type: application/json' \
     --data-raw '{
       "query": "select * from access limit 5",
       "domain_key": 261,
       "Priority":0
}

Sample Response

{
  "job": "1120145314",
  "job_id": "1120145314",
  "database": "td_audit_log"
}

Get Query Results

You might want to get query results for a specific job.

Sample Request

curl --location --request GET 'https://api.treasuredata.com/v3/job/result/1145768692?format=json' \
    --header 'AUTHORIZATION: TD1 YOUR_API_KEY_HERE' \
    --header 'Content-Type: application/json' \

Sample Response

[1,"400","num_requests",1601378369]
[2,"404","http_status_code",1601378369]
[3,"200","num_requests",1601378369]

List Connectors

You might need to retrieve a list of connectors or to check if one exists or see if one has been added.

Sample Request

curl -I -X GET \
  https://api.treasuredata.com/v3/result/list \
  -H 'Authorization: TD1 YOUR_API_KEY_HERE'

Sample Response

{
    "name": "mb_s3_sep_6",
    "url": "s3://test:xxx@s3-us-west-1.amazonaws.com/",
    "organization": null
}

Further Reading

Now that you have the basics down, here are some suggested resources for you.

• Tutorial - Use a continuous integration / continuous deployment service (CI / CD) tools such as Travis CI or Circle CI with TD-API
• Treasure Toolbelt - Command Line Interface for Treasure Data services.

Treasure Data Clients - native language wrappers for the TD-API REST API.

• Python td-client
• Ruby td-client
• Java td-client
• NodeJS td-client
• Go td-client