Treasure Data TD-API Quickstart

The TD-API makes it easy to integrate with other tools to better manipulate your data. Through the TD API you can programatically access the functionality typically provided through the web based TD Console.

This functionality includes:

  • Manage bulk imports of data
  • List databases and tables
  • Schedule jobs
  • Manage connectors
  • Manage users
  • Access control policies

By leveraging the TD-API endpoints you can create an external application, tool, or CLI to automate common tasks. For example, you may want to integrate TD-API with your continous deployment service to run tests or setup complex automations.

Setup

Get Your API Key

Before you can use TD-API, you'll need to get your API key, which you'll use to authenticate with the API.

You can find your master or write-only key in your profile via TD Console or using TD Toolbelt.

API Key Type Description
master Can be used to perform all permitted operations based on the user’s permission level and access, no exception.
write-only Provides an additional layer of security in controlling access to a Treasure Data account through the REST APIs.

Useful when access has to be provided to 3rd parties or API keys need to be embedded in ingestion libraries (for example, web page integrations). Based on the permissions and access levels associated with a user, the user’s Write-only API key only allows importing data into Treasure Data to those databases it has write access to.

Find your Endpoint

Confirm the appropriate endpoint for your region, for example The US is in.treasuredata.com.

Basic Use

List Databases

In this case, you want to return a list of databases. This example issues a call to the API to return a list of your databases.

Sample Request

Copy
Copied
curl -H "AUTHORIZATION: TD1 YOUR_API_KEY_HERE" \ 
          "https://api.treasuredata.com/v3/database/list"

Sample Response

Copy
Copied
{
  "databases": [
    {
      "name": "database1",
      "count": 5000,
      "created_at": "2013-11-01 16:48:41 -0700",
      "updated_at": "2013-11-01 16:48:41 -0700",
      "organization": null
    },
    {
      "name": "database2",
      "count": 5000,
      "created_at": "2013-11-08 17:47:22 -0800",
      "updated_at": "2013-11-08 17:47:22 -0800",
      "organization": null
    }
  ]
}

Issue a Query

You might want to run a query to count how many visitors have come to your website.

URI: https://api.treasuredata.com/v3/job/issue/presto/td_audit_log

Sample Request

Copy
Copied
curl --location --request POST          
     'https://api.treasuredata.com/v3/job/issue/presto/td_audit_log' \
     --header 'AUTHORIZATION: TD1 YOUR_API_KEY_HERE' \
     --header 'Content-Type: application/json' \
     --data-raw '{
       "query": "select * from access limit 5",
       "domain_key": 261,
       "Priority":0
}

Sample Response

Copy
Copied
{
  "job": "1120145314",
  "job_id": "1120145314",
  "database": "td_audit_log"
}

Get Query Results

You might want to get query results for a specific job.

Sample Request

Copy
Copied
curl --location --request GET 'https://api.treasuredata.com/v3/job/result/1145768692?format=json' \
    --header 'AUTHORIZATION: TD1 YOUR_API_KEY_HERE' \
    --header 'Content-Type: application/json' \

Sample Response

Copy
Copied
[1,"400","num_requests",1601378369]
[2,"404","http_status_code",1601378369]
[3,"200","num_requests",1601378369]

List Connectors

You might need to retrieve a list of connectors or to check if one exists or see if one has been added.

Sample Request

Copy
Copied
curl -I -X GET \
  https://api.treasuredata.com/v3/result/list \
  -H 'Authorization: TD1 YOUR_API_KEY_HERE'

Sample Response

Copy
Copied
{
    "name": "mb_s3_sep_6",
    "url": "s3://test:xxx@s3-us-west-1.amazonaws.com/",
    "organization": null
}

Further Reading

Now that you have the basics down, here are some suggested resources for you.

  • Tutorial - Use a continuous integration / continuous deployment service (CI / CD) tools such as Travis CI or Circle CI with TD-API
  • Treasure Toolbelt - Command Line Interface for Treasure Data services.

Treasure Data Clients - native language wrappers for the TD-API REST API.