Treasure Data TD-API Quickstart
The TD-API makes it easy to integrate with other tools to better manipulate your data. Through the TD API you can programatically access the functionality typically provided through the web based TD Console.
This functionality includes:
- Manage bulk imports of data
- List databases and tables
- Schedule jobs
- Manage connectors
- Manage users
- Access control policies
By leveraging the TD-API endpoints you can create an external application, tool, or CLI to automate common tasks. For example, you may want to integrate TD-API with your continous deployment service to run tests or setup complex automations.
Setup
Get Your API Key
Before you can use TD-API, you'll need to get your API key, which you'll use to authenticate with the API.
You can find your master or write-only key in your profile via TD Console or using TD Toolbelt.
API Key Type | Description |
---|---|
master | Can be used to perform all permitted operations based on the user’s permission level and access, no exception. |
write-only | Provides an additional layer of security in controlling access to a Treasure Data account through the REST APIs. Useful when access has to be provided to 3rd parties or API keys need to be embedded in ingestion libraries (for example, web page integrations). Based on the permissions and access levels associated with a user, the user’s Write-only API key only allows importing data into Treasure Data to those databases it has write access to. |
Find your Endpoint
Confirm the appropriate endpoint for your region, for example The US is in.treasuredata.com
.
Basic Use
List Databases
In this case, you want to return a list of databases. This example issues a call to the API to return a list of your databases.
Sample Request
curl -H "AUTHORIZATION: TD1 YOUR_API_KEY_HERE" \
"https://api.treasuredata.com/v3/database/list"
Sample Response
{
"databases": [
{
"name": "database1",
"count": 5000,
"created_at": "2013-11-01 16:48:41 -0700",
"updated_at": "2013-11-01 16:48:41 -0700",
"organization": null
},
{
"name": "database2",
"count": 5000,
"created_at": "2013-11-08 17:47:22 -0800",
"updated_at": "2013-11-08 17:47:22 -0800",
"organization": null
}
]
}
Issue a Query
You might want to run a query to count how many visitors have come to your website.
URI: https://api.treasuredata.com/v3/job/issue/presto/td_audit_log
Sample Request
curl --location --request POST
'https://api.treasuredata.com/v3/job/issue/presto/td_audit_log' \
--header 'AUTHORIZATION: TD1 YOUR_API_KEY_HERE' \
--header 'Content-Type: application/json' \
--data-raw '{
"query": "select * from access limit 5",
"domain_key": 261,
"Priority":0
}
Sample Response
{
"job": "1120145314",
"job_id": "1120145314",
"database": "td_audit_log"
}
Get Query Results
You might want to get query results for a specific job.
Sample Request
curl --location --request GET 'https://api.treasuredata.com/v3/job/result/1145768692?format=json' \
--header 'AUTHORIZATION: TD1 YOUR_API_KEY_HERE' \
--header 'Content-Type: application/json' \
Sample Response
[1,"400","num_requests",1601378369]
[2,"404","http_status_code",1601378369]
[3,"200","num_requests",1601378369]
List Connectors
You might need to retrieve a list of connectors or to check if one exists or see if one has been added.
Sample Request
curl -I -X GET \
https://api.treasuredata.com/v3/result/list \
-H 'Authorization: TD1 YOUR_API_KEY_HERE'
Sample Response
{
"name": "mb_s3_sep_6",
"url": "s3://test:xxx@s3-us-west-1.amazonaws.com/",
"organization": null
}
Further Reading
Now that you have the basics down, here are some suggested resources for you.
- Tutorial - Use a continuous integration / continuous deployment service (CI / CD) tools such as Travis CI or Circle CI with TD-API
- Treasure Toolbelt - Command Line Interface for Treasure Data services.
Treasure Data Clients - native language wrappers for the TD-API REST API.