Skip to content

Treasure Data API (3)

Download OpenAPI description
td-api.json
td-api.yaml
Languages
Servers
https://api.treasuredata.com/v3/
https://api.treasuredata.co.jp/v3/
https://api.ap02.treasuredata.com/v3/
https://api.eu01.treasuredata.com/v3/

Access Control - Policies

Operations

Access Control - Permissions

Operations

Access Control - Users

Operations

Access Control - Policy Groups

Operations

Connections

Operations

Databases

Operations

Jobs

Operations

Get jobs

Request

Retrieves a list of jobs.

Security
ApiKeyAuth
curl -i -X GET \
  https://api.treasuredata.com/v3/job/list \
  -H 'Authorization: TD1 <YOUR_TOKEN_HERE>'

Responses

OK

Bodyapplication/json
jobsArray of objects(Job)required
jobs[].​cpu_timeinteger or nullrequired
jobs[].​created_atstring(DateTimeAsString)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s(([+-]\...required

Datetime in custom format (2020-12-31 11:22:33 -0500 or 2020-12-31 11:22:33 UTC for UTC)

Example: "2020-12-31 11:22:33 UTC"
jobs[].​databasestring or nullrequired

database name

jobs[].​durationinteger or nullrequired

Duration

Example: 10
jobs[].​end_atDateTimeAsString (string) or EmptyString (string)required
One of:

Datetime in custom format (2020-12-31 11:22:33 -0500 or 2020-12-31 11:22:33 UTC for UTC)

string(DateTimeAsString)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s(([+-]\...
jobs[].​hive_result_schemastring or nullrequired

the schema for the job's result table

jobs[].​job_idstring(JobIdAsString)required

unique identifier of job

Example: "19"
jobs[].​linked_result_export_job_idJobId (integer) or (integer or null)required
Any of:

unique identifier of job

integer(JobId)
jobs[].​num_recordsinteger or nullrequired

the number of records in the result. Will be null until job has finished, and may not be set for all query types

Example: 100
jobs[].​priorityinteger[ -2 .. 2 ]required

job execution priority

Example: 0
jobs[].​query(string or null) or QueryAsString (string) or BulkLoadConfig (object)required
Any of:
string or null
jobs[].​resultstring

...?

jobs[].​result_export_target_job_idJobId (integer) or (integer or null)required
Any of:

unique identifier of job

integer(JobId)
jobs[].​result_sizeinteger or nullrequired

size in bytes of the job's result, in msgpack.gz format

Example: 1048576
jobs[].​retry_limitinteger[ 0 .. 32 ]required

automatically retry this many times on failure

Default 0
Example: 3
jobs[].​start_atDateTimeAsString (string) or EmptyString (string)required
One of:

Datetime in custom format (2020-12-31 11:22:33 -0500 or 2020-12-31 11:22:33 UTC for UTC)

string(DateTimeAsString)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s(([+-]\...
jobs[].​statusstringrequired

current status of the job

Enum"queued""running""killed""success""error"
Example: "success"
jobs[].​typestringrequired
Example: "presto"
jobs[].​updated_atstring(DateTimeAsString)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s(([+-]\...required

Datetime in custom format (2020-12-31 11:22:33 -0500 or 2020-12-31 11:22:33 UTC for UTC)

Example: "2020-12-31 11:22:33 UTC"
jobs[].​urlstringrequired
jobs[].​user_namestringrequired

user's full name or email

Example: "Treasure Data"
jobs[].​organizationstring or nullDeprecatedrequired

always "null"

countintegerrequired
frominteger or nullrequired
tointeger or nullrequired
Response
application/json
{
  "jobs": [
    { … }
  ],
  "count": 0,
  "from": 0,
  "to": 0
}

Get job

Request

Retrieves a list of jobs with specified job id.

Security
ApiKeyAuth
Path
job_idinteger(JobId)required

the job id

Example: 19
curl -i -X GET \
  https://api.treasuredata.com/v3/job/show/19 \
  -H 'Authorization: TD1 <YOUR_TOKEN_HERE>'

Responses

OK

Bodyapplication/json
cpu_timeinteger or nullrequired
created_atstring(DateTimeAsString)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s(([+-]\...required

Datetime in custom format (2020-12-31 11:22:33 -0500 or 2020-12-31 11:22:33 UTC for UTC)

Example: "2020-12-31 11:22:33 UTC"
databasestring or nullrequired

database name

durationinteger or nullrequired

Duration

Example: 10
end_atDateTimeAsString (string) or EmptyString (string)required
One of:

Datetime in custom format (2020-12-31 11:22:33 -0500 or 2020-12-31 11:22:33 UTC for UTC)

string(DateTimeAsString)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s(([+-]\...
hive_result_schemastring or nullrequired

the schema for the job's result table

job_idstring(JobIdAsString)required

unique identifier of job

Example: "19"
linked_result_export_job_idJobId (integer) or (integer or null)required
Any of:

unique identifier of job

integer(JobId)
num_recordsinteger or nullrequired

the number of records in the result. Will be null until job has finished, and may not be set for all query types

Example: 100
priorityinteger[ -2 .. 2 ]required

job execution priority

Example: 0
query(string or null) or QueryAsString (string) or BulkLoadConfig (object)required
Any of:
string or null
resultstring

...?

result_export_target_job_idJobId (integer) or (integer or null)required
Any of:

unique identifier of job

integer(JobId)
result_sizeinteger or nullrequired

size in bytes of the job's result, in msgpack.gz format

Example: 1048576
retry_limitinteger[ 0 .. 32 ]required

automatically retry this many times on failure

Default 0
Example: 3
start_atDateTimeAsString (string) or EmptyString (string)required
One of:

Datetime in custom format (2020-12-31 11:22:33 -0500 or 2020-12-31 11:22:33 UTC for UTC)

string(DateTimeAsString)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s(([+-]\...
statusstringrequired

current status of the job

Enum"queued""running""killed""success""error"
Example: "success"
typestringrequired
Example: "presto"
updated_atstring(DateTimeAsString)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s(([+-]\...required

Datetime in custom format (2020-12-31 11:22:33 -0500 or 2020-12-31 11:22:33 UTC for UTC)

Example: "2020-12-31 11:22:33 UTC"
urlstringrequired
user_namestringrequired

user's full name or email

Example: "Treasure Data"
debugobjectrequired
debug.​cmdoutstring or nullrequired
debug.​stderrstring or nullrequired
organizationstring or nullDeprecatedrequired

always "null"

Response
application/json
{
  "cpu_time": 0,
  "created_at": "2020-12-31 11:22:33 UTC",
  "database": "string",
  "duration": 10,
  "end_at": "2020-12-31 11:22:33 UTC",
  "hive_result_schema": "string",
  "job_id": "19",
  "linked_result_export_job_id": 19,
  "num_records": 100,
  "organization": "string",
  "priority": 0,
  "query": "string",
  "result": "string",
  "result_export_target_job_id": 19,
  "result_size": 1048576,
  "retry_limit": 3,
  "start_at": "2020-12-31 11:22:33 UTC",
  "status": "success",
  "type": "presto",
  "updated_at": "2020-12-31 11:22:33 UTC",
  "url": "string",
  "user_name": "Treasure Data",
  "debug": {
    "cmdout": "string",
    "stderr": "string"
  }
}

Get job status by id

Request

Show the status information of the given job id at last execution as well as other related information.

Security
ApiKeyAuth
Path
job_idinteger(JobId)required

the job id

Example: 19
curl -i -X GET \
  https://api.treasuredata.com/v3/job/status/19 \
  -H 'Authorization: TD1 <YOUR_TOKEN_HERE>'

Responses

OK

Bodyapplication/json
job_idstring(JobIdAsString)required

unique identifier of job

Example: "19"
cpu_timeinteger or nullrequired
created_atstring(DateTimeAsString)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s(([+-]\...required

Datetime in custom format (2020-12-31 11:22:33 -0500 or 2020-12-31 11:22:33 UTC for UTC)

Example: "2020-12-31 11:22:33 UTC"
end_atDateTimeAsString (string) or EmptyString (string)required
One of:

Datetime in custom format (2020-12-31 11:22:33 -0500 or 2020-12-31 11:22:33 UTC for UTC)

string(DateTimeAsString)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s(([+-]\...
durationinteger or nullrequired

Duration

Example: 10
num_recordsinteger or nullrequired

the number of records in the result. Will be null until job has finished, and may not be set for all query types

Example: 100
result_sizeinteger or nullrequired

size in bytes of the job's result, in msgpack.gz format

Example: 1048576
start_atDateTimeAsString (string) or EmptyString (string)required
One of:

Datetime in custom format (2020-12-31 11:22:33 -0500 or 2020-12-31 11:22:33 UTC for UTC)

string(DateTimeAsString)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s(([+-]\...
statusstringrequired

current status of the job

Enum"queued""running""killed""success""error"
Example: "success"
updated_atstring(DateTimeAsString)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s(([+-]\...required

Datetime in custom format (2020-12-31 11:22:33 -0500 or 2020-12-31 11:22:33 UTC for UTC)

Example: "2020-12-31 11:22:33 UTC"
Response
application/json
{
  "job_id": "19",
  "cpu_time": 0,
  "created_at": "2020-12-31 11:22:33 UTC",
  "end_at": "2020-12-31 11:22:33 UTC",
  "duration": 10,
  "num_records": 100,
  "result_size": 1048576,
  "start_at": "2020-12-31 11:22:33 UTC",
  "status": "success",
  "updated_at": "2020-12-31 11:22:33 UTC"
}

Get job status by domain_key

Request

Checks the status of a job using the domain key.

Security
ApiKeyAuth
Path
domain_keystring(DomainKey)required
Example: domain_key_123
curl -i -X GET \
  https://api.treasuredata.com/v3/job/status_by_domain_key/domain_key_123 \
  -H 'Authorization: TD1 <YOUR_TOKEN_HERE>'

Responses

OK

Bodyapplication/json
job_idstring(JobIdAsString)required

unique identifier of job

Example: "19"
cpu_timeinteger or nullrequired
created_atstring(DateTimeAsString)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s(([+-]\...required

Datetime in custom format (2020-12-31 11:22:33 -0500 or 2020-12-31 11:22:33 UTC for UTC)

Example: "2020-12-31 11:22:33 UTC"
end_atDateTimeAsString (string) or EmptyString (string)required
One of:

Datetime in custom format (2020-12-31 11:22:33 -0500 or 2020-12-31 11:22:33 UTC for UTC)

string(DateTimeAsString)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s(([+-]\...
durationinteger or nullrequired

Duration

Example: 10
num_recordsinteger or nullrequired

the number of records in the result. Will be null until job has finished, and may not be set for all query types

Example: 100
result_sizeinteger or nullrequired

size in bytes of the job's result, in msgpack.gz format

Example: 1048576
start_atDateTimeAsString (string) or EmptyString (string)required
One of:

Datetime in custom format (2020-12-31 11:22:33 -0500 or 2020-12-31 11:22:33 UTC for UTC)

string(DateTimeAsString)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s(([+-]\...
statusstringrequired

current status of the job

Enum"queued""running""killed""success""error"
Example: "success"
updated_atstring(DateTimeAsString)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s(([+-]\...required

Datetime in custom format (2020-12-31 11:22:33 -0500 or 2020-12-31 11:22:33 UTC for UTC)

Example: "2020-12-31 11:22:33 UTC"
Response
application/json
{
  "job_id": "19",
  "cpu_time": 0,
  "created_at": "2020-12-31 11:22:33 UTC",
  "end_at": "2020-12-31 11:22:33 UTC",
  "duration": 10,
  "num_records": 100,
  "result_size": 1048576,
  "start_at": "2020-12-31 11:22:33 UTC",
  "status": "success",
  "updated_at": "2020-12-31 11:22:33 UTC"
}

Get job statistics by id

Request

Show the statistics of the job, including records inserted.

Security
ApiKeyAuth
Path
job_idinteger(JobId)required

the job id

Example: 19
curl -i -X GET \
  https://api.treasuredata.com/v3/job/statistics/19 \
  -H 'Authorization: TD1 <YOUR_TOKEN_HERE>'

Responses

OK

Bodyapplication/json
records_insertedstring
Example: "28345"
Response
application/json
{
  "records_inserted": "28345"
}

Create job

Request

Creates a job of a given type under a specified database.

Security
ApiKeyAuth
Path
job_typestringrequired

the job type

Example: presto
database_namestringrequired

database name

Example: Alpha
Bodyapplication/json
querystring

the raw SQL query

Example: "SELECT COUNT(1) FROM www_access"
priorityinteger[ -2 .. 2 ]

job execution priority

Example: 0
resultstring

Export Connector location

retry_limitinteger[ 0 .. 32 ]

the number of automatic retries on failure

Example: 0
pool_namestring

resource pool name

property name*anyadditional property
curl -i -X POST \
  https://api.treasuredata.com/v3/job/issue/presto/Alpha \
  -H 'Authorization: TD1 <YOUR_TOKEN_HERE>' \
  -H 'Content-Type: application/json' \
  -d '{
    "query": "SELECT COUNT(1) FROM www_access",
    "priority": 0,
    "result": "string",
    "retry_limit": 0,
    "pool_name": "string"
  }'

Responses

OK

Bodyapplication/json
jobstring(JobIdAsString)required

unique identifier of job

Example: "19"
job_idstring(JobIdAsString)required

unique identifier of job

Example: "19"
databasestringrequired

database name

Example: "Alpha"
Response
application/json
{
  "job": "19",
  "job_id": "19",
  "database": "Alpha"
}

kill job by id

Request

Stops a specified job ID.

Security
ApiKeyAuth
Path
job_idinteger(JobId)required

the job id

Example: 19
curl -i -X POST \
  https://api.treasuredata.com/v3/job/kill/19 \
  -H 'Authorization: TD1 <YOUR_TOKEN_HERE>'

Responses

OK

Bodyapplication/json
One of:
object(EmptyObject)
Response
application/json
{}

Set result export

Request

Sets the results of a specified job ID.

Security
ApiKeyAuth
Path
job_idinteger(JobId)required

the job id

Example: 19
Bodyapplication/x-www-form-urlencoded
domain_keystring(DomainKey)
Example: "domain_key_123"
pool_namestring

resource pool name

Example: "pool_1"
priorityinteger[ -2 .. 2 ]

job execution priority

Example: 0
querystring(QueryAsString)

the raw SQL query

Example: "SELECT * FROM my_table;"
retry_limitinteger[ 0 .. 32 ]

automatically retry this many times on failure

Default 0
Example: 3
scheduled_timestring(DateTimeISO8601)\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(([+-]\d{2...

Datetime in ISO 8601 format (2020-12-31T11:22:33-05:00 or 2020-12-31T11:22:33Z for UTC)

Example: "2020-12-31T11:22:33Z"
target_job_idinteger(JobId)

unique identifier of job

Example: 19
result_connection_idinteger(ConnectionId)

the connection id

Example: 17
resultstring

...?

databasestring

database name

Example: "Alpha"
tablestring
Example: "Bravo"
curl -i -X POST \
  https://api.treasuredata.com/v3/job/result_export/19 \
  -H 'Authorization: TD1 <YOUR_TOKEN_HERE>' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d domain_key=domain_key_123 \
  -d pool_name=pool_1 \
  -d priority=0 \
  -d 'query=SELECT * FROM my_table;' \
  -d retry_limit=3 \
  -d scheduled_time=2020-12-31T11:22:33Z \
  -d target_job_id=19 \
  -d result_connection_id=17 \
  -d result=string \
  -d database=Alpha \
  -d table=Bravo

Responses

OK

Bodyapplication/json
cpu_timeinteger or nullrequired
created_atstring(DateTimeAsString)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s(([+-]\...required

Datetime in custom format (2020-12-31 11:22:33 -0500 or 2020-12-31 11:22:33 UTC for UTC)

Example: "2020-12-31 11:22:33 UTC"
databasestring or nullrequired

database name

durationinteger or nullrequired

Duration

Example: 10
end_atDateTimeAsString (string) or EmptyString (string)required
One of:

Datetime in custom format (2020-12-31 11:22:33 -0500 or 2020-12-31 11:22:33 UTC for UTC)

string(DateTimeAsString)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s(([+-]\...
hive_result_schemastring or nullrequired

the schema for the job's result table

job_idstring(JobIdAsString)required

unique identifier of job

Example: "19"
linked_result_export_job_idJobId (integer) or (integer or null)required
Any of:

unique identifier of job

integer(JobId)
num_recordsinteger or nullrequired

the number of records in the result. Will be null until job has finished, and may not be set for all query types

Example: 100
priorityinteger[ -2 .. 2 ]required

job execution priority

Example: 0
query(string or null) or QueryAsString (string) or BulkLoadConfig (object)required
Any of:
string or null
resultstring

...?

result_export_target_job_idJobId (integer) or (integer or null)required
Any of:

unique identifier of job

integer(JobId)
result_sizeinteger or nullrequired

size in bytes of the job's result, in msgpack.gz format

Example: 1048576
retry_limitinteger[ 0 .. 32 ]required

automatically retry this many times on failure

Default 0
Example: 3
start_atDateTimeAsString (string) or EmptyString (string)required
One of:

Datetime in custom format (2020-12-31 11:22:33 -0500 or 2020-12-31 11:22:33 UTC for UTC)

string(DateTimeAsString)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s(([+-]\...
statusstringrequired

current status of the job

Enum"queued""running""killed""success""error"
Example: "success"
typestringrequired
Example: "presto"
updated_atstring(DateTimeAsString)\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2}\s(([+-]\...required

Datetime in custom format (2020-12-31 11:22:33 -0500 or 2020-12-31 11:22:33 UTC for UTC)

Example: "2020-12-31 11:22:33 UTC"
urlstringrequired
user_namestringrequired

user's full name or email

Example: "Treasure Data"
debugobjectrequired
debug.​cmdoutstring or nullrequired
debug.​stderrstring or nullrequired
organizationstring or nullDeprecatedrequired

always "null"

Response
application/json
{
  "cpu_time": 0,
  "created_at": "2020-12-31 11:22:33 UTC",
  "database": "string",
  "duration": 10,
  "end_at": "2020-12-31 11:22:33 UTC",
  "hive_result_schema": "string",
  "job_id": "19",
  "linked_result_export_job_id": 19,
  "num_records": 100,
  "organization": "string",
  "priority": 0,
  "query": "string",
  "result": "string",
  "result_export_target_job_id": 19,
  "result_size": 1048576,
  "retry_limit": 3,
  "start_at": "2020-12-31 11:22:33 UTC",
  "status": "success",
  "type": "presto",
  "updated_at": "2020-12-31 11:22:33 UTC",
  "url": "string",
  "user_name": "Treasure Data",
  "debug": {
    "cmdout": "string",
    "stderr": "string"
  }
}

Create export job

Request

Creates an export job.

Security
ApiKeyAuth
Path
database_namestringrequired

database name

Example: Alpha
table_namestringrequired
Example: Bravo
Bodyapplication/x-www-form-urlencoded
access_key_idstring
Example: "a_key"
assume_rolestring
bucketstring
Example: "test_bucket"
domain_keystring(DomainKey)
Example: "domain_key_123"
encryptionstring
file_formatstring
Enum"jsonl.gz""tsv.gz""json.gz""line-json.gz"
Example: "json.gz"
file_prefixstring
frominteger(Timestamp)
Example: 1356998400
pool_namestring
Example: "pool_1"
secret_access_keystring
Example: "secret_abc"
storage_typestring
Value"s3"
Example: "s3"
tointeger(Timestamp)
Example: 1356998400
curl -i -X POST \
  https://api.treasuredata.com/v3/export/run/Alpha/Bravo \
  -H 'Authorization: TD1 <YOUR_TOKEN_HERE>' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d access_key_id=a_key \
  -d assume_role=string \
  -d bucket=test_bucket \
  -d domain_key=domain_key_123 \
  -d encryption=string \
  -d file_format=json.gz \
  -d file_prefix=string \
  -d from=1356998400 \
  -d pool_name=pool_1 \
  -d secret_access_key=secret_abc \
  -d storage_type=s3 \
  -d to=1356998400

Responses

OK

Bodyapplication/json
jobstring(JobIdAsString)required

unique identifier of job

Example: "19"
job_idstring(JobIdAsString)required

unique identifier of job

Example: "19"
databasestringrequired

database name

Example: "Alpha"
Response
application/json
{
  "job": "19",
  "job_id": "19",
  "database": "Alpha"
}

Get job results

Request

Retrieves job results.

Security
ApiKeyAuth
Path
job_idinteger(JobId)required

the job id

Example: 19
Query
headerboolean

whether to show the headers or not

nullify_specialboolean

convert NaN and Infinity values into null. Not available for formats msgpack or msgpack.gz

curl -i -X GET \
  'https://api.treasuredata.com/v3/job/result/19?header=true&nullify_special=true' \
  -H 'Authorization: TD1 <YOUR_TOKEN_HERE>'

Responses

OK

Body
string(binary)

Create partial delete jobDeprecated

Request

Creates a partial delete job.

Security
ApiKeyAuth
Path
database_namestringrequired

database name

Example: Alpha
table_namestringrequired
Example: Bravo
Bodyapplication/x-www-form-urlencoded
domain_keystring(DomainKey)
Example: "domain_key_123"
fromTimestamp (integer) or DateTimeAsString (string) or DateTimeISO8601 (string)
Any of:
integer(Timestamp)
pool_namestring
Example: "pool_1"
toTimestamp (integer) or DateTimeAsString (string) or DateTimeISO8601 (string)
Any of:
integer(Timestamp)
curl -i -X POST \
  https://api.treasuredata.com/v3/table/partialdelete/Alpha/Bravo \
  -H 'Authorization: TD1 <YOUR_TOKEN_HERE>' \
  -H 'Content-Type: application/x-www-form-urlencoded' \
  -d domain_key=domain_key_123 \
  -d from=1356998400 \
  -d pool_name=pool_1 \
  -d to=1356998400

Responses

OK

Bodyapplication/json
job_idinteger(JobId)required

unique identifier of job

Example: 19
databasestringrequired

database name

Example: "Alpha"
tablestringrequired
Example: "Bravo"
frominteger(Timestamp)required
Example: 1356998400
tointeger(Timestamp)required
Example: 1356998400
Response
application/json
{
  "job_id": 19,
  "database": "Alpha",
  "table": "Bravo",
  "from": 1356998400,
  "to": 1356998400
}

Tables

Operations

Policy Group Tags

Operations

Schedules

Operations

Users

Operations