Go Client for TD-API

This guide covers how to use the Go programming language module for the td-client API.

Prerequisites

Installation

Install the td-client via the go command line.

Copy
Copied
go get github.com/treasure-data/td-client-go

Setup

Initialize a client object using your TD API key. Best practice is to pull the API key from an environment variable.

Copy
Copied
import (
    td_client "github.com/treasure-data/td-client-go"
    "time"
)

func main() {
    apiKey := os.Getenv("TD_CLIENT_API_KEY")
    client, err := td_client.NewTDClient(td_client.Settings{
        ApiKey: apiKey,
    })
    if err != nil {
        fmt.Println(err.Error())
        return
    }
// code to be continued in the rest of this article

Basic Use

List Databases and Tables

The code below demonstrates how to list databases and tables from your Treasure Data account.

Copy
Copied
databases, err := client.ListDatabases()
if err != nil {
    fmt.Println(err.Error())
    return
}
fmt.Printf("%d databases\n", len(*databases))
for _, database := range *databases {
    fmt.Printf("  name: %s\n", database.Name)
    tables, err := client.ListTables(database.Name)
    if err != nil {
        fmt.Println(err.Error())
        return
    }
    fmt.Printf("  %d tables\n", len(*tables))
    for _, table := range *tables {
        fmt.Printf("    name: %s\n", table.Name)
        fmt.Printf("    type: %s\n", table.Type)
        fmt.Printf("    count: %d\n", table.Count)
        fmt.Printf("    primaryKey: %s\n", table.PrimaryKey)
        fmt.Printf("    schema: %v\n", table.Schema)
    }
}

Issue Queries

The example below issues a query from a Go program. The query API is asynchronous--you can check for query completion by polling the job periodically (e.g. by checking the response for JobStatus(jobID) calls).

Copy
Copied
jobId, err := client.SubmitQuery("sample_db2", td_client.Query{
    Type:       "hive", // can also use "presto"
    Query:      "SELECT COUNT(*) AS c FROM test WHERE a >= 5000",
    ResultUrl:  "", // can use the Result Output feature
    Priority:   0,
    RetryLimit: 0,
})
if err != nil {
    fmt.Println(err.Error())
    return
}
for {
    time.Sleep(1000000000)
    status, err := client.JobStatus(jobId)
    if err != nil {
        fmt.Println(err.Error())
        return
    }
    fmt.Printf("jobStatus:%s\n", status)
    if status != "queued" && status != "running" {
        break
    }
}

Get the Status of Jobs

The example below gets the status of a job by jobId.

Copy
Copied
// get job result
jobDesc, err := client.ShowJob(jobId)
if err != nil {
    fmt.Println(err.Error())
    return
}
fmt.Printf("query:%s\n", jobDesc.Query)
fmt.Printf("debug.cmdOut:%s\n", jobDesc.Debug.CmdOut)
fmt.Printf("debug.stdErr:%s\n", jobDesc.Debug.StdErr)
fmt.Printf("url:%s\n", jobDesc.Url)
fmt.Printf("startAt:%s\n", jobDesc.StartAt.String())
fmt.Printf("endAt:%s\n", jobDesc.EndAt.String())
fmt.Printf("cpuTime:%g\n", jobDesc.CpuTime)
fmt.Printf("resultSize:%d\n", jobDesc.ResultSize)
fmt.Printf("priority:%d\n", jobDesc.Priority)
fmt.Printf("hiveResultSchema:%v\n", jobDesc.HiveResultSchema)

Further Reading