Building an API

Building an API

API stands for Application Programming Interface, it is just an interface to the webapp. When we use a browser to access a web application, we interact in HTTP and get back HTML pages, which the browser will render for us. Let's say we want to interact with our web app to get some data out of it using a host programming language like Go or Python. We'd have to maintain cookies in Python, or write a python module as an extension of a browser to handle this scenario.

There is a simple way out of this, we equip our web application itself to interact with any host language which can talk in HTTP. This way developers can easily get access to our system, using valid credentials, of course.

Browser:

1. We send the username,password and get a cookie stored on our machine.

2. We use the token in the cookie until it is valid to send HTTP requests.

3. The browser is responsible for rendering the HTML pages sent by the server.

API:

1. We send the username, password and get a token.

2. We send this token in each of our request to the server

Typically we send the token in a custom HTTP header called token.

When we use a browser, the server stores our information as a session, when we send it a request, it is aware of our session. A web app typically uses cookies to store the session ID, which is used to identify the user. Such a server is called a stateful server.

When we write APIs, they are stateless servers, they do not store sessions information anywhere on the server. To it, each request is unique. Which is why, we need to pass along the authentication token in each request.

Note: Don't mess around with tokens

There are apps where "single sign in" feature is available, the user has to log in only once and they are logged in forever, this is very dangerous. Because if a malicious person gets their hands on the security token, they can send malicious requests for data which look genuine and are impossible to clasify as malicious. Don't do this, always have some expiration time for security tokens, depends on your application really, two hours, six hours, but never infinite hours.

JWT

JSON Web Tokens is a standard for generating tokens. We will use the jwt-go library.

Lets start by defining our routes

http.HandleFunc("/api/get-task/", views.GetTasksFuncAPI)
http.HandleFunc("/api/get-deleted-task/", views.GetDeletedTaskFuncAPI)
http.HandleFunc("/api/add-task/", views.AddTaskFuncAPI)
http.HandleFunc("/api/update-task/", views.UpdateTaskFuncAPI)
http.HandleFunc("/api/delete-task/", views.DeleteTaskFuncAPI)

http.HandleFunc("/api/get-token/", views.GetTokenHandler)
http.HandleFunc("/api/get-category/", views.GetCategoryFuncAPI)
http.HandleFunc("/api/add-category/", views.AddCategoryFuncAPI)
http.HandleFunc("/api/update-category/", views.UpdateCategoryFuncAPI)
http.HandleFunc("/api/delete-category/", views.DeleteCategoryFuncAPI)

file: $GOPATH/src/github.com/thewhitetulip/Tasks/main.go

We will have the same URLs for the API, but it'll start with /api/

Our logic is that we will send the username and password in a POST request to /api/get-token/ that will return the token for us.

We are using jwt-go version 3. We have to define a custom claims struct which we will use in token generation and token verification. It'll contain the StandardClaims class and whichever extra fields we require.

type MyCustomClaims struct {
    Username string `json:"username"`
    jwt.StandardClaims
}

file: $GOPATH/src/github.com/thewhitetulip/Tasks/views/api.go

import "github.com/dgrijalva/jwt-go"
var mySigningKey = []byte("secret")


//GetTokenHandler will get a token for the username and password
func GetTokenHandler(w http.ResponseWriter, r *http.Request) {
    if r.Method != "POST" {
        w.Write([]byte("Method not allowed"))
        return
    }

    r.ParseForm()
    username := r.Form.Get("username")
    password := r.Form.Get("password")
    log.Println(username, " ", password)
    if username == "" || password == "" {
        w.Write([]byte("Invalid Username or password"))
        return
    }
    if db.ValidUser(username, password) {
        /* Set token claims */

        // Create the Claims
        claims := MyCustomClaims{
            username,
            jwt.StandardClaims{
                ExpiresAt: time.Now().Add(time.Hour * 5).Unix(),
            },
        }

        token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)

        /* Sign the token with our secret */
        tokenString, err := token.SignedString(mySigningKey)
        if err != nil {
            log.Println("Something went wrong with signing token")
            w.Write([]byte("Authentication failed"))
            return
        }

        /* Finally, write the token to the browser window */
        w.Write([]byte(tokenString))
    } else {
        w.Write([]byte("Authentication failed"))
    }
}

We need to send a POST request to /api/get-token/ with the username and password in the Form data.

If the HTTP method is anything other than POST, we throw an error, if the username and password is wrong we throw an error, otherwise, we generate a token and send it to the client.

The client will now use this token for future requests, the ValidateToken function will validate the token.

//ValidateToken will validate the token
func ValidateToken(myToken string) (bool, string) {
    token, err := jwt.ParseWithClaims(myToken, &MyCustomClaims{}, func(token *jwt.Token) (interface{}, error) {
        return []byte(mySigningKey), nil
    })

    if err != nil {
        return false, ""
    }

    claims := token.Claims.(*MyCustomClaims)
    return token.Valid, claims.Username
}

We'll call the Parse method on the token which we receive as a parameter in the function call. The token.Valid field is a boolen variable which is true if the token is valid and false otherwise.

Making an API call

Making an API call is analogous to our normal view.

//GetCategoryFuncAPI will return the categories for the user
//depends on the ID that we get, if we get all, then return all 
//categories of the user
func GetCategoryFuncAPI(w http.ResponseWriter, r *http.Request) {
    if r.Method == "GET" {
        var err error
        var message string
        var status types.Status
        //get the custom HTTP header called Token
        token := r.Header["Token"][0]

        w.Header().Set("Content-Type", "application/json; charset=UTF-8")

        IsTokenValid, username := ValidateToken(token)
        //When the token is not valid show the 
        //default error JSON document
        if !IsTokenValid {
            status = types.Status
            {
                StatusCode: http.StatusInternalServerError, 
                Message: message
            }
            w.WriteHeader(http.StatusInternalServerError)
            //the following statement will write the JSON document to
            //the HTTP ResponseWriter object.
            err = json.NewEncoder(w).Encode(status)

            if err != nil {
                panic(err)
            }
            return
        }

        log.Println("token is valid " + username + " is logged in")
        categories := db.GetCategories(username)
        w.Header().Set("Content-Type", "application/json; charset=UTF-8")

        w.WriteHeader(http.StatusOK)

        err = json.NewEncoder(w).Encode(categories)
        if err != nil {
            panic(err)
        }
    }
}

During an API call, we send data in JSON format, for that, we need to set our content-type as application/json, by doing this, even a web browser will detect that it is getting a JSON document. When we need to write a JSON document to the response writer object, we use the json.NewEncoder(w).Encode(categories) method, where categories is our JSON document.

Formatting a JSON document

This, below, is our Tasks struct, which will be populated as a JSON document when we run our server. As you might know, we can't use Capital letter as the first letter in a JSON title, by convention they should all be small letters. Go has a special way of letting us do that. The example is below, when we write json:"id", we are telling Go that the name of this field in a JSON rendering should be id and not Id. There is another special syntax called omitempty, in some JSON documents you might want some field to not be displayed. It so happens that there are fields which you would want to disappear when their values aren't present, they may not be important or it'd be too clunky to have them as NULL in all JSON documents.

type Task struct {
    Id        int       `json:"id"`
    Title     string    `json:"title"`
    Content   string    `json:"content"`
    Created   string    `json:"created"`
    Priority  string    `json:"priority"`
    Category  string    `json:"category"`
    Referer   string    `json:"referer,omitempty"`
    Comments  []Comment `json:"comments,omitempty"`
    IsOverdue bool      `json:"isoverdue, omitempty"`
}

Testing API

We'll use Firefox and RestClient extension to test our API. RestClient allows us to send various requests to our API server, if you are on Chrome, POSTman is the best alternative.

For RestClient to send Form data, set a custom header

Name: Content-Type

Value: application/x-www-form-urlencoded

Otherwise you'll be sending blank POST requests all the time. The server needs to understand the content type of the data it is getting form the client.

To send the actual form data, example: we have three fields, username, password and name. Then we write it in the body section like this:

username=thewhitetulip&password=password

Also set a custom HTTP header by the Name: token Value: the token which you get in /api/get-token/ call

Writing a client

APIs are built so that we can write applications on top of our service. APIs allow us to write clients that access our service apart from the browser.

package main

import "net/http"
import "fmt"
import "io/ioutil"

func main() {
resp, err := http.Get("http://127.0.0.1:8081/")
if err != nil {
    fmt.Println(err)
}

defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
    fmt.Println("Error reading body")
}
fmt.Println(string(body))
}

When we use the service via a browser, the browser is essentially making the GET and POST calls on our behalf. When we have to write an app to access our service, we have to write those methods in the app.

For executing the above code, we need to ensure that our Tasks application is running on port 8081. When we execute, it must print the login html page on the terminal.

This might be a great example to get started with using HTTP methods from Go, but for our application we require to send POST reqests with content type as form and send the form data as request body.

Getting the token

We use the PostForm method which will send a Form via a POST method, effectively saving the trouble of setting the content type field for us. If you run the below code, it will print the authentication token on the terminal.

import "net/http"
import "net/url"
import "fmt"
import "io/ioutil"

func main() {
    usernamePwd := url.Values{}
    usernamePwd.Set("username", "suraj")
    usernamePwd.Set("password", "suraj")

    resp, err := http.PostForm("http://127.0.0.1:8081/api/get-token/", usernamePwd)
    if err != nil {
        fmt.Println(err)
    }

    defer resp.Body.Close()
    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        fmt.Println("Error reading body")
    }
    fmt.Println(string(body))
}

The next steps are to store this token and get the task list.

Error handling

We should throw a user friendly error message if the Tasks application isn't running in the background. And then exit the client.

Example

The below code will get a new token and print all the tasks for that particular user.

package main

import (
    "fmt"
    "io/ioutil"
    "net/http"
    "net/url"
    "os"
)

func main() {
    baseURL := "http://127.0.0.1:8081/"

    usernamePwd := url.Values{}
    usernamePwd.Set("username", "suraj")
    usernamePwd.Set("password", "suraj")

    resp, err := http.PostForm(baseURL+"api/get-token/", usernamePwd)
    if err != nil {
        fmt.Println("Is the server running?")
        os.Exit(1)
    } else {
        fmt.Println("response received")
    }

    defer resp.Body.Close()
    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        fmt.Println("Error reading body")
    } else {
        fmt.Println("Token received")
    }
    token := string(body)

    client := &http.Client{}
    req, err := http.NewRequest("GET", baseURL+"api/get-task/", nil)

    if err != nil {
        fmt.Println("Unable to form a GET /api/get-task/")
    }

    req.Header.Add("Token", token)
    resp, err = client.Do(req)

    if (err != nil) || (resp.StatusCode != 200) {
        fmt.Println("Something went wrong in the getting a response")
    }

    defer resp.Body.Close()
    body, err = ioutil.ReadAll(resp.Body)
    fmt.Println(string(body))

}

Advanced Usage

We can write a complete command line client for tasks in Go as we saw in this chapter, it would mean implementing a lot of command line flags to get what the user wants us to get and we would want to store the key in a text file to cache it.

Homework

• Read the unit testing chapter and write a unit test to test the API client rather than using the firefox addon.

• Build a REST api client to Tasks and add/delete notes via the command line. Note that you'll have to modify how things look & not generate HTML. Render just the markdown text.

Links

-Previous section -Next section