Mixing gRPC and HTTP traffic with grpc-gateway

In the first of our series on designing the Clarifai API we mentioned our transition from Django, to Goji, to grpc-gateway. Today, we would love to share some of what we have learned by running grpc-gateway in production. 

Grpc-gateway, a server for gRPC and RESTful styles

Grpc-gateway is a server that routes HTTP/1.1 request with JSON bodies to gRPC handlers with protobuf bodies. This means you can define your entire API as gRPC methods with requests and responses defined as protobuf, and then implement those gRPC handlers on your API server. Grpc-gateway can then transform your gRPC service into a RESTful API.

Even better, you can mix the gRPC and HTTP requests into the same hostport of your server. We will walk through a simple API server setup that you can try for yourself (so that you don't have to spend three months banging your head trying to get things right).

First, let’s define a simple gRPC method called TestRPC that we will implement in our API server:

You’ll see some atypical fields in the .proto that are leveraged by grpc-gateway. One of the most important of these fields is the option (google.api.http) where we define what HTTP URL(s) will be used to handle our request. We also specify POST as the HTTP method we will accept. Finally, if you have a request body that you expect (typical for POST requests and others), you must use the body field. If you don’t, then the request won’t be passed along to the handler.

In this example, we show a few of the features of protobufs and grpc-gateway:

1. You can use fields from the request proto (in this example TestRequest) in the URL template so that multiple URLs can be parsed together.
2. You can type the fields easily.
3. You can share objects between multiple requests and responses to make it an object-oriented interface.

In this example we will simply echo back the request fields to the response, so TestResponse is setup with similar fields to the request. There is in-depth documentation for the google.api.http proto here which is very helpful in understanding the various options and how handling happens with grpc-gateway. 

Getting started with protos in golang

To use the protos in golang, you need to compile them into their generated code. The easiest way to get the protoc compiler we’ve found is to use Python’s pip installer: 

pip install grpcio

Installation for the grpc-gateway in golang is done using:

go get github.com/grpc-ecosystem/grpc-gateway

Then you can compile your proto files with a command-line something like:

This will result in two new files: 

example.pb.go

example.pb.gw.go

The first being the typical golang compiled proto and the second being the grpc-gateway specific compile proto which will contain all the translation layers between HTTP and gRPC that grpc-gateway uses.

The main.go file

Next, let’s look at the example main.go file that will serve on a given port and listening at the endpoint we defined above.

At the top of the file, we define the handlers for our gRPC methods. Typically you would do this in a different package, but for simplicity, it was done here. 

Here we listen on a port and then pass that listener into cmux which is a high-performance mux written by Cockroach Labs. This cmux allows us to register more than one type of handler. From here we register both the gRPC handler to accept binary gRPC requests directly and the HTTP handler to allow grpc-gateway to translate HTTP requests to gRPC for us. 

We also show how to gracefully stop the process, so that if a signal is received by your server, you can do some cleanup before exiting. Finally, each of the two listeners are started using goroutines, and then errors are handled from the channel that is created.

Two years and counting

This type of server has been handling our requests for over two years behind an nginx proxy and has been great for many reasons:

1. Our API is fully defined in protocal buffers which are easy to read, extensible and object oriented.

2. We support HTTP RESTful API endpoints as well as direct binary gRPC requests, while implementing only gRPC handles for both.

3. We can automatically generate gRPC based API clients.

4. We have built a variety of tools around gRPC handles such as authorization scopes in our fine-grained API keys.

5. We have option to generate swagger files defining the API from the protobuf API spec.

All these things and more have allowed us to now add over 140 endpoints to our API in the last couple years. Check out these endpoints by signing up at clarifai.com.