You may want to provide a RESTful API in addition to your gRPC service without the need to duplicate your code.
gRPC Gateway is a popular tool for generating RESTful APIs from gRPC service definitions.
In this tutorial, we'll take a detailed look at how to use gRPC Gateway to generate an OpenAPI schema based on a Protocol Buffers (protobuf) gRPC service definition. Then we'll use Speakeasy to read our generated OpenAPI schema and create a production-ready SDK.
If you want to follow along, you can use the gRPC Speakeasy Bar example repository.
An Overview of gRPC Gateway
gRPC Gateway is a protoc plugin that reads gRPC service definitions and generates a reverse proxy server that translates a RESTful JSON API into gRPC.
This way, you can expose an HTTP endpoint that can be called by clients that don't support gRPC. The generated server code will forward incoming JSON requests to your gRPC server and translate the responses to JSON.
gRPC Gateway also generates an OpenAPI schema that describes your API. You can use this schema to create SDKs for your API.
OpenAPI Versions
gRPC Gateway outputs OpenAPI 2.0, and Speakeasy supports OpenAPI 3.0 and 3.1. To generate an OpenAPI 3.0 or 3.1 schema, you'll need to convert the OpenAPI 2.0 schema to at least OpenAPI 3.0.
The Protobuf to REST SDK Pipeline
To generate a REST API with a developer-friendly SDK, we'll follow these three core steps:
gRPC to OpenAPI: First, we will use gRPC Gateway to produce an OpenAPI schema based on our protobuf service definition. This generated schema is in OpenAPI 2.0 format.
OpenAPI 2.0 to OpenAPI 3.x: Next, as gRPC Gateway's output schema is in OpenAPI 2.0 and we need at least OpenAPI 3.0 for our SDK, we will convert the generated schema from OpenAPI 2.0 to OpenAPI 3.0.
OpenAPI 3.x to SDK: Finally, once we have the OpenAPI 3.0 schema, we will leverage Speakeasy to create our SDK based on the OpenAPI 3.0 schema derived from the previous steps.
By following these steps, we can ensure we have a robust, production-ready SDK that adheres to our API's specifications.
Step-by-Step Tutorial: From Protobuf to OpenAPI to an SDK
Now let's walk through generating an OpenAPI schema and SDK for our Speakeasy Bar gRPC service.
Check Out the Example Repository
If you would like to follow along, start by cloning the example repository:
git clone git@github.com:speakeasy-api/speakeasy-grpc-gateway-example.git
cd speakeasy-grpc-gateway-example
Install Go
To generate an OpenAPI schema from a protobuf file, we'll need to install Go and protoc.
This tutorial was written using Go 1.21.4.
On macOS, install Go by running:
brew install go
Alternatively, follow the Go installation instructions for your platform.
Install Buf
We'll use the Buf CLI as an alternative to protoc so that we can save our generation configuration as YAML. Buf is compatible with protoc plugins.
On macOS, install Buf by running:
brew install bufbuild/buf/buf
Alternatively, follow the Buf CLI installation instructions for your platform.
Install Buf Modules
We'll use Buf modules to manage our dependencies.
cd proto
buf mod update
cd ..
Install protoc-gen-go
Buf requires the protoc-gen-go plugin to generate Go code from protobuf files.
Install protoc-gen-go by running:
go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
Make sure that the protoc-gen-go binary is in your $PATH. On macOS, you can achieve that by running the following command if the go/bin directory is not already in your path.
export PATH=${PATH}:`go env GOPATH`/bin
Install Go Requirements
go mod tidy
go install \
github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway \
github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-openapiv2 \
google.golang.org/protobuf/cmd/protoc-gen-go \
google.golang.org/grpc/cmd/protoc-gen-go-grpc
Generate the Go Code
We'll use Buf to generate the Go code from the protobuf file.
Run the following in the terminal:
buf generate
Buf reads the configuration in buf.gen.yaml, then generates the Go code in the proto directory.
This will generate the proto/speakeasy/v1/speakeasy.pb.go, proto/speakeasy/v1/speakeasy_grpc.pb.go, and proto/speakeasy/v1/speakeasy.pb.gw.go files.
Generate the OpenAPI Schema
Because we have the openapiv2 protoc plugin configured in our buf.gen.yaml file, Buf will generate an OpenAPI schema and save it as openapi/speakeasy/v1/speakeasy.swagger.json.
This is the OpenAPI 2.0 schema that gRPC Gateway generates by default.
Convert the OpenAPI Schema to OpenAPI 3.0
We'll use the excellent kin-openapi Go library to convert the OpenAPI 2.0 schema to OpenAPI 3.0.
In convert/convert.go, we use kin-openapi to unmarshal openapi/speakeasy/v1/speakeasy.swagger.json, then convert it to OpenAPI 3.0, then marshal it back to JSON, and finally write it to openapi/speakeasy/v1/speakeasy.openapi.json.
To do the conversion, run the following in the terminal:
go run convert/convert.go
Create an SDK With Speakeasy
Now that we have an OpenAPI 3.0 schema, we can create an SDK with Speakeasy.
We'll use the speakeasy generate command to create an SDK for the Speakeasy Bar gRPC service.
Run the following in the terminal:
speakeasy generate sdk \
--schema openapi/speakeasy/v1/speakeasy.openapi.json \
--lang typescript \
--out ./sdk
This will create a TypeScript SDK in the sdk directory.
How To Customize the SDK
By modifying the protobuf service definition, we can customize the SDK via the generated OpenAPI schema.
We'll start with a basic example and add options to enhance the SDK.
<CH.Scrollycoding className="ch-scrollycoding-full-height">
Meet the Speakeasy Bar Protobuf Service
We'll start by taking a look at the Speakeasy Bar protobuf service definition in proto/speakeasy/v1/speakeasy.proto.
// from ./grpc-assets/speakeasy-raw.proto
The service defines one object type, called Drink.
// from ./grpc-assets/speakeasy-raw.proto
A service called SpeakeasyService has two methods, GetDrink and ListDrinks.
// from ./grpc-assets/speakeasy-raw.proto
Add API Information to the Service
We'll add information about the API to the service definition using options.openapiv2_swagger from grpc.gateway.protoc_gen_openapiv2.
// from ./grpc-assets/speakeasy.proto
We'll add a title, description, and version to the API.
This appears in the info object in the generated OpenAPI schema.
// from ./grpc-assets/speakeasy.proto
// from ./grpc-assets/speakeasy.openapi.json
Speakeasy requires a servers object in the OpenAPI schema. We'll add a server to the API using the host key.
Our conversion script will add the servers object to the generated OpenAPI schema.
// from ./grpc-assets/speakeasy.proto
// from ./grpc-assets/speakeasy.openapi.json
Add Descriptions and Examples to Components
To create an SDK that offers great developer experience, we recommend adding descriptions and examples to all fields in OpenAPI components.
Speakeasy will create documentation and usage examples based on the descriptions and examples we added.
We'll start with the Drink object type.
// from ./grpc-assets/speakeasy.proto
We added a title, description, and example to the Drink object type.
Note that the example is a stringified JSON object.
// from ./grpc-assets/speakeasy.proto
We use openapiv2_field to add options to the fields in the Drink object type.
For example, we added a description, pattern, format, and example to the productCode field.
This description and example will appear in the generated OpenAPI document.
// from ./grpc-assets/speakeasy.proto
// from ./grpc-assets/speakeasy.openapi.json
When Speakeasy creates an SDK, this description and example will appear in the generated documentation and usage examples.
This usage example is from the TypeScript SDK's documentation.
Note how the productCode field is represented by our UUID example instead of a random string.
import { SDK } from "openapi";
(async() => {
const sdk = new SDK();
const res = await sdk.drinks.getDrink({
productCode: "602a7da9-b8bb-46e6-b288-457b561029b8",
});
if (res.statusCode == 200) {
// handle response
}
})();
Customize the OperationId
Speakeasy uses the operationId for each operation in the OpenAPI document to create method names in the SDK. By default, the operationId is the method name in the protobuf service definition.
We can customize the operationId for each method using options.openapiv2_operation.
// from ./grpc-assets/speakeasy.proto
// from ./grpc-assets/speakeasy.openapi.json
Add Descriptions and Tags to Methods
Speakeasy uses the description and tags in the OpenAPI document to create documentation for each method in the SDK and group methods in the SDK.
We can add descriptions and tags to methods using options.openapiv2_operation.
// from ./grpc-assets/speakeasy.proto
// from ./grpc-assets/speakeasy.openapi.json
Add OpenAPI Extensions
gRPC Gateway allows us to add OpenAPI extensions to the OpenAPI schema using the extensions key in our protobuf service definition.
For example, we can add the Speakeasy retries extension x-speakeasy-retries, which will cause the SDK to retry failed requests.
In the code example, we added the x-speakeasy-retries extension to the GetDrink method.
// from ./grpc-assets/speakeasy.proto
// from ./grpc-assets/speakeasy.openapi.json
Add Tag Descriptions
Speakeasy uses each tag's description in the OpenAPI document to create documentation in the SDK.
We can add descriptions to tags in the protobuf definition by using options.openapiv2_swagger.
In the code example, we added a description to the drinks tag.
// from ./grpc-assets/speakeasy.proto
// from ./grpc-assets/speakeasy.openapi.json
Example Protobuf Definition and SDK Generator
The source code for our complete example is available in the gRPC Speakeasy Bar example repository.
The repository contains a TypeScript SDK and instructions on how to create more SDKs.
You can clone this repository to test how changes to the protobuf definition result in changes to the SDK.
After modifying your protobuf definition, you can run the following in the terminal to create a new SDK:
buf generate && go run convert/convert.go && speakeasy generate sdk \
--schema openapi/speakeasy/v1/speakeasy.openapi.json \
--lang typescript \
--out ./sdk
Happy generating!