Generate a Terraform Provider from OpenAPI

Brandon Croft,•terraform

Is it possible to generate a Terraform provider plugin from an API specification, in the same way we already generate API client SDKs (opens in a new tab)?

This is part of a series of notes about code generating a Terraform Provider using OpenAPI. The project can be found on GitHub (opens in a new tab)

What is a Terraform Provider?

Terraform is an infrastructure as code (IaC) tool, meaning that it lets you define your software infrastructure declaratively and then it figures out which changes are necessary to automatically provision that infrastructure, usually on a cloud provider, without any manual effort. There are many benefits to this approach, and there are a couple thousand (opens in a new tab) Terraform Providers for all kinds of uses.

provider "aws" {
  profile = "default"
  region  = "us-west-2"
}
 
# An AWS EC2 instance to host our app
resource "aws_instance" "app_server" {
  ami           = "ami-830c94e3"
  instance_type = "t2.micro"
 
  tags = {
    Name = "ExampleAppServerInstance"
  }
}

A Terraform Provider, in this case, "aws", only specifically knows how to interact with AWS. Terraform itself knows all the information about the resources, aws or otherwise, it has created and updated. It generates a graph and a diff with what it already knows about what is has previously done to determine which resources to create, destroy, or update. (The "What" and "When") while the provider takes care of the "How". In this regard, Terraform plus providers can be basically described as a fancy API client with state.

Writing a Terraform Provider from scratch is supported by HashiCorp frameworks to be extremely simple, but it remains tedious. You have to know a lot about how Terraform works. You have to keep track of what Terraform knows about (its state) and what is coming over the wire from your provider's API. What would it take to code generate one if we know everything about the API itself?

Sub-problems

  1. Binding OpenAPI Schema to Terraform resource schema.
  2. Open API doesn't clearly define what a resource is.
  3. Not all of an API spec can or should be exposed as Terraform resources.
  4. There are a lot of entities to map
  5. Terraform data sources can take several forms.

Milestones

  1. Generate resource schemas for Terraform 1.0+
  2. Generate go types for state data
  3. Use config bindings for Computed, Force Replacement, Sensitive, etc Terraform attribute options
  4. Generate HTTP request code for all framework operations
  5. Generate integration tests
  6. Generate common authentication strategies
  7. Generate data sources

Alternative Approaches

There are a couple of published alternatives to static code generation that I want to highlight:

© Brandon Croft.