How to use API Gateway as a proxy to DynamoDB using Terraform?

Amazon API Gateway is a fully managed service which helps developers to create and deploy scalable APIs on AWS. These APIs act as an entry point for the applications to connect and get access to data, perform business logic or access any other AWS service. There are mainly two types: RESTful and Web-Socket APIs. In this article we will use RESTful API.

The second Amazon service we are going to talk about in this article is Amazon DynamoDB. It is a NO-SQL key value, fully managed database provided by AWS. It has features like low-latency reads/writes, multi-master and multi-region.

Recently, I came across a requirement to store something in DynamoDB and retrieve the data using hash key whenever it is required. Now one way is to implement the logic in the code where it is required using aws-sdk. But in my case, this is required by many of the different services having access to the same set of config stored in the DynamoDB. Following are the option I can move towards:

  • Create a shared lib of code which can be referenced and shared across all the services which need access to this data.
  • I can create it as a separate service which can be exposed as an api for everyone to consume.

So me and my teammates, we decided to use the second method but with a twist. Rather than writing a code for the service which we can use it as an API, we decided to use API Gateway and its AWS service integration to directly talk to DynamoDB to return the result. We will be done after some lines of deployment code (yeah, all deployment should be automated) and no actual app code to maintain once we are done. We write all our infrastructure code in terraform and following is the steps through it to how to create one and deploy it.

Assumptions:

Here are some of the assumtions before we head towards the terraform scripts:-

  • You have an undersatnding about terraform and know your way around it.
  • You have an AWS account which you can use to run and deploy the scripts if you wish to and there can be some charges when you try it out. So please be aware of that.

Repository

If you wish to follow along, here is the repository which have an example. Terraform-Example-Script

Step 1 - Create a DynamoDB table with a defined hash and range key

# Create dynamo db table
resource "aws_dynamodb_table" "example" {
  name         = "example"
  hash_key     = "id"
  range_key    = "range"
  billing_mode = "PAY_PER_REQUEST"
  attribute {
    name = "id"
    type = "S"
  }
  attribute {
    name = "range"
    type = "S"
  }
}

Step 2 - Create an IAM policy and role for the execution

In this step we will create an IAM policy and role which can be assumed by any API gateway. The policy attached to the role will give access to the API Gateway to perform operations on the Dynamodb table defined above.

# The policy document to access the role
data "aws_iam_policy_document" "dynamodb_table_policy_example" {
  depends_on = [aws_dynamodb_table.example]
  statement {
    sid = "dynamodbtablepolicy"
    actions = [
      "dynamodb:Query"
    ]
    resources = [
      aws_dynamodb_table.example.arn,
    ]
  }
}
# The IAM Role for the execution
resource "aws_iam_role" "api_gateway_dynamodb_example" {
  name               = "api_gateway_dynamodb_example"
  assume_role_policy = <<-EOF
  {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Action": "sts:AssumeRole",
        "Principal": {
          "Service": "apigateway.amazonaws.com"
        },
        "Effect": "Allow",
        "Sid": "iamroletrustpolicy"
      }
    ]
  }
  EOF
}
resource "aws_iam_role_policy" "example_policy" {
  name = "example_policy"
  role = aws_iam_role.api_gateway_dynamodb_example.id
  policy = data.aws_iam_policy_document.dynamodb_table_policy_example.json
}

Step 3 - Create an API Gateway

resource "aws_api_gateway_rest_api" "exampleApi" {
  name        = "exampleApi"
  description = "Example API"
}

Step 4 - Create a Resource in API Gateway

In order to create routes in API Gateway, we need to create a resource and attach it to API Gateway. In the following code, the path is created as {val}. This allows us to create a parametreised route so we can execute anything like {API Gateway Route}/test. Here value test will be assigned to val keyword and we can later use it.

# Create a resource
resource "aws_api_gateway_resource" "resource-example" {
  rest_api_id = aws_api_gateway_rest_api.exampleApi.id
  parent_id   = aws_api_gateway_rest_api.exampleApi.root_resource_id
  path_part   = "{val}"
}

Step 5 - Create a Method

We will create a GET method under the resource we created above. This will allow us to make GET request on the resource and fire off the integration attached with it.

# Create a Method
resource "aws_api_gateway_method" "get-example-method" {
  rest_api_id   = aws_api_gateway_rest_api.exampleApi.id
  resource_id   = aws_api_gateway_resource.resource-example.id
  http_method   = "GET"
  authorization = "NONE"
}

Step 6 - Creating a Request Intergation.

This is the step which will link our API Gateway with our Dynamodb. Request integration will route any request coming to the defined resource and method to the integration service. The most widely used integration is Lambda but in our case we are going to use an AWS service integration with Dynamodb.

# Create an integration with the dynamo db
resource "aws_api_gateway_integration" "get-example-integration" {
  rest_api_id             = aws_api_gateway_rest_api.exampleApi.id
  resource_id             = aws_api_gateway_resource.resource-example.id
  http_method             = aws_api_gateway_method.get-example-method.http_method
  type                    = "AWS"
  integration_http_method = "POST"
  uri                     = "arn:aws:apigateway:{aws_region}:dynamodb:action/Query"
  credentials             = aws_iam_role.api_gateway_dynamodb_example.arn
  request_templates = {
    "application/json" = <<EOF
      {
        "TableName": "{aws_dynamodb_table.example.name}",
        "KeyConditionExpression": "id = :val",
        "ExpressionAttributeValues": {
          ":val": {
              "S": "$input.params('val')"
          }
        }
      }
    EOF
  }
}
In the request template, we are making a request to the Dynamodb table and our condition is to query where our hash_key i.e. id is equal to the val (which we setup in parameterized route). You have to define the tablename and aws_region to make it work. You can use terraform vars or referencing by adding $ sign in front of it. Please see the example script mentioned above to see the working copy.

Step 7 - Creating a Response Code and template for the request.

We have created a request and now we need to create a response mapping so it can transform and return the response to us.

#Add a response code with the method
resource "aws_api_gateway_method_response" "get-example-response-200" {
  rest_api_id = aws_api_gateway_rest_api.exampleApi.id
  resource_id = aws_api_gateway_resource.resource-example.id
  http_method = aws_api_gateway_method.get-example-method.http_method
  status_code = "200"
  response_parameters = {
    "method.response.header.Access-Control-Allow-Origin" = true
  }
}
# Create a response template for dynamo db structure this will return single item
resource "aws_api_gateway_integration_response" "get-example-response" {
  depends_on  = [aws_api_gateway_integration.get-example-integration]
  rest_api_id = aws_api_gateway_rest_api.exampleApi.id
  resource_id = aws_api_gateway_resource.resource-example.id
  http_method = aws_api_gateway_method.get-example-method.http_method
  status_code = aws_api_gateway_method_response.get-example-response-200.status_code
  response_parameters = {
    "method.response.header.Access-Control-Allow-Origin" = "'*'"
  }
  response_templates = {
    "application/json" = <<EOF
      #set($inputRoot = $input.path('$'))
      {
        #foreach($elem in $inputRoot.Items)
        "id": "id.S",
        #if($foreach.hasNext),#end
        #end
      }
    EOF
  }
}
# Create a response template for dynamo db structure this will return list of item
resource "aws_api_gateway_integration_response" "get-example-response" {
  depends_on  = [aws_api_gateway_integration.get-example-integration]
  rest_api_id = aws_api_gateway_rest_api.exampleApi.id
  resource_id = aws_api_gateway_resource.resource-example.id
  http_method = aws_api_gateway_method.get-example-method.http_method
  status_code = aws_api_gateway_method_response.get-example-response-200.status_code
  response_parameters = {
    "method.response.header.Access-Control-Allow-Origin" = "'*'"
  }
  response_templates = {
    "application/json" = <<EOF
      #set($inputRoot = $input.path('$'))
      {
        "items"=[
          #foreach($elem in $inputRoot.Items)
          {"id": "id.S"},
          #if($foreach.hasNext),#end
          #end
        ]
      }
    EOF
  }
}
You should change the response template as per the response structure and name of the keys you have. Also, the two different response template mentioned above are for example puprposes only. You have to choose one while deployment for returning the response. This is where REST shines by having different route for different resource.

Step 8 - Deployment of the API Gateway

In order to use the API Gateway we have to deploy it along with a stage. Anytime you need to change anything defined above, the API Gateway will be required to deploy again so new changes can come into effect. In order to fulfil this requirement, we will add a stage variable and assign the timestamp to it. So everytime the plan is created, the deployment for API Gateway is forced.

resource "aws_api_gateway_deployment" "exampleApiDeployment" {
  depends_on = [aws_api_gateway_integration.get-example-integration]
  rest_api_id = aws_api_gateway_rest_api.exampleApi.id
  stage_name  = var.stage_name
  variables = {
    "deployedAt" = timestamp()
  }
  lifecycle {
    create_before_destroy = true
  }
}

Step 9 - Test it

Add some records in the Dynamodb, copy the URL from the stage in API Gateway and make a GET request and see the response. Everything should be working if not, get your debugging hat on the head and lets see where we have screwed.

What is Next?

  • In the example, you will find some additional steps to attach a custom domain to the script so that can be used as a reference.
  • If you are more of a console guy, there is an article by AWS which have steps with screenshots using AWS Console. Check it out here ==> Using Amazon API Gateway as a proxy for DynamoDB
I would love to hear what you think about this article and if there anything you would like me to cover in future. Drop me a line on @awsmag.