Terraform DynamoDB Module

This module is no longer being maintained

Terraform module to provision a DynamoDB table with autoscaling.

Autoscaler scales up/down the provisioned OPS for the DynamoDB table based on the load.

Requirements

This module requires Terraform version ~> 0.11 to run.

This module requires the following provider version:

• AWS Provider >= 2.6.0
• null Provider >= 2.1.0

Architectural Decisions Register (ADR's)

All decisions about this service are recorded as ADR's and are stored in the repo. View the TOC

Usage

module "dynamodb_table" {
  source                       = "git::https://github.com/lic-nz/terraform-dynamodb-module.git?ref=master"
  enabled                      = "true"
  dynamodb_enabled             = "true"
  namespace                    = "lic"
  stage                        = "dev"
  name                         = "cluster"
  hash_key                     = "HashKey"
  range_key                    = "RangeKey"
  autoscale_write_target       = 50
  autoscale_read_target        = 50
  autoscale_min_read_capacity  = 5
  autoscale_max_read_capacity  = 20
  autoscale_min_write_capacity = 5
  autoscale_max_write_capacity = 20
  enable_autoscaler            = "true"
}

Advanced Usage

With additional attributes, global secondary indexes and non_key_attributes (see examples/complete).

module "dynamodb_table" {
  source                       = "git::https://github.com/lic-nz/terraform-dynamodb-module.git?ref=master"
  enabled                      = "true"
  dynamodb_enabled             = "true"
  namespace                    = "lic"
  stage                        = "dev"
  name                         = "cluster"
  hash_key                     = "HashKey"
  range_key                    = "RangeKey"
  autoscale_write_target       = 50
  autoscale_read_target        = 50
  autoscale_min_read_capacity  = 5
  autoscale_max_read_capacity  = 20
  autoscale_min_write_capacity = 5
  autoscale_max_write_capacity = 20
  enable_autoscaler            = "true"

  dynamodb_attributes = [
    {
      name = "DailyAverage"
      type = "N"
    },
    {
      name = "HighWater"
      type = "N"
    },
  ]

  local_secondary_index_map = [
  {
    name      = "TimestampSortIndex"
    range_key = "Timestamp"
    projection_type    = "INCLUDE"
    non_key_attributes = ["HashKey", "RangeKey"]
  },
  {
    name      = "HighWaterIndex"
    range_key = "Timestamp"
    projection_type    = "INCLUDE"
    non_key_attributes = ["HashKey", "RangeKey"]
  }
]

  global_secondary_index_map = [
    {
      name               = "DailyAverageIndex"
      hash_key           = "DailyAverage"
      range_key          = "HighWater"
      write_capacity     = 5
      read_capacity      = 5
      projection_type    = "INCLUDE"
      non_key_attributes = ["HashKey", "RangeKey"]
    },
    {
      name            = "HighWaterIndex"
      hash_key        = "HighWater"
      write_capacity  = 5
      read_capacity   = 5
      projection_type = "KEYS_ONLY"
    },
  ]
}

Examples

Some base and complete example implementations of module usage can be found in the examples folder. There are examples for implementation using both Terraform and Terragrunt.

Inputs

IMPORTANT INFORMATION: Please see billing-mode for advice on choosing the appropriate option for billing_mode when using this module.

Name	Description	Type	Default	Required
attributes	Additional attributes (e.g policy or role)	list	<list>	no
autoscale_max_read_capacity	DynamoDB autoscaling max read capacity	string	20	no
autoscale_max_write_capacity	DynamoDB autoscaling max write capacity	string	20	no
autoscale_min_read_capacity	DynamoDB autoscaling min read capacity	string	5	no
autoscale_min_write_capacity	DynamoDB autoscaling min write capacity	string	5	no
autoscale_read_target	The target value (in %) for DynamoDB read autoscaling	string	50	no
autoscale_write_target	The target value (in %) for DynamoDB write autoscaling	string	50	no
aws_region	AWS region where resources will be created	string	ap-southeast-2	no
billing_mode	DynamoDB Billing mode. Can be PROVISIONED or PAY_PER_REQUEST	string	PROVISIONED	no
delimiter	Delimiter to be used between namespace, stage, name and attributes	string	-	no
dynamodb_attributes	Additional DynamoDB attributes in the form of a list of mapped values	list	<list>	no
dynamodb_enabled	Whether or not to enable DynamoDB resources	string	false	no
enable_autoscaler	Whether or not to enable DynamoDB autoscaling	string	true	no
enable_encryption	Enable DynamoDB server-side encryption	string	true	no
enable_point_in_time_recovery	Enable DynamoDB point in time recovery	string	true	no
enable_streams	Enable DynamoDB streams	string	false	no
enabled	Whether or not the module should create any resources	string	true	no
global_secondary_index_map	Additional global secondary indexes in the form of a list of mapped values	list	<list>	no
hash_key	DynamoDB table Hash Key	string	-	yes
hash_key_type	Hash Key type, which must be a scalar type: S, N, or B for (S)tring, (N)umber or (B)inary data	string	S	no
local_secondary_index_map	Additional local secondary indexes in the form of a list of mapped values	list	<list>	no
name	Name (e.g. app or cluster)	string	-	yes
namespace	Namespace (e.g. eg or cp)	string	-	yes
range_key	DynamoDB table Range Key	string	``	no
range_key_type	Range Key type, which must be a scalar type: S, N or B for (S)tring, (N)umber or (B)inary data	string	S	no
stage	Stage (e.g. prod, dev, staging, infra)	string	-	yes
stream_view_type	When an item in a table is modified, what information is written to the stream	string	``	no
tags	Additional tags (e.g map(BusinessUnit,XYX)	map	<map>	no
ttl_attribute	DynamoDB table ttl attribute	string	Expires	no
ttl_enabled	Whether ttl is enabled or disabled	string	true	no

Outputs

Name	Description
dynamodb_enabled	Whether or not DynamoDB is enabled
enabled	Whether or not this module is enabled
global_secondary_index_names	DynamoDB secondary index names
local_secondary_index_names	DynamoDB local index names
table_id	DynamoDB table ID
table_name	DynamoDB table name
table_stream_arn	DynamoDB table stream ARN
table_stream_label	DynamoDB table stream label

Billing Mode

The choice of PROVISIONED or PAY_PER_REQUEST billing, is largely financial, but also may be influenced by limiting latency, development time spent on right sizing, and the risk of paying for unexpectedly high loads.

As at May 2019 PAY_PER_REQUEST costs about 7x as much per request than PROVISIONED, which means that any DynamoDB Table that can achieve more than 14.4% average utilization with provisioned capacity will cost less dollars using PROVISIONED capacity. Note that:

• Low utilization in dev, and accp environments and configuring each environment identically will mean PROVISIONED capacity is less attractive.
• The minimum provisioned capacity is 1 read and 1 write per second which may make PAY_PER_REQUEST more attractive for lightly used tables, even tif they have predicatable load.
• Using auto scaling for PROVISIONED capacity will help to increase the utilization of tables that require more than the minimum capacity.

WIth regards to limitting latency, both choices can be problematic in different situations. Rapid bursts of traffic to either PAY_PER_REQUEST or autoscaled PROVISIONED tables will mean some requests are throttled. However if this is an issue internet commentary suggests that PAY_PER_REQUEST scales smoothly up to 4000 requests/sec so this might be a good starting point for comparison.

PAY_PER_REQUEST has a clear advantage in terms of development time spent on right sizing until very heavy loads are encountered where latency due to autoscaling is an issue.

PAY_PER_REQUEST does have the risk of generating unexpectedly large bills if something happens to cause unexpectedly high demand. This can be managed in several ways, such as configuring alarms when traffic is unexpectedly high.