How to manage AWS ECS environment variables with Chamber?

Written by

Michał Kleszcz

Published on

March 2, 2023

TL;DR

Chamber is a tool for managing secrets in SSM Parameter Store, an AWS service for storing secrets. And today, we're going to uncover the secret of using the opportunities Chamber provides in a proper way. Follow the guide for step-by-step instructions, examples, and code snippets that you can copy.

Author

Michał Kleszcz

CTO

My LinkedIn

Download 2024 SaaS Report

Thank you! Your submission has been received

Oops! Something went wrong while submitting the form.

Storing and injecting AWS ECS environment variables into Docker containers is not trivial if it should be elastic and easy to maintain. Modern-day applications will require you to safely store variables such as API Tokens and services' secrets while allowing you to deploy multiple instances – Dev, QA & Prod, for example. There can be a lot of work to maintain these variables and safely split them between the environments.

In this article, I would like to show how to achieve an elastic way of setting environment variables using the Chamber and AWS SSM Parameter Store.

Assumptions:

you know how Docker works
you know and use AWS ECS

What’s the goal?

To create a single storage place for all environment variables used by each service or container for each application environment. The available variables should not be hardcoded anywhere, so adding or changing new ones shouldn’t require rebuilding the entire application.

The following diagram presents the solution:

‍

‍

What tools will we use for managing AWS ECS environment variables?

Chamber

Chamber is a library that will live inside a docker container. It will connect to AWS SSM Parameter Store, load environment variables for the container, and inject them into the main process.

AWS SSM Parameter store

AWS SSM Parameter store is where we will put all the environment variables that we would like to store securely.

Solution

First, let's prepare a simple backend service. For the sake of this example, I will use a simple NodeJS application:

const http = require('http');

const GREETING = process.env.GREETING;
const port = 3000;

const server = http.createServer((req, res) => {
 res.statusCode = 200;
 res.setHeader(	'Content-Type', 	'text/plain');
 res.end(`Hello ${GREETING}`);
});

server.listen(port, () => {
 console.log(`Server running at http://${hostname}:${port}/`);
});

This application will display a value from the `GREETING` environment variable and live inside a docker container.

Dockerfile:

FROM node:16

WORKDIR /usr/src/app
COPY index.js ./
EXPOSE 3000
CMD [ "node", "server.js" ]

Adding secrets to AWS SSM Parameter Store

There are two ways of adding variables to the AWS SSM Parameter Store. You can do it manually in the AWS console or use a simple script to upload them using CLI.

In this example, I want to store variables so that Chamber can select them for each service and environment. So I'll prefix each variable name with env-{environment}-{service}. Then the parameter name for the GREETING variable for the stage environment will be `env-stage-backend/GREETING`.

Adding manually via AWS console:

Go to Systems Manager > Parameter Store on your AWS account, then press the `Create Parameter` button.

In the form, fill the variable’s name as per the instructions above (with the prefix) and value.

You can select String as a type (for the secrets, we strongly recommend using SecureString).

‍

Adding via CLI:

Let’s use a simple script that will allow us to create and edit multiple environment variables from your command line.

I call this tool `ssm-editor`. It will contain two files:

scripts/run.sh

#!/bin/bash

set -e

FULL_SERVICE_NAME="env-${CHAMBER_ENV}-${CHAMBER_SERVICE}"
CHAMBER_KMS_KEY_ALIAS="${CHAMBER_ENV}-env-kms-key"

CHAMBER_KMS_KEY_ALIAS="${CHAMBER_KMS_KEY_ALIAS}" 
/bin/chamber export "${FULL_SERVICE_NAME}" \
 | jq '.' \
 | vipe \
 | CHAMBER_KMS_KEY_ALIAS="${CHAMBER_KMS_KEY_ALIAS}" 
 /bin/chamber import "${FULL_SERVICE_NAME}" -

Dockerfile

FROM segment/chamber:2 AS chamber
FROM alpine

RUN apk --no-cache add bash ca-certificates jq moreutils

COPY --from=chamber /chamber /bin/chamber

RUN mkdir -p /scripts
COPY scripts /scripts
RUN chmod a+x /scripts/*

ENTRYPOINT ["/bin/bash", "/scripts/run.sh"]

‍

Assuming that you built the above image with the `ssmeditor` name and you have your AWS credentials configured correctly, you can run the tool using simple docker command:

docker run --env CHAMBER_ENV=stage \
--env CHAMBER_SERVICE=backend \
--env AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID} \
--env AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY} \
--env AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION} \
--env AWS_SESSION_TOKEN=${AWS_SESSION_TOKEN} \
--env AWS_SECURITY_TOKEN=${AWS_SECURITY_TOKEN} \
--env AWS_SESSION_EXPIRATION=${AWS_SESSION_EXPIRATION} --rm ssmeditor

‍

Use variables in the Docker container

Now, if you have variables in the correct place, you can pass them into a container that will run inside the ECS task.

To achieve this, you will need to modify the Docker file of the service slightly:

FROM segment/chamber:2 AS chamber

FROM node:16 AS backend
COPY --from=chamber /chamber /bin/chamber
WORKDIR /usr/src/app
COPY . .
EXPOSE 3000
CMD [ "sh", "-c", "/bin/chamber exec env-stage-backend -- node server.js" ]

‍

The above code copies the Chamber tool to the final image, making it available to call. Also, the CMD is changed, so Chamber will fetch the correct list of variables and inject them into the nodejs process. Suppose you would like to use this solution with a more production-ready environment. In that case, I suggest replacing the hardcoded variable name prefix with other environment variables that will be defined in the task configuration.

There is only one piece left: IAM permissions. The ECS task role should have access to the AWS Parameter Store and KMS key if you use secure variables. If you are using AWS CDK, the code should look like this:

taskRole.addToPolicy(
 new PolicyStatement({
   actions: ["kms:Get*", "kms:Describe*", "kms:List*", "kms:Decrypt"],
   resources: [
     "... your KMS key ARN"
   ],
 })
);

taskRole.addToPolicy(
 new PolicyStatement({
   actions: ["ssm:DescribeParameters"],
   resources: ["*"],
 })
);

taskRole.addToPolicy(
 new PolicyStatement({
   actions: ["ssm:GetParameters*"],
   resources: [
     `arn:aws:ssm:{region}:{account}:parameter/env-stage-backend/*`,
   ],
 })
);

AWS ECS environment variables with Chamber – the sum up

Above I presented how to inject environment variables into the ECS task from AWS SSM Parameter Store. If you want to try this solution, I prepared a minimal GitHub repository with ready-to-use code with stack and infrastructure written in CDK.

If this article becomes helpful, check the SaaS boilerplate made by Apptension, as this and many more exciting things are included.

Check AWS ECS environment variables with Chamber on GitHub