AWS Secrets Manager is a service that helps you manage, retrieve, and rotate database credentials, application credentials, OAuth tokens, API keys, and other secrets throughout their lifecycles. You can use Secrets Manager to replace hard-coded credentials in application source code with a runtime call to the Secrets Manager service to retrieve credentials dynamically when you need them. Storing the credentials in Secrets Manager helps to avoid unintended access by anyone who inspects your application’s source code, configuration, or components.
In this blog post, we introduce a new feature, the Secrets Manager Agent, and walk through how you can use it to retrieve Secretes Manager secrets.
New approach: Secrets Manager Agent
Previously, if you had an application that used Secrets Manager and needed to retrieve secrets, you had to use the AWS SDK or one of our existing caching libraries. Both these options are specific to a certain coding language and allow only limited scope for customization.
The Secrets Manager Agent is a client-side agent that allows you to standardize consumption of secrets from Secrets Manager across your AWS compute environments. (AWS has published the code for the agent as open source code.) Secrets Manager Agent pulls and caches secrets in your compute environment and allows your applications to consume secrets directly from the in-memory cache. The Secrets Manager Agent opens a localhost port inside your application environment. With this port, you fetch the secret value from the local agent instead of making network calls to the service. This allows you to improve the overall availability of your application while reducing your API calls. Because the Secrets Manager Agent is language agnostic, you can install the binary file of the agent on many types of AWS compute environments.
Although you can use this feature to retrieve and cache secrets in your application’s compute environment, the access controls for Secrets Manager secrets remain unchanged. This means that AWS Identity and Access Management (IAM) principals need the same permissions as if they were to retrieve each of the secrets. You will need to provide GetSecretValue and DescribeSecret permissions to the secrets that you want to consume by using the Secrets Manager Agent.
The Secrets Manager Agent offers protection against server-side request forgery (SSRF). When you install the Secrets Manager Agent, the script generates a random SSRF token on startup and stores it in the file /var/run/awssmatoken. The token is readable by the awssmatokenreader group that the install script creates. The Secrets Manager Agent denies requests that don’t have an SSRF token in the header or that have an invalid SSRF token.
Solution overview
The Secrets Manager Agent provides a language-agnostic way to consume secrets in your application code. It supports various AWS compute services, such as Amazon Elastic Compute Cloud (Amazon EC2), Amazon Elastic Container Service (Amazon ECS), Amazon Elastic Kubernetes Service (Amazon EKS), and AWS Lambda functions. In this solution, we share how you can install the Secrets Manager Agent on an EC2 machine and retrieve secrets in your application code by using CURL commands. See the AWS Secrets Manager Agent documentation to learn how you can use this agent with other types of compute services.
Prerequisites
You need to have the following:
An AWS account
The AWS Command Line Interface (AWS CLI) version 2
jq
Follow the steps on the Install or update to the latest version of the AWS CLI page to install the AWS CLI and the Configure the AWS CLI page to configure it.
Create the secret
The first step will be to create a secret in Secrets Manager by using the AWS CLI.
To create a secret
Enter the following command in a terminal to create a secret:
aws secretsmanager create-secret –name MySecret –description “My Secret”
–secret-string “{“user”: “my_user”, “password:”: “my-password”}”
You will see an output like the following:
% aws secretsmanager create-secret —name MySecret —description “My Secret”
—secret-string “{“user”: “my_user”, “password:”: “my-password”}”
{
“ARN”: “arn:aws:secretsmanager:us-east-1:XXXXXXXXXXXX:secret:MySecret-LrBlpm”,
“Name”: “MySecret”,
“VersionId”: “b5e73e9b-6ec5-4144-a176-3648304b2d60”
}
Record the secret ARN as , because you will use it in the next section.
Create the IAM role
The Lambda function, the EC2 instance, and the ECS task definition need an IAM role that grants permission to retrieve the secret you just created.
To create the IAM role
Using an editor, create a file named ec2_iam_policy.json with the following content:
{
“Version”: “2012-10-17”,
“Statement”: [
{
“Effect”: “Allow”,
“Principal”: {
“Service”: “ec2.amazonaws.com”
},
“Action”: “sts:AssumeRole”
}
]
}
Type the following command in a terminal to create the IAM role:
aws iam create-role –role-name ec2-secret-execution-role
–assume-role-policy-document file://ec2_iam_policy.json
Create a file named iam_permission.json with the following content, replacing with the secret ARN you noted earlier:
{
“Version”: “2012-10-17”,
“Statement”: [
{
“Effect”: “Allow”,
“Action”: [
“secretsmanager:GetSecretValue”,
“secretsmanager:DescribeSecret”
],
“Resource”: “”
}
]
}
Type the following command to create a policy:
aws iam create-policy
–policy-name get-secret-policy
–policy-document file://iam_permission.json
Record the Arn as , because you will need that value next.
Type the following command to add this policy to the IAM role, replacing with the value you just noted:
aws iam attach-role-policy
–role-name ec2-secret-execution-role
–policy-arn
Type the following command to add the AWS Systems Manager policy to the role:
aws iam attach-role-policy
–role-name ec2-secret-execution-role
–policy-arn arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore
Launch an EC2 instance
Use the steps in this section to launch an EC2 instance.
To create an instance profile
Type the following command to create an instance profile:
aws iam create-instance-profile –instance-profile-name secret-profile
Type the following command to associate this instance profile with the role you just created:
aws iam add-role-to-instance-profile –instance-profile-name secret-profile
–role-name ec2-secret-execution-role
To create a security group
Type the following command to create a security group:
aws ec2 create-security-group –group-name secret-security-group
–description “Secrets Manager Security Group”
Record the group ID as , because you will need this value in the next step.
To launch an EC2 instance
Run the following command to launch an EC2 instance, replacing with the security group ID:
aws ec2 run-instances
–image-id resolve:ssm:/aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2
–instance-type t3.micro
–security-group-ids
–iam-instance-profile Name=secret-profile
–tag-specifications ‘ResourceType=instance,Tags=[{Key=Name,Value=secret-instance}]’
Record the InstanceId value as .
Check the status of this launch by running the following command:
aws ec2 describe-instances –filters Name=tag:Name,Values=secret-instance |
jq “.Reservations[0].Instances[0].State”
You will see a response like the following, which shows that the instance is running:
% aws ec2 describe-instances —filters Name=tag:Name,Values=secret-instance | jq “.Reservations[0].Instances[0].State”
{
“Code”: 16,
“Name”: “running”
}
After the instance is in running state, type the following command to connect to the EC2 instance, replacing with the value you noted earlier:
aws ssm start-session –target
Leave the session open, because you will use it in the next step.
Install the Secrets Manager Agent to the EC2 instance
Use the steps in this section to install the Secrets Manager Agent in the EC2 instance. You will run these commands in the EC2 instance you created earlier.
To download the Secrets Manager Agent code
Type the following command to install git in the EC2 instance:
sudo yum install -y git
Type the following command to download the Secrets Manager Agent code:
cd ~;git clone https://github.com/awslabs/aws-secretsmanager-agent
To install the Secrets Manager Agent
Type the following command to install the Secrets Manager Agent:
cd aws-secretsmanager-agent/release
sudo ./install
To grant permission to read the token file
Type the following command to copy the token file and grant permission for the current user (ec2-user) to read it:
sudo cp /var/run/awssmatoken /tmp
sudo chown ssm-user /tmp/awssmatoken
Retrieve the secret
Now you can use the local web server to retrieve the agent. Processes running in this EC2 instance can retrieve the secret with a REST API call from the web server.
To retrieve a secret
Retrieving a secret is now possible for the process in this EC2 instance, thanks to the local agent.
Run the following command to retrieve the secret:
curl -H “X-Aws-Parameters-Secrets-Token: $(
