AWS Secrets Manager and Kubernetes: A Comprehensive Guide

Managing application secrets, such as API keys and database credentials, is a critical task for securing environments. AWS Secrets Manager offers a robust solution for secret management. Combined with Kubernetes, it offers a way to secure secrets for applications deployed in a Kubernetes cluster efficiently.

Introduction to AWS Secrets Manager

AWS Secrets Manager is a service for managing sensitive information. It allows easy rotation, management, and retrieval of database credentials, API keys, and other secrets throughout their life cycle. By centralizing secret management, it reduces the risk of exposing credentials in codebases.

Secrets Manager integrates with AWS Identity and Access Management (IAM) to ensure that only authorized users and applications can access these secrets. It also supports automatic secret rotation, which can be customized with AWS Lambda to meet specific needs.

Kubernetes Secrets

Kubernetes provides native support for managing sensitive information through its Secret resource. These secrets can store various types of data, including passwords, tokens, and keys, within the cluster. Managed securely, Kubernetes secrets are base64 encoded and stored in etcd, the key-value store for Kubernetes.

However, by default, Kubernetes does not encrypt these secrets at rest. This creates a vulnerability. By integrating with AWS Secrets Manager, you can enhance the security of your Kubernetes secrets without significant overhead.

Integrating AWS Secrets Manager With Kubernetes

There are several ways to retrieve secrets from AWS Secrets Manager and use them within a Kubernetes cluster. Below, we explore a few common approaches.

Using the AWS IAM Role for Service Accounts (IRSA)

With IRSA, Kubernetes workloads can use IAM roles to access AWS resources securely. This allows a Kubernetes pod to retrieve secrets from AWS Secrets Manager without embedding AWS credentials in the pods.

1. Create an IAM role with permissions to access the necessary secrets in AWS Secrets Manager.
2. Annotate the Kubernetes service account with the ARN of the IAM role.
3. Configure the Kubernetes pod to use the annotated service account.

The pod will automatically assume the IAM role and can use AWS SDKs to retrieve secrets from Secrets Manager.

External Secrets Operator

The External Secrets Operator provides an automated way to synchronize secrets from external secret stores, like AWS Secrets Manager, into Kubernetes secrets. It reads secret values and injects them into Kubernetes resources.

1. Install the External Secrets Operator in your Kubernetes cluster.
2. Define an ExternalSecret resource that specifies the secret store provider and the secrets to be synchronized.
3. The operator fetches the secrets and creates or updates the corresponding Kubernetes secrets.

This approach simplifies managing secrets by allowing Kubernetes resources to reference secrets directly from the external store.

Step-by-Step: Integrating AWS Secrets Manager Using IRSA

Step 1: Create an IAM Role

Create an IAM role with the necessary permissions to access AWS Secrets Manager:

{    Version: 2012-10-17,    Statement: [        {            Effect: Allow,            Action: [                secretsmanager:GetSecretValue            ],            Resource: [                arn:aws:secretsmanager:region:account-id:secret:secret-name            ]        }    ]}

Ensure the trust relationship allows the Kubernetes service account to assume this role:

{    Version: 2012-10-17,    Statement: [        {            Effect: Allow,            Principal: {                Federated: arn:aws:iam::account-id:oidc-provider/oidc.eks.region.amazonaws.com/id/cluster-id            },            Action: sts:AssumeRoleWithWebIdentity,            Condition: {                StringEquals: {                    oidc.eks.region.amazonaws.com/id/cluster-id:sub: system:serviceaccount:namespace:service-account-name                }            }        }    ]}

Step 2: Annotate the Kubernetes Service Account

apiVersion: v1kind: ServiceAccountmetadata:  name: my-service-account  namespace: my-namespace  annotations:    eks.amazonaws.com/role-arn: arn:aws:iam::account-id:role/role-name

This annotation links the service account to the IAM role created earlier.

Step 3: Configure Your Pod

Make sure your pod is using the annotated service account:

apiVersion: v1kind: Podmetadata:  name: my-pod  namespace: my-namespacespec:  serviceAccountName: my-service-account  containers:  - name: my-container    image: my-image    ...

Within the container, use the AWS SDK to retrieve secrets from AWS Secrets Manager:

import boto3def get_secret(secret_name):    client = boto3.client('secretsmanager')    response = client.get_secret_value(SecretId=secret_name)    secret = response['SecretString']    return secret

Secure Access to Secrets in AWS Secrets Manager

In practice, it is essential to limit access only to specific secrets required for an application. Use IAM policies to enforce the principle of least privilege. Update the IAM role policy to allow only the necessary actions and resources.

Automated Secret Rotation

One of the standout features of AWS Secrets Manager is its ability to automate secret rotation. By integrating secret rotation with Lambda functions, applications can handle secrets seamlessly without manual intervention. Configure rotation rules based on security requirements for enhanced security and compliance.

Final Thoughts

Integrating AWS Secrets Manager with Kubernetes can significantly improve secret management and security. By using IRSA and tools like the External Secrets Operator, you can streamline secret retrieval and ensure that your applications adhere to best practices. Always remember to audit and update permissions regularly to maintain a secure environment.