How to Create Lambda Layers for AWS Success

Lambda Layers have gotten complicated with all the packaging requirements and runtime compatibility issues flying around. As someone who’s built and maintained shared Lambda Layers across dozens of functions in production, I learned everything there is to know about keeping your serverless code clean and modular. Today, I will share it all with you.

Before Lambda Layers existed, I was copying the same utility libraries into every single Lambda function’s deployment package. Twenty functions, twenty copies of the same code. When I needed to update a dependency, I had to redeploy all twenty. It was a nightmare. Layers changed everything.

What Lambda Layers Actually Are

A Lambda Layer is a ZIP archive that contains libraries, custom runtimes, or other dependencies that your Lambda functions need. Instead of bundling these dependencies into each function’s deployment package, you put them in a Layer that multiple functions can share. Think of it as a shared library that gets mounted into your function’s execution environment at runtime.

Why You Should Use Layers

• No redundant code: One copy of your shared libraries instead of one per function
• Smaller deployment packages: Your function code stays lean, which means faster deployments
• Centralized dependency management: Update the Layer once, and all functions using it get the new version
• Version control: Each Layer version is immutable, so you can roll back if a new version breaks something

Creating Your First Lambda Layer

Probably should have led with this section, honestly, because the process is simpler than most people think. Let me walk through creating a Python Layer since that’s what I build most often.

Step 1: Set Up Your Directory Structure

Lambda expects a specific directory structure inside the ZIP file. For Python, your libraries need to be in a python/ folder:

my-layer/
  python/
    requests/
    boto3/
    my_shared_utils.py

Step 2: Install Your Dependencies

I create a clean directory and install packages into it. Here’s the command I use:

mkdir -p my-layer/python
pip install requests -t my-layer/python/
pip install pandas -t my-layer/python/

Important: install for the same Python version and architecture your Lambda function uses. If your function runs on Python 3.12 with ARM64 (Graviton), install your packages on a matching environment. I use Docker for this to ensure compatibility:

docker run --rm -v $(pwd):/var/task public.ecr.aws/lambda/python:3.12 pip install requests -t /var/task/my-layer/python/

Step 3: Package It Up

ZIP the contents, making sure the python/ directory is at the root of the archive:

cd my-layer
zip -r ../my-layer.zip python/

Step 4: Publish the Layer

Use the AWS CLI to publish your Layer:

aws lambda publish-layer-version \
  --layer-name my-shared-utils \
  --description "Shared utilities and common dependencies" \
  --zip-file fileb://my-layer.zip \
  --compatible-runtimes python3.11 python3.12 \
  --compatible-architectures arm64 x86_64

That returns a Layer ARN with a version number. You’ll use this ARN when attaching the Layer to your functions.

Step 5: Attach to Your Functions

Add the Layer to any function that needs it:

aws lambda update-function-configuration \
  --function-name my-function \
  --layers arn:aws:lambda:us-east-1:123456789:layer:my-shared-utils:1

Now your function can import requests or import my_shared_utils just as if those packages were in the deployment package. They’re loaded from the Layer at runtime.

Layer Best Practices

After managing Layers across multiple projects, here’s what I’ve learned works best:

• Keep Layers focused: One Layer for data processing libraries, another for utility functions, another for AWS SDK extensions. Don’t dump everything into one mega-Layer.
• Version everything: Each publish creates a new immutable version. Pin your functions to specific versions and only upgrade when you’ve tested the new version.
• Watch the size limit: Lambda has a 250 MB limit for the total unzipped size of your function code plus all Layers. Large libraries like pandas or numpy eat into this quickly.
• Use CloudFormation or SAM for Layer management: Define your Layers in your infrastructure code so they’re versioned and reproducible alongside your functions.
• Test Layers independently: Before publishing a new version, test the Layer’s contents in a dedicated test function. Don’t push untested Layers to production functions.

Real-World Layer Examples

That’s what makes Lambda Layers endearing to us serverless developers — they solve real packaging problems. Here are Layers I’ve built and used in production:

• Common utilities Layer: Logging setup, error handling helpers, configuration loading — things every function in the project needs
• Data processing Layer: pandas, numpy, and custom data transformation functions for ETL workflows
• External API client Layer: Shared client libraries for third-party APIs, complete with retry logic and authentication handling
• Custom runtime Layer: A custom Python runtime with specific patches or extensions

Using AWS-Provided Layers

AWS provides some useful public Layers, including the AWS Parameters and Secrets Lambda Extension. Third-party providers also publish public Layers. Before building your own, check if someone’s already built what you need — it can save you significant maintenance effort.

Debugging Layer Issues

When a Layer isn’t working, it’s almost always one of these problems:

1. Wrong directory structure: The most common mistake. Python libraries must be in python/, Node.js in nodejs/node_modules/.
2. Architecture mismatch: Built on x86_64 but running on ARM64 (or vice versa). Native compiled libraries won’t work cross-architecture.
3. Python version mismatch: Some packages compile C extensions that are version-specific. Build for the exact runtime you’re targeting.
4. Size limit exceeded: Your function code plus all Layers exceeds 250 MB unzipped. Trim dependencies or split into separate functions.

Lambda Layers aren’t complicated once you understand the packaging rules. They’re one of those features that seem trivial but fundamentally improve how you organize and maintain serverless applications. If you’re managing more than a few Lambda functions, take the time to set up shared Layers. Your future self will appreciate the cleaner, more maintainable codebase.