Step Functions 2025: Distributed Map and HTTPS Endpoints

AWS Step Functions has evolved from a simple state machine service to a complete serverless orchestration platform. The 2025 updates focus on massive parallelism and easier integrations—processing millions of items and connecting to any API.

Distributed Map: Million-Item Parallelism

Traditional Map states were limited to 40 concurrent iterations. Distributed Map removes this constraint entirely:

Feature	Standard Map	Distributed Map
Max Concurrency	40	10,000
Max Items	In execution input	Millions (S3 source)
Input Source	JSON array	S3 bucket, JSON, CSV
Progress Tracking	Limited	Built-in dashboard

Distributed Map Example

# Process 10 million S3 objects in parallel
{
  "StartAt": "ProcessAllFiles",
  "States": {
    "ProcessAllFiles": {
      "Type": "Map",
      "ItemReader": {
        "Resource": "arn:aws:states:::s3:listObjectsV2",
        "Parameters": {
          "Bucket": "my-data-lake",
          "Prefix": "raw-data/"
        }
      },
      "ItemProcessor": {
        "ProcessorConfig": {
          "Mode": "DISTRIBUTED",
          "ExecutionType": "EXPRESS"
        },
        "StartAt": "TransformFile",
        "States": {
          "TransformFile": {
            "Type": "Task",
            "Resource": "arn:aws:lambda:...",
            "End": true
          }
        }
      },
      "MaxConcurrency": 1000,
      "End": true
    }
  }
}

HTTPS Endpoints: Call Any API

New in 2025: Call external APIs directly from Step Functions without Lambda:

# Call Stripe API directly from Step Functions
{
  "CreatePayment": {
    "Type": "Task",
    "Resource": "arn:aws:states:::http:invoke",
    "Parameters": {
      "ApiEndpoint": "https://api.stripe.com/v1/charges",
      "Method": "POST",
      "Authentication": {
        "ConnectionArn": "arn:aws:events:...:connection/stripe"
      },
      "RequestBody": {
        "amount.$": "$.orderTotal",
        "currency": "usd"
      }
    },
    "Next": "SendConfirmation"
  }
}

Use Cases

When to Use Step Functions

ETL pipelines: Process millions of files with Distributed Map
Order processing: Coordinate payment, inventory, shipping
ML pipelines: Orchestrate SageMaker training and deployment
Human approval: Wait for manual approval with callbacks
Third-party APIs: HTTPS endpoints without Lambda cold starts

Step Functions continues to be the backbone of serverless architectures. The 2025 features make it viable for data engineering at any scale.