Simplifying application orchestration with AWS Step Functions and AWS SAM

Amazon Web Services Feed
Simplifying application orchestration with AWS Step Functions and AWS SAM

Modern software applications consist of multiple components distributed across many services. AWS Step Functions lets you define serverless workflows to orchestrate these services so you can build and update your apps quickly. Step Functions manages its own state and retries when there are errors, enabling you to focus on your business logic. Now, with support for Step Functions in the AWS Serverless Application Model (AWS SAM), you can easily create, deploy, and maintain your serverless applications.

The most recent AWS SAM update introduces the AWS::Serverless::StateMachine component that simplifies the definition of workflows in your application. Because the StateMachine is an AWS SAM component, you can apply AWS SAM policy templates to scope the permissions of your workflows. AWS SAM also provides configuration options for invoking your workflows based on events or a schedule that you specify.

Defining a simple state machine

The simplest way to begin orchestrating your applications with Step Functions and AWS SAM is to install the latest version of the AWS SAM CLI.

Creating a state machine with AWS SAM CLI

To create a state machine with the AWS SAM CLI, perform the following steps:

1. From a command line prompt, enter sam init
2. Choose AWS Quick Start Templates
3. Select nodejs12.x as the runtime
4. Provide a project name
5. Choose the Hello World Example quick start application template

The AWS SAM CLI downloads the quick start application template and creates a new directory with sample code. Change into the sam-app directory and replace the contents of template.yaml with the following code:

# Copyright 2020 Amazon.com, Inc. or its affiliates. All Rights Reserved.
# SPDX-License-Identifier: Apache-2.0
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31 Resources: SimpleStateMachine: Type: AWS::Serverless::StateMachine Properties: Definition: StartAt: Single State States: Single State: Type: Pass End: true Policies: - CloudWatchPutMetricPolicy: {}

This is a simple yet complete template that defines a Step Functions Standard Workflow with a single Pass state. The Transform: AWS::Serverless-2016-10-31 line indicates that this is an AWS SAM template and not a basic AWS CloudFormation template. This enables the AWS::Serverless components and policy templates such as CloudWatchPutMetricPolicy on the last line, which allows you to publish metrics to Amazon CloudWatch.

Deploying a state machine with AWS SAM CLI

To deploy your state machine with the AWS SAM CLI:

1. Save your template.yaml file
2. Delete any function code in the directory, such as hello-world
3. Enter sam deploy --guided into the terminal and follow the prompts
4. Enter simple-state-machine as the stack name
5. Select the defaults for the remaining prompts

For additional information on visualizing, executing, and monitoring your workflow, see the tutorial Create a Step Functions State Machine Using AWS SAM.

Refining your workflow

The StateMachine component not only simplifies creation of your workflows, but also provides powerful control over how your workflow executes. You can compose complex workflows from all available Amazon States Language (ASL) states. Definition substitution allows you to reference resources. Finally, you can manage access permissions using AWS Identity and Access Management (IAM) policies and roles.

Service integrations

Step Functions service integrations allow you to call other AWS services directly from Task states. The following example shows you how to use a service integration to store information about a workflow execution directly in an Amazon DynamoDB table. Replace the Resources section of your template.yaml file with the following code:

Resources: SAMTable: Type: AWS::Serverless::SimpleTable SimpleStateMachine: Type: AWS::Serverless::StateMachine Properties: Definition: StartAt: FirstState States: FirstState: Type: Pass Next: Write to DynamoDB Write to DynamoDB: Type: Task Resource: arn:aws:states:::dynamodb:putItem Parameters: TableName: !Ref SAMTable Item: id: S.$: $$.Execution.Id ResultPath: $.DynamoDB End: true Policies: - DynamoDBWritePolicy: TableName: !Ref SAMTable

The AWS::Serverless::SimpleTable is an AWS SAM component that creates a DynamoDB table with on-demand capacity and reasonable defaults. To learn more, see the SimpleTable component documentation.

The Write to DynamoDB state is a Task with a service integration to the DynamoDB PutItem API call. The above code stores a single item with a field id containing the execution ID, taken from the context object of the current workflow execution.

Notice that DynamoDBWritePolicy replaces the CloudWatchPutMetricPolicy policy from the previous workflow. This is another AWS SAM policy template that provides write access only to a named DynamoDB table.

Definition substitutions

AWS SAM supports definition substitutions when defining a StateMachine resource. Definition substitutions work like template string substitution. First, you specify a Definition or DefinitionUri property of the StateMachine that contains variables specified in ${dollar_sign_brace} notation. Then you provide values for those variables as a map via the DefinitionSubstitution property.

The AWS SAM CLI provides a quick start template that demonstrates definition substitutions. To create a workflow using this template, perform the following steps:

1. From a command line prompt in an empty directory, enter sam init
2. Choose AWS Quick Start Templates
3. Select your preferred runtime
4. Provide a project name
5. Choose the Step Functions Sample App (Stock Trader) quick start application template

Change into the newly created directory and open the template.yaml file with your preferred text editor. Note that the Definition property is a path to a file, not a string as in your previous template. The DefinitionSubstitutions property is a map of key-value pairs. These pairs should match variables in the statemachine/stockTrader.asl.json file referenced under DefinitionUri.

 DefinitionUri: statemachine/stockTrader.asl.json DefinitionSubstitutions: StockCheckerFunctionArn: !GetAtt StockCheckerFunction.Arn StockSellerFunctionArn: !GetAtt StockSellerFunction.Arn StockBuyerFunctionArn: !GetAtt StockBuyerFunction.Arn DDBPutItem: !Sub arn:${AWS::Partition}:states:::dynamodb:putItem DDBTable: !Ref TransactionTable

Open the statemachine/stockTrader.asl.json file and look for the first state, Check Stock Value. The Resource property for this state is not a Lambda function ARN, but a replacement expression, “${StockCheckerFunctionArn}”. You see from DefinitionSubstitutions that this maps to the ARN of the StockCheckerFunction resource, an AWS::Serverless::Function also defined in template.yaml. AWS SAM CLI transforms these components into a complete, standard CloudFormation template at deploy time.

Separating the state machine definition into its own file allows you to benefit from integration with the AWS Toolkit for Visual Studio Code. With your state machine in a separate file, you can make changes and visualize your workflow within the IDE while still referencing it from your AWS SAM template.

Managing permissions and access

AWS SAM support allows you to apply policy templates to your state machines. AWS SAM policy templates provide pre-defined IAM policies for common scenarios. These templates appropriately limit the scope of permissions for your state machine while simultaneously simplifying your AWS SAM templates. You can also apply AWS managed policies to your state machines.

If AWS SAM policy templates and AWS managed policies do not suit your needs, you can also create inline policies or attach an IAM role. This allows you to tailor the permissions of your state machine to your exact use case.

Additional configuration

AWS SAM provides additional simplification for configuring event sources and logging.

Event sources

Event sources determine what events can start execution of your workflow. These sources can include HTTP requests to Amazon API Gateway REST APIs and Amazon EventBridge rules. For example, the below Events block creates an API Gateway REST API. Whenever that API receives an HTTP POST request to the path /request, it starts an execution of the state machine:

 Events: HttpRequest: Type: Api Properties: Method: POST Path: /request

Event sources can also start executions of your workflow on a schedule that you specify. The quick start template you created above provides the following example. When this event source is enabled, the workflow executes once every hour:

 Events: HourlyTradingSchedule: Type: Schedule Properties: Enabled: False Schedule: "rate(1 hour)"

To learn more about schedules as event sources, see the AWS SAM documentation on GitHub.

Logging

Both Standard Workflows and Express Workflows support logging execution history to CloudWatch Logs. To enable logging for your workflow, you must define an AWS::Logs::LogGroup and add a Logging property to your StateMachine definition. You also must attach an IAM policy or role that provides sufficient permissions to create and publish logs. The following code shows how to add logging to an existing workflow:

Resources: SAMLogs: Type: AWS::Logs::LogGroup SimpleStateMachine: Type: AWS::Serverless::StateMachine Properties: Definition: {…} Logging: Destinations: - CloudWatchLogsLogGroup: LogGroupArn: !GetAtt SAMLogs.Arn IncludeExecutionData: true Level: ALL Policies: - CloudWatchLogsFullAccess Type: EXPRESS

Conclusion

Step Functions workflows simplify orchestration of distributed services and accelerate application development. AWS SAM support for Step Functions compounds those benefits by helping you build, deploy, and monitor your workflows more quickly and more precisely. In this post, you learned how to use AWS SAM to define simple workflows and more complex workflows with service integrations. You also learned how to manage security permissions, event sources, and logging for your Step Functions workflows.

To learn more about building with Step Functions, see the AWS Step Functions playlist on the AWS Serverless YouTube channel. To learn more about orchestrating modern, event-driven applications with Step Functions, see the App 2025 playlist.

Now go build!