We’re thrilled to announce the general availability of Azure Storage Actions, our fully managed platform that transforms how organizations automate data management tasks for Azure Blob and Data Lake Storage.

Businesses continuously generate, analyze, and store data, accumulating vast datasets with billions of objects that span multiple regions. Relentless data sprawl makes it difficult for data owners to manage their global data assets consistently, increasing risks for data security, protection, and governance, and spiraling costs. Ensuring uniform standards of data management becomes difficult, and diverts attention and resources away from generating business value towards routine administrative tasks. 

Today, customers use disparate tools to manage their data estates. Depending on dataset size and use case, they may use analytic queries with inventory reports, write programs or scripts to list all objects and metadata, or subscribe to storage events or change feed for filtering. To operate on the filtered objects, they write custom scripts for smaller datasets, use automation runbooks, or develop applications using SDKs and frameworks depending on frequency and scale. Errors can cause data loss, so they validate against test datasets that mimic production. Deploying resources in each region to access production environments securely and match the scale adds to the overhead. This must be repeated if future datasets have the same requirements. We designed Azure Storage Actions to help you streamline data management across your organization and put you in control. 

Transform your data management with Azure Storage Actions

With Azure Storage Actions, you can quickly compose a task with complex conditions that targets objects based on their properties, select the operations you want to perform, and safely validate it against production data for peace of mind, without utilizing any programming skills. You can consistently and securely deploy the task you create for recurring or one-off execution on thousands of storage accounts without provisioning or managing any resources. You can monitor execution using charts that summarize the metrics and drill down into detailed reports efficiently. Task definitions are reusable and, once validated, can be applied to any dataset with similar requirements. Key advantages it offers are: 

Eliminate complexity
Scans through your data estate to find objects meeting your defined criteria and perform operations with built-in resiliency.

Boost efficiency
Saves days of coding and validation efforts and provides a zero-touch serverless infrastructure that automatically scales with your data estate and prioritizes your critical workloads.

Drive consistency
Task definitions can be reused—once validated, they can be assigned with confidence to other datasets for similar use cases in any region for consistent data management outcomes.

Hands-free operation
Seamlessly focus on execution summary through dashboards or drilldown to reports that capture every operation.

Supported use cases include managing retention and expiry, reducing costs, data protection, consistent tag management, restore from archive, with more to come. 

Customer stories 

Since we announced the preview, the positive response from Azure Storage customers has reinforced the necessity of simplified, scalable, and reproducible data management processes. Hundreds of organizations, from start-ups to the largest enterprises across industries, turned to Azure Storage Actions for regular and one-off maintenance of datasets. Here are a few examples from the preview. 

Efficient management of backup data

When PointsBet implemented their data protection solution, they started accumulating petabytes of full and incremental backups and transaction logs captured at hourly, daily, and weekly frequencies in dozens of storage accounts in several regions. Artefacts were identified with complex naming patterns and categorized using blob tags and other metadata. They had the requirement to retain in archive or delete objects based on different ages in dev, test, and production environments—such as key snapshots had to be retained 10 years for production. Maintaining multiple automation runbooks to manage this required continuous updates, monitoring, tuning, and scaling. They needed a single solution that managed the processing of backup objects consistently and reliably based on this complex set of conditions at scales that easily kept pace with their growing data volumes. 

They implemented their solution using Azure Storage Actions, which enabled them to quickly compose tasks tailored to their requirements. The condition validation experience provided confidence that objects would be processed for archive or deletion as intended. And they could schedule this task for execution across all their storage accounts to get consistent results. The simplified solution saved weeks of development and eliminated the continuous overhead of resource management and monitoring, boosting productivity and consistency.  

We successfully completed the POC for Azure Storage Actions and are excited to unlock its full potential with GA. Storage Actions helped us get to a one-stop solution for backup management. It helps us tag metadata, automate blob lifecycle management, and archive to comply with our retention policy. We can eliminate multiple resource-heavy automation runbooks, and handle vast numbers of blobs effortlessly, moving them around as needed. We can drive down our storage costs, as well as streamline our entire backup process —making it scalable and consistent.

—Pooja Goel, Data Services Engineer, at PointsBet

Integrated data governance for business systems

One of our customers provides data analytics services to environmental non-profits. Input data is retained for 90 days by default for analysis, but some datasets have additional retention requirements. Organizations can purchase additional storage for key ecological datasets extending to several years for research and compliance. With Azure Storage Actions, they have been able to streamline the retention and lifecycle process for datasets using container metadata. Now, with their simplified design—when a non-profit purchases extended storage for a dataset, the corresponding blob containers are directly updated with metadata values including the dates and duration of retention. They then use Azure Storage Actions that examine the container metadata to apply immutability settings to preserve data that needs to be retained for compliance, while deleting data whose retention period has expired. This streamlined flow using Storage Actions significantly reduced the development overhead of integrating their transaction systems with lifecycle and governance subsystems, uses fewer resource types they must manage, and achieved better conformance.

Restoring selected blobs from archive at scale

When a United States state tax and revenue agency found they’d accidentally lost access to a very large volume of records, they had to rapidly restore about 600 million objects selectively from amongst billions in archive. They used Azure Storage Actions to scan their storage accounts and rehydrate specific versions of these blobs based on a naming pattern with wildcards and a time range. They were able to respond with urgency by quickly composing the condition using the no-code experience and using the interactive condition preview to immediately verify that the right blobs would be restored. The tasks they configured scaled to their data effortlessly to get the data restored in hours. Using Azure Storage Actions saved them critical time during a crisis by focusing on selectively restoring just the required objects, without having to write any code to scan billions of blobs or provision resources to speed up the process, and limited the disruption to hours instead of days.

Here are a couple more reflections from customers on their experience:

The Azure Storage Actions has transformed our approach to managing storage, enabling faster migrations and more efficient operations for our customers. This solution has streamlined data management and significantly boosted productivity.

—Charbel Nemnom, Senior Architect Azure, itnetX (Switzerland) AG

In today’s data-driven landscape, effective data governance is essential for our customers striving to maintain compliance and uphold data integrity. With Azure Storage Actions, our customers are able to define conditions and operations for retention and expiry through a no-code interface, streamlining the implementation of data governance policies. We found this approach accelerates governance initiatives and ensures data integrity consistently across our customers’ data estates. By leveraging Azure Storage Actions, our customers can confidently manage their data assets and comply with their organization policies in an increasingly complex data environment.

—Viral Patel, Principal Platform Consultant, Mesh-AI

Get started today

With general availability, you can now use Azure Storage Actions to manage the largest Storage deployments with confidence. You can start transforming data management across your organization, using our Quickstart guide and by visiting the Azure portal. You can find Azure Storage Actions in the Azure Marketplace, or from the left navigation pane of your storage account blade under Data Management. Check out our documentation for details. Pricing information is available through the product page. 

We’re committed to continuous learning and improvement, so let us know of additional data management operations you’d like to automate. And watch this space for expanded set of supported operations, more advanced conditional logic, and enhancements to the composition and management experience.

The post Unlock seamless data management with Azure Storage Actions—now generally available  appeared first on Microsoft Azure Blog.