Kinesis Data Firehose now supports dynamic partitioning to Amazon S3

By Dustin Ward

AWS FeedKinesis Data Firehose now supports dynamic partitioning to Amazon S3 Amazon Kinesis Data Firehose provides a convenient way to reliably load streaming data into data lakes, data stores, and analytics services. It can capture, transform, and deliver streaming data to Amazon Simple Storage Service (Amazon S3), Amazon Redshift, Amazon Elasticsearch Service, generic HTTP endpoints,…

Define and run Machine Learning pipelines on Step Functions using Python, Workflow Studio, or States Language

By Dustin Ward

AWS FeedDefine and run Machine Learning pipelines on Step Functions using Python, Workflow Studio, or States Language You can use various tools to define and run machine learning (ML) pipelines or DAGs (Directed Acyclic Graphs). Some popular options include AWS Step Functions, Apache Airflow, KubeFlow Pipelines (KFP), TensorFlow Extended (TFX), Argo, Luigi, and Amazon SageMaker…

Practical Entity Resolution on AWS to Reconcile Data in the Real World

By Dustin Ward

AWS FeedPractical Entity Resolution on AWS to Reconcile Data in the Real World This post was co-written with Mamoon Chowdry, Solutions Architect, previously at AWS. Businesses and organizations from many industries often struggle to ensure that their data is accurate. Data often has to match people or things exactly in the real world, such as…