PostgreSQL Logical Replication (on a MacBook) example POC

This text based blog borrows from and somewhat enhances the example shown in: https://www.youtube.com/watch?v=OvSzLjkMmQo The enhancement is showing you how to create these replications in custom schemas – not just public – and some gotch’yas tips when creating multiple instances on Mac OS. Regardless many thanks to the folks that created the YouTube above, it …

PostgreSQL Logical Replication (on a MacBook) example POC Read More »

AWS Glue Crawler Schema Inference

https://repost.aws/knowledge-center/glue-crawler-detect-schema important: https://docs.aws.amazon.com/glue/latest/dg/crawler-configuration.html#crawler-grouping-policy (copied from the transient link above) When you run your AWS Glue crawler, the crawler does the following: Classifies the data Groups the data into tables or partitions Writes metadata to the AWS Glue Data Catalog Review the following to learn what happens when you run the crawler and how the crawler …

AWS Glue Crawler Schema Inference Read More »

Apache Hudi

When define a Hudi table you must define one of two supported data storage types. Supported Hudi dataset storage types: Copy on write Merge on read When you create a Hudi dataset, you specify that the dataset is either copy on write or merge on read. Copy on Write (CoW) – Data is stored in a …

Apache Hudi Read More »

Securing AWS API Gateway

A good place to get started on this important subject: https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-control-access-to-api.html https://docs.aws.amazon.com/apigateway/latest/developerguide/security.html