Month: July 2022

Using the Gitlab built in CI/CD Pipeline by writing and putting it in the root of your repo .gitlab-ci.yml

Well there is a lot to it but, this link has a little demo that will get you going on how to manually (hint use the “manaul” keyword and automatically kick off a build with Gitlab .gitlab-ci.yml https://about.gitlab.com/blog/2020/12/10/basics-of-gitlab-ci-updated/ Ok, that link does not show you how to use “when: manual” here is how: You can …

Using the Gitlab built in CI/CD Pipeline by writing and putting it in the root of your repo .gitlab-ci.yml Read More »

Redhat 6 / RHEL 6 / CentOS 6 – /etc/resolv.conf overwritten on reboot

The overwrite of /etc/resolv.conf is normal when the network is restarted (and host reboot), so if your DNS Servers change, I updated the file: Assuming eth0 is the only active NIC (check with “service network status”). /etc/sysconfig/network-scripts/ifcfg-eth0 PEERDNS=no DNS1=10.0.0.60 DNS2=10.2.0.32 You will have to restart the network service or reboot the Linux server. service network …

Redhat 6 / RHEL 6 / CentOS 6 – /etc/resolv.conf overwritten on reboot Read More »

Snowflake – Materialized Views – Limits and one huge advantage

The really nice feature of Snowflake’s materialized views is that they are automatically kept in sync with their underlying SQL. Snowflake is doing this in the background for us. And even if it has not been done yet, Snowflake realizes that from you SQL query, and as quickly as possible updates the MV and uses …

Snowflake – Materialized Views – Limits and one huge advantage Read More »

Data Warehouse design – Type 2 Slowly Changing Dimensions

The blog outlines one method for handling Slowly Changing Dimensions – probably the most popular method referred to as type 2. Could be applied to any data base product – but – this applies to Redshift and Snowflake and moreā€¦ A type 2 slowly changing dimension enables you to track the history of updates to …

Data Warehouse design – Type 2 Slowly Changing Dimensions Read More »

Snowflake resuming all tasks including children of a parent task

There are rules about the order or resumption, that make resuming parent and child task a pain, but here is a command that just resumes all tasks related to a parent. SELECT SYSTEM$TASK_DEPENDENTS_ENABLE(‘MYTASK’); Show tasks <database.schema>; show tasks analytics.dbt; BTW, you must suspend a tasks with parents/children before dropping them – the same restriction does …

Snowflake resuming all tasks including children of a parent task Read More »

Data Change Management for example with Snowflake

Imperative (aggregated changes) versus Declarative (by declaring end state intention – e.g. Terraform Style) A couple of popular examples of Imperative Data Change Management are: Flyway – https://flywaydb.org/download schemachange – https://pypi.org/project/schemachange/ And here is an example of a popular build tool that is declarative: Terraform – https://www.terraform.io (by Hashicorp)