26 create additional metadata columns for control snap tables v0 and v1 #419
26 create additional metadata columns for control snap tables v0 and v1 #419
Conversation
… to v1 snapshot tables
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
|
dbt test combined result: ❌ DetailsRESULTS for Synapse: RESULTS for Postgres: RESULTS for BigQuery: RESULTS for Redshift: RESULTS for Snowflake: RESULTS for Exasol: RESULTS for Fabric: RESULTS for Oracle: RESULTS for Databricks: RESULTS for SQL Server: Link to workflow summary: https://github.com/ScalefreeCOM/datavault4dbt-ci-cd/actions/runs/23790859872 |
…v1 (#419) * REDSHIFT.control_snap_v0: added is_eom, is_quarterly, is_eoq, is_eoy * BIGQUERY.control_snap_v0: added is_eoq * SNOWFLAKE.control_snap_v0: added is_eom, is_quarterly, is_eoq, is_eoy * DATABRICKS.control_snap_v0: added is_eoq * POSTGRES.control_snap_v0: added is_eom, is_quarterly, is_eoq, is_eoy * ORACLE.control_snap_v0: added is_eom, is_quarterly, is_eoq, is_eoy * feat: reworked control_snap_v0 functionality * feat: updated the v1 dependencies * feat: added documentation for weekly calculations * feat: removed explicit date columns in v0 and added all boolean flags to v1 snapshot tables * fix: syntax corrections * fix: added macro file for correct call and updated adapter calls * fix: corrected snowflake syntax * feat: added adapter specific macros to first_day_of_week macro * fix: fixed macro call * fix: added else statement to macro block for missing var * feat: updated snapshot logic for user handling * fix: fixed bigquery control snap adapter * fix: fixed some snapshot logic that was wrong in testing * fix: reworked snapshot logic once more and fixed the adapter tables * feat: updated docs --------- Co-authored-by: umattern Co-authored-by: Theo Kiehn <162969167+tkiehn@users.noreply.github.com> Co-authored-by: Tim Kirschke <81677440+tkirschke@users.noreply.github.com>
Description
following additional metadata columns in all Control Snap v0 and v1 models were added
in dbt project yml file there is now an additional variable, called datavault4dbt.first_day_of_week, which takes in different values for adapters. For example BigQuery defaults to a 1 = Sunday and 2 = Monday, whereas Databricks defaults to 0 = Sunday and 1 = Monday. The default setting for all adapters is Monday as the start of the week.
The possible values for all adapters are documented in a table in the snapshot v0 docs.
Fixes #(issue)
Type of change
How Has This Been Tested?
Ran the snapshot models and tests in automatic tests repo on databricks adapter for both datavault4dbt.first_day_of_week values 1 and 7. Created several new tests such as a golden snapshot table to verify the snapshot logic in general and especially the first_day_of_week logic.