Nov 2024 schema #32

fivetran-reneeli · 2025-01-31T17:33:02Z

PR Overview

This PR will address the following Issue/Feature: #28

This PR will result in the following new package version: v0.5.0-a1

Schema changes from Nov 2024

Please provide the finalized CHANGELOG entry which details the relevant changes included in this PR:

to be completed

PR Checklist

Basic Validation

Please acknowledge that you have successfully performed the following commands locally:

dbt run –full-refresh && dbt test
dbt run (if incremental models are present) && dbt test

Before marking this PR as "ready for review" the following have been applied:

The appropriate issue has been linked, tagged, and properly assigned
All necessary documentation and version upgrades have been applied
docs were regenerated (unless this PR does not include any code or yml updates)
BuildKite integration tests are passing
Detailed validation steps have been provided below

Detailed Validation

Please share any and all of your validation steps:

If you had to summarize this PR in an emoji, which would it be?

💃

models/apple_store__app_version_report.sql

…sistency test, update changelog

fivetran-joemarkiewicz

@fivetran-reneeli thanks for this PR. A few comments below following this review.

models/apple_store__app_version_report.sql

…d update models

fivetran-reneeli · 2025-02-05T20:45:28Z

models/intermediate/int_apple_store__date_spine.sql

+
+{% set first_date_query %}
+
+    select min(date_day) as min_date_day


If this makes sense, I will add the subscription staging models too. Just wanted to have a proof of concept first before spending time adding the subscription logic.

So the idea here is to get a date spine generated using the earliest date across all events and then use that earliest date as the starting point? I think this is a good approach!

However, to brainstorm a bit more - is there a simper way we could achieve this by using one table? This could be a performance update in the future, but I want to think through this. Can we make the claim that crashes and engagements wouldn't come until there is a download and therefore could just rely on the min date of the download daily? I'm not saying that's true, but want to think through any of these simpler possibilities.

Not required for pre-release, but wanting to bump this question to ensure it doesn't get lost.

fivetran-joemarkiewicz

@fivetran-reneeli please see my review comments below. There are a few change requests that will require a number of changes. Let me know if you would like to sync on any of my comments or open ended questions and approaches.

Ultimately, once the date spine issue is sorted out the end reports will be in a good spot! Although we will want to get the other comments addressed before pre-release.

README.md

packages.yml

models/intermediate/int_apple_store__date_spine.sql

CHANGELOG.md

DECISIONLOG.md

models/apple_store__app_version_report.sql

models/intermediate/int_apple_store__date_spine.sql

fivetran-joemarkiewicz · 2025-02-10T20:26:24Z

models/intermediate/int_apple_store__date_spine.sql

+
+{% set first_date_query %}
+
+    select min(date_day) as min_date_day


So the idea here is to get a date spine generated using the earliest date across all events and then use that earliest date as the starting point? I think this is a good approach!

However, to brainstorm a bit more - is there a simper way we could achieve this by using one table? This could be a performance update in the future, but I want to think through this. Can we make the claim that crashes and engagements wouldn't come until there is a download and therefore could just rely on the min date of the download daily? I'm not saying that's true, but want to think through any of these simpler possibilities.

models/apple_store__app_version_report.sql

…p as grain

fivetran-joemarkiewicz

A few change requests and questions. Once these are addressed this should be ready for release review for pre-release.

CHANGELOG.md

README.md

dbt_project.yml

fivetran-joemarkiewicz · 2025-02-11T21:34:58Z

models/intermediate/int_apple_store__date_spine.sql

+
+{% set first_date_query %}
+
+    select min(date_day) as min_date_day


Not required for pre-release, but wanting to bump this question to ensure it doesn't get lost.

fivetran-joemarkiewicz · 2025-02-11T21:36:55Z

packages.yml

+  - git: https://github.com/fivetran/dbt_apple_store_source.git
+    revision: nov_2024_schema
+    warn-unpinned: false


Reminder that this will need to be the following before cutting the pre-release.

Suggested change

- git: https://github.com/fivetran/dbt_apple_store_source.git

revision: nov_2024_schema

warn-unpinned: false

- package: fivetran/apple_store_source

version: 0.5.0-a1

models/apple_store__territory_report.sql

models/apple_store.yml

README.md

Co-authored-by: Joe Markiewicz <[email protected]>

packages.yml

Co-authored-by: Joe Markiewicz <[email protected]>

fivetran-joemarkiewicz

A few small change requests before approval

fivetran-joemarkiewicz · 2025-02-19T15:08:15Z

CHANGELOG.md

@@ -1,4 +1,14 @@
-# dbt_apple_store version.version
+# dbt_apple_store v0.5.0-a1  


Same comment from the source to create a duplicate entry for the official v0.5.0 release while also retaining the v0.5.0-a1 release for posterity.

fivetran-joemarkiewicz · 2025-02-19T15:10:03Z

README.md

@@ -54,7 +55,7 @@ Include the following apple_store package version in your `packages.yml` file:
 ```yaml
 packages:
  - package: fivetran/apple_store
-    version: [">=0.4.0", "<0.5.0"] # we recommend using ranges to capture non-breaking changes automatically
+    version: 0.5.0-a1


This needs to now reflect what will be the updated official range.

fivetran-joemarkiewicz · 2025-02-19T15:10:20Z

README.md

@@ -146,7 +147,7 @@ This dbt package is dependent on the following dbt packages. These dependencies
 ```yml
 packages:
    - package: fivetran/apple_store_source
-      version: [">=0.4.0", "<0.5.0"]
+      version: v0.5.0-a1


This will need to now be the updated official range of the source

fivetran-joemarkiewicz · 2025-02-19T15:11:05Z

CHANGELOG.md

+# dbt_apple_store v0.5.0-a1  
+[PR #32](https://github.com/fivetran/dbt_apple_store/pull/32) includes the following updates:
+
+## Breaking Changes: Schema Change


I would also add in here the bugfix of materializing the reporting grains as tables. This will result in extra tables, but it's important to highlight why this change was made.

fivetran-joemarkiewicz · 2025-02-19T15:12:17Z

packages.yml

+  - git: https://github.com/fivetran/dbt_apple_store_source.git
+    revision: nov_2024_schema
+    warn-unpinned: false
+
+  # - package: fivetran/apple_store_source
+  #     version: 0.5.0-a1


Reminder that this will need to be updated before merge and release

fivetran-reneeli added 7 commits January 30, 2025 23:10

update source type report

2f1fdc1

update app version report

046da4c

updates

632def4

seed file deletions and changes and additions

7084f63

update version and configs

c3a9b78

model revamps

5559078

subscription report

1ddd8c0

fivetran-reneeli commented Jan 31, 2025

View reviewed changes

models/apple_store__app_version_report.sql Outdated Show resolved Hide resolved

fivetran-reneeli added 12 commits January 31, 2025 16:23

rm int models

9b0b948

end model revisions

6746c2b

new int models

dd7458d

rename references

2bf9dfe

update yml with new fields, rm old fields, update uniqueness

c7df6f7

readme update and version bump

f941c5f

new int models

f61eafe

rm old int models

5c26fa4

updated reports, changelog, docs

77fd1f0

docs

5555efc

deps

105389c

schema

aeb2564

fivetran-reneeli requested a review from fivetran-joemarkiewicz February 1, 2025 06:33

fivetran-reneeli self-assigned this Feb 1, 2025

rm integrity territory report since staging model is removed, fix con…

e3e45db

…sistency test, update changelog

fivetran-joemarkiewicz requested changes Feb 3, 2025

View reviewed changes

models/apple_store__app_version_report.sql Outdated Show resolved Hide resolved

models/apple_store__app_version_report.sql Outdated Show resolved Hide resolved

models/apple_store__app_version_report.sql Outdated Show resolved Hide resolved

fivetran-reneeli added 5 commits February 4, 2025 10:26

seed updates

9fc03a1

style updates and regen docs

90753f0

rm active devices l30 days and update decision log. add date spine an…

43d4f81

…d update models

docs

cfafe69

switch from cross join to left join

4dc1890

fivetran-reneeli commented Feb 5, 2025

View reviewed changes

fivetran-reneeli added 2 commits February 5, 2025 23:35

rm empty value for source_type

75bfc3d

schema

661da32

fivetran-reneeli requested a review from fivetran-joemarkiewicz February 6, 2025 16:04

fivetran-reneeli added 4 commits February 6, 2025 15:42

make date spine a table

729bef4

update date spine and docs

c387358

make prerelease a1

4e4d751

docs

25ffcac

fivetran-joemarkiewicz requested changes Feb 10, 2025

View reviewed changes

fivetran-reneeli added 6 commits February 10, 2025 19:15

fix to use cross join to populate dimensions on 0 value rows

5d48c99

fix for overview report, remove prereporting logic since it's just ap…

1a1c0ea

…p as grain

updates

76dd744

make int models for reporting grain

b153019

modulate CTEs into int models

964b8a3

updates

5f3333e

fivetran-reneeli requested a review from fivetran-joemarkiewicz February 11, 2025 21:28

fivetran-joemarkiewicz requested changes Feb 11, 2025

View reviewed changes

fivetran-reneeli and others added 5 commits February 11, 2025 17:30

Update dbt_project.yml

1e7c869

Co-authored-by: Joe Markiewicz <[email protected]>

Update README.md

afc2caa

Co-authored-by: Joe Markiewicz <[email protected]>

add upstream changes to doc, update territory def and logic, gen docs

4589fb7

rmm unnecessary coalesce

32e773e

changelog and deps

2a0ba4a

fivetran-joemarkiewicz reviewed Feb 11, 2025

View reviewed changes

packages.yml Outdated Show resolved Hide resolved

fivetran-reneeli and others added 6 commits February 11, 2025 18:47

Update packages.yml

1e57b4b

Co-authored-by: Joe Markiewicz <[email protected]>

temp change back deps

f27cd33

schema

42cfa02

modified seeds

cb9818f

change name of int model bs same as downsteam in app reporting

9dda153

updates, add bit about detailed vs standard, update min date

8076c46

fivetran-reneeli requested a review from fivetran-joemarkiewicz February 14, 2025 23:06

fivetran-joemarkiewicz requested changes Feb 19, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Nov 2024 schema #32

Nov 2024 schema #32

fivetran-reneeli commented Jan 31, 2025 •

edited

Loading

fivetran-joemarkiewicz left a comment

fivetran-reneeli Feb 5, 2025

fivetran-joemarkiewicz Feb 10, 2025

fivetran-joemarkiewicz Feb 11, 2025

fivetran-joemarkiewicz left a comment

fivetran-joemarkiewicz Feb 10, 2025

fivetran-joemarkiewicz left a comment

fivetran-joemarkiewicz Feb 11, 2025

fivetran-joemarkiewicz Feb 11, 2025 •

edited

Loading

fivetran-joemarkiewicz left a comment

fivetran-joemarkiewicz Feb 19, 2025

fivetran-joemarkiewicz Feb 19, 2025

fivetran-joemarkiewicz Feb 19, 2025

fivetran-joemarkiewicz Feb 19, 2025

fivetran-joemarkiewicz Feb 19, 2025


		{% set first_date_query %}

		select min(date_day) as min_date_day

		@@ -1,4 +1,14 @@
		# dbt_apple_store version.version
		# dbt_apple_store v0.5.0-a1

Nov 2024 schema #32

Are you sure you want to change the base?

Nov 2024 schema #32

Conversation

fivetran-reneeli commented Jan 31, 2025 • edited Loading

PR Overview

PR Checklist

Basic Validation

Detailed Validation

If you had to summarize this PR in an emoji, which would it be?

fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fivetran-joemarkiewicz Feb 11, 2025 • edited Loading

Choose a reason for hiding this comment

fivetran-joemarkiewicz left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fivetran-reneeli commented Jan 31, 2025 •

edited

Loading

fivetran-joemarkiewicz Feb 11, 2025 •

edited

Loading