Join Shane Gibson as he chats with Andy Cutler about the art of data modeling and the potential of AI tools to improve the art.

https://agiledata.substack.com/i/182448381/google-notebooklm-briefing

If you want to download the transcript for the podcast, head over to:

https://agiledata.substack.com/i/182448381/transcript

Listen to more podcasts on applying Agile Data patterns over at https://podcast.agiledata.io/

If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.

Subscribe: Apple Podcast | Spotify | Google Podcast  | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser |  Deezer | Podcast Addict | 

Simply Magical Data Ways of Working

Join Shane Gibson as he chats with Timo Dechau about the pattern of Metric Trees.

https://agiledata.substack.com/i/179577155/google-notebooklm-briefing

If you want to download the transcript for the podcast, head over to:

https://agiledata.substack.com/i/179577155/transcript

Listen to more podcasts on applying Agile Data patterns over at https://podcast.agiledata.io/

If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.

Subscribe: Apple Podcast | Spotify | Google Podcast  | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser |  Deezer | Podcast Addict | 

Simply Magical Data Ways of Working

Join Shane Gibson as he chats with Rob Long about patterns that you can use to document and understand your data estate.

If you want want to read a summary generated with GenAI, head over to:

https://agiledata.substack.com/i/176072946/google-notebooklm-briefing

If you want to download the transcript for the podcast, head over to:

https://agiledata.substack.com/i/176072946/transcript

Listen to more podcasts on applying Agile Data patterns over at https://podcast.agiledata.io/

If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.

Subscribe: Apple Podcast | Spotify | Google Podcast  | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser |  Deezer | Podcast Addict | 

Simply Magical Data Ways of Working

Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern for the Data Match.

The Data Match pattern provides an automated, granular comparison capability to efficiently identify and report discrepancies between two datasets, moving from row counts to specific data values.

This 'data diff' solution transforms hours of manual data reconciliation into minutes by optimising comparisons for cloud analytics databases like BigQuery, serving as a support feature for on-demand exception handling rather than a continuous trust rule.

An AgileData Engineering Pattern is a repeatable, proven approach for solving a common data engineering challenge in a simple, consistent, and scalable way, designed to reduce rework, speed up delivery, and embed quality by default.

If you want a copy of the pattern template head over to:

https://agiledata.substack.com/i/172820886/pattern-name

Discover more  AgileData Engineering Patterns over at https://agiledata.substack.com/s/agiledata-engineering-patterns

If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.

Subscribe: Apple Podcast | Spotify | Google Podcast  | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser |  Deezer | Podcast Addict | 

Simply Magical Data Ways of Working

Join Shane Gibson as he chats with Nick Zervoudis about patterns that you can use to quickly and easily define the ROI of your data products, before you build them.

https://agiledata.substack.com/i/172748370/google-notebooklm-briefing

If you want to download the transcript for the podcast, head over to:

https://agiledata.substack.com/i/172748370/transcript

Listen to more podcasts on applying Agile Data patterns over at https://podcast.agiledata.io/

If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.

Subscribe: Apple Podcast | Spotify | Google Podcast  | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser |  Deezer | Podcast Addict | 

Simply Magical Data Ways of Working

Join Shane Gibson as he chats with Johnny Winter about the core patterns that make up Dimensional (Star Schema) Modeling.

https://agiledata.substack.com/i/171239242/google-notebooklm-briefing

If you want to download the transcript for the podcast, head over to:

https://agiledata.substack.com/i/171239242/transcript

Listen to more podcasts on applying Agile Data patterns over at https://podcast.agiledata.io/

If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.

Subscribe: Apple Podcast | Spotify | Google Podcast  | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser |  Deezer | Podcast Addict | 

Simply Magical Data Ways of Working

Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern for the AgileData Activity Event Tile.

The Activity Event Tile pattern tracks a sequence of events (e.g., subscription signups, subscription payments) for a core business concept by storing minimal event data and using a hydration layer to automatically generate pre-calculated metrics as views.

This allows business users and analysts to easily understand business performance and event sequencing without complex SQL, providing immediate, consistent insights into activity movements and financial impacts

An AgileData Engineering Pattern is a repeatable, proven approach for solving a common data engineering challenge in a simple, consistent, and scalable way, designed to reduce rework, speed up delivery, and embed quality by default.

If you want a copy of the pattern template head over to:

https://agiledata.substack.com/i/170387919/pattern-name

Discover more  AgileData Engineering Patterns over at https://agiledata.substack.com/s/agiledata-engineering-patterns

If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.

Subscribe: Apple Podcast | Spotify | Google Podcast  | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser |  Deezer | Podcast Addict | 

Simply Magical Data Ways of Working

Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern for the AgileData Event Tile.

The Event Tile Data Modeling pattern captures business processes and transactions within AgileData's three-layered data architecture, modeling "who does what" at a specific point in time. It achieves this by storing only a small number of core business keys in insert-only event tables. These event records are then automatically hydrated into a "consume tile" by joining with all relevant detail attributes that were current at the exact moment the event occurred, ensuring an accurate, time-staged narrative. This provides data consumers with a rich, wide, and historically accurate view of events, simplifying the querying of complex historical changes and state transitions.

An AgileData Engineering Pattern is a repeatable, proven approach for solving a common data engineering challenge in a simple, consistent, and scalable way, designed to reduce rework, speed up delivery, and embed quality by default.

If you want a copy of the pattern template head over to:

https://agiledata.substack.com/i/169697974/pattern-name

Discover more  AgileData Engineering Patterns over at https://agiledata.substack.com/s/agiledata-engineering-patterns

If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.

Subscribe: Apple Podcast | Spotify | Google Podcast  | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser |  Deezer | Podcast Addict | 

Simply Magical Data Ways of Working

Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern of Automated Load Patterns based on Source Data Profiles.

The Automated Load Patterns based on Source Data Profiles pattern automatically profiles incoming data to determine its optimal loading method. It classifies data as either event data (immutable, append-only) or change data (evolving records) by analysing characteristics like unique key volume and column names over time. This automated classification then dictates the load strategy: an efficient partition replace for event data or a cost-effective upsert (end-dating historical records) for change data. This capability removes the manual burden and accelerates data onboarding, providing a trustworthy and efficient process even when the data type is initially unknown.

An AgileData Engineering Pattern is a repeatable, proven approach for solving a common data engineering challenge in a simple, consistent, and scalable way, designed to reduce rework, speed up delivery, and embed quality by default.

If you want a copy of the pattern template head over to:

https://agiledata.substack.com/i/168912285/pattern-name

Discover more  AgileData Engineering Patterns over at https://agiledata.substack.com/s/agiledata-engineering-patterns

If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.

Subscribe: Apple Podcast | Spotify | Google Podcast  | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser |  Deezer | Podcast Addict | 

Simply Magical Data Ways of Working

Join Shane Gibson and Nigel Vining as they describe and discuss the AgileData Engineering Pattern of Trust Rules to validate data.

The Trust Rules pattern provides automated data validation to ensure all incoming data is fit for purpose and trustworthy. It bakes in essential checks such as unique business keys and business effective dates, which run automatically upon data load or table refresh. Users can also define custom validation rules for specific columns. Results are collected, persisted, and surfaced via applications or alerts, with the system optimising validation for cost and speed through smart partitioning and clustered columns.

An AgileData Engineering Pattern is a repeatable, proven approach for solving a common data engineering challenge in a simple, consistent, and scalable way, designed to reduce rework, speed up delivery, and embed quality by default.

If you want a copy of the pattern template head over to:

https://agiledata.substack.com/i/167964917/pattern-name

Discover more  AgileData Engineering Patterns over at https://agiledata.substack.com/s/agiledata-engineering-patterns

If you just want to talk about making magic happen with agile and data you can connect with Shane @shagility on LinkedIn.

Subscribe: Apple Podcast | Spotify | Google Podcast  | Amazon Audible | TuneIn | iHeartRadio | PlayerFM | Listen Notes | Podchaser |  Deezer | Podcast Addict | 

Simply Magical Data Ways of Working