Data Mesh Radio

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Dan's LinkedIn: https://www.linkedin.com/in/demersdan/

Cloud Information Model Standard: https://cloudinformationmodel.org/

Cinchy website: https://cinchy.com/

In this episode, Scott interviews Dan DeMers, Co-Founder and CEO of Cinchy, a dataware platform / data fabric provider. Dan shares his thoughts on why data-centric application design is the best way to deal with the challenges of applications and analytics needing the same data for different purposes - the current approach is to let the application schema evolve whenever and however necessary and the underlying data applications suffer. Dan's view is "share access to data, not copies."

The interview ties loosely with previous interviews re schema/data contracts and data testing as Dan argues using a dataware approach will prevent the issues of an evolving application schema breaking data consumption downstream.

It isn't all rosy as this will take a fair bit of work for an organization to move to this approach. Food for thought and the first of a series of interviews re DDD (domain-driven design) for data and data-centric application development.

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

In this episode, Scott discusses three concepts that are at best a concern. Consider it a late Grinch-inspired present for Xmas :)

Reverse ETL meets a real need for analytical data being pushed into CRM, marketing, and other similar systems. But treating another pipeline as a first order concern is fraught with the same issues of most similar data pipeline treatment: who owns it, how does it evolve, who is observing/monitoring it for uptime and semantic drift, etc.? Should we look to create data products on the mesh to serve those needs instead of another ETL tool?

Some organizations implementing data mesh are forcing their domains to consume any analytics from their own data products on the mesh. The good of this is that it aligns the domain with creating a high-quality data products. But will those data products be designed to fit the general organizational needs or specifically the domain's needs?

There is an emerging push for software engineers to also own the data modeling. To get to a place where this is even feasible, don't we need far better abstractions for domains to _do_ the data modeling? And will this overload software engineers that are already dealing with a metric buttload of technologies and requirements already? Where would a junior engineer fit in that kind of organization? Does this mean more software engineers on the team -> 2 pizza teams now 3? 4? 5? 10? Maybe we pump the brakes on this for now?

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Jesse's contact info:

Email: jesse at tag.bio

LinkedIn: https://www.linkedin.com/in/jessepaquette/

Twitter: @bzdyelnik / https://twitter.com/bzdyelnik

Website: https://tag.bio/

Tag.bio vendor interview for Data Mesh Learning: https://www.youtube.com/watch?v=acQADu7ttqQ

In this episode, Jesse Paquette, Chief Science Officer and Co-founder at Tag.bio - a data platform vendor in the life sciences space, and Scott dive a bit deeper into data quality in general, especially data testing and versioning.

You can see the LinkedIn post that sparked this discussion here

Jesse recommends a number of things to ensure data quality, especially data testing and versioning. This includes versioning of 1) the code used to create the data (generally the ETL code), 2 the schema, 3) the business logic layer, and 4) timestamping / temporality based versioning.

Jesse's general calls to action are 1) make data testing frameworks so testing is much less tedious and time consuming; 2) work with stakeholders to gain trust in the data and then continue the dialogue to keep said trust; and 3) create schema/domain model blueprints so that domains have a starting point - whether they use it is irrelevant but shortening the path to a working domain model is crucial.

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm,

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Abhi's contact info:

Twitter: https://twitter.com/_abhisivasailam

LinkedIn: https://www.linkedin.com/in/abhi-sivasailam/

Abhi's Data Mesh Learning Meetup: https://www.youtube.com/watch?v=-POiudR2_R0

Debezium blog post mentioned by Abhi: https://debezium.io/blog/2019/02/19/reliable-microservices-data-exchange-with-the-outbox-pattern/

In this episode, Abhi Sivasailam, Head of Growth and Analytics at unicorn startup Flexport, and Scott deep dive into data contracts. They covered a LOT of ground including:

• What is a data contract and how does it relate to API contracts and schema contracts
• Why data contracts are so crucial to treating data like a product and keeping data usable by consumers
• Abhi's rules for a viable data contract
• The importance of the analytics engineer to overall data usefulness, especially re domain data ownership in data mesh
• Why of the socio of the socio-technical approach to data ownership is the most crucial aspect
• The lack of proper tooling to monitor and execute data contracts
• How to minimize disruptive changes to downstream data consumers
• The issues with domains sharing their data as it is persisted/stored in the database instead of sharing the context via the domain model
• Much, much more

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes,

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Matt's contact info:

LinkedIn: https://www.linkedin.com/in/matthewdarwindba/

Twitter: @EvoDBA / https://twitter.com/EvoDBA

Matt's post on platform reuse: https://medium.com/slalom-data-analytics/data-mesh-is-the-argument-a-strawman-3cffaf55ce5e

Matt's LinkedIn poll on testing data pipelines: https://www.linkedin.com/feed/update/urn%3Ali%3Aactivity%3A6877216459458719744/

In this episode, Matthew Darwin, Principal Data Engineer at Slalom Consulting, and Scott cover a wide range of topics including:

Re an article Matt had posted on data platform re-use and shifting data ownership left:

• You can re-use the technologies you already know and love when building a data platform - there is literally no good reason to toss those out the window
• If your existing setup enables domains to easily transform, serve, and store their data, sure use your existing configuration; if not, there will need to be changes
• Your data platform will need to evolve and it is okay to start with a bit of an underwhelming data platform

Other nuggets and interesting topics:

• Insights from direct client engagements doing data mesh
• What makes for a good data mesh PoC
• The usefulness of data product blueprints
• How data mesh is still bleeding edge and is therefore not for everyone
• Slowing down to move faster / the long-term negatives of always looking for quick wins
• Why you can't just expose your operational data model as a data product
• The importance of data product interoperability - even at the PoC phase
• How crucial the organizational aspects of data mesh really are
• Much more

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

All...

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

In this episode, Olivier Wulveryck, Senior Consultant at consulting company OCTO Technology, and Scott discuss all things schema contracts. The episode will probably leave you with more questions than answers at this point as the concept/practice is still emerging. But you will get a good sense for how to do it right - and how not to do it too! - from the conversation.

Olivier's contact info

LinkedIn: https://www.linkedin.com/in/olivierwulveryck/

Email: olivier.wulveryck at octo.com

Twitter: @owulveryck / https://twitter.com/owulveryck

Olivier's post on schema contracts: https://blog.octo.com/en/pov-a-streaming-communication-platform-for-the-data-mesh/

(mentioned in BLUF) Sofia Tania presentation on Data Mesh Testing: https://www.youtube.com/watch?v=stNZQESndAA

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

In this episode, Scott interviews Wannes Rosiers, CTO at Golazo Group, who previously led DPG Media's data mesh transition. They discussed a lot of different sub-topics around data products in data mesh including:

1. Wannes' framework re his 3 types of data products
2. The importance of a specific purpose for each data product
3. Data product evolution from single purpose to multi-purpose
4. The owner roles/functions related to data products in data mesh
5. How to get moving with an initial data mesh implementation
6. Data product interoperability and global definitions and identifiers

Wannes' LinkedIn: https://www.linkedin.com/in/wannes-rosiers/

Wannes' data mesh related content:

Meetup presentation: https://www.youtube.com/watch?v=l_5fkpweQwM

Domain Driven Design for Data at DPG Media: https://dpgmedia-engineering.medium.com/ddd-data-area-at-dpg-media-f0130e4d9766

A specific run down of data mesh at DPG: https://dpgmedia-engineering.medium.com/data-mesh-at-dpg-media-dfebdd612087

A co-authored post with Snowflake: https://levelup.gitconnected.com/data-mesh-a-self-service-infrastructure-at-dpg-media-with-snowflake-566f108a98db

Short video describing DPG's overall data journey, not just data mesh: https://itexecutive.nl/leveraging-the-power-of-data-technology/data-strategy/van-goed-op-weg-zijn-met-data-naar-data-rock-star-in-twee-jaar/

Golazo Careers Page: https://www.golazo.com/jobs/

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

If you want to be a guest or give feedback (suggestions...

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Host Scott Hirleman gives some helpful context around 3 separate data mesh topics:

1. Why Data Virtualization is not a good fit for creating and managing your data products in data mesh
2. A reply to a wonderful article by Matthew Darwin (link below) asking if we can keep our existing data platform / architecture and only decentralize our data ownership - surprise, surprise, Scott argues that it isn't that simple
3. Introducing the concept of a speculative data product to solicit feedback in a bit of internal data product marketing

Matthew Darwin article: https://medium.com/slalom-data-analytics/data-mesh-is-the-argument-a-strawman-3cffaf55ce5e

Intuit article referenced in spec data product segment: https://medium.com/intuit-engineering/intuits-data-mesh-strategy-778e3edaa017

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Another short one, basically asking people to be on the show and to provide feedback. Let's work together as a broader community to provide content that is useful and helpful. So please let Scott know if you want to be a guest or if you have feedback about existing content or what content you want to see.

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts (most interviews from #32 on) here

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

In this episode, Scott interviews Paolo Platter, CTO at Agile Lab, about data product flow or 1) how to identify what data products you already have that you need to migrate to the data mesh and 2) what data products you need to create to further the business processes that will move your organization forward. Paolo has been trialing this methodology on a number of Agile Lab's consulting customers.

Paolo's contact info and his great blog post on this topic:

Email: paolo.platter AT agilelab.it

LinkedIn: https://www.linkedin.com/in/paoloplatter/

Blog post: https://www.agilelab.it/how-to-identify-data-products-welcome-data-product-flow/

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him at community at datameshlearning.com or on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf