Data Mesh Radio

Very quick one! The Data Mesh Summit (aka Datanova) is taking over Data Mesh Radio for the week! Sign up for an awesome two-day event (Feb 9th and 10th) here.

We will have 4 (scheduling willing) amazing guests from the Data Mesh Summit on over the next week: Max Schultze (Zalando), Dr. Colleen Tartow (Starburst), Dr. Daniel Abadi (University of Maryland), and Dr. Teresa Tung (Accenture).

In exchange, Starburst is doing a beta, sponsoring transcripts. So please let them know you want more transcripts and again, use the link to sign up to show your support!

Again, sign up here.

To get Max Schultze + Dr. Arif Wider's 'Data Mesh In Practice' book, click through here

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Mentioned articles and books:

Data Management at Scale book: https://www.oreilly.com/library/view/data-management-at/9781492054771/

Data Domains — Where do I start? https://towardsdatascience.com/data-domains-where-do-i-start-a6d52fef95d1

Implementing Data Mesh on Azure: https://towardsdatascience.com/implementing-data-mesh-on-azure-c01ee94306cd

Data Domains and Data Products: https://towardsdatascience.com/data-domains-and-data-products-64cc9d28283e

"The Blue Book" AKA Domain-Driven Design: Tackling Complexity in the Heart of Software: https://www.oreilly.com/library/view/domain-driven-design-tackling/0321125215/

Find Piethein online:

LinkedIn: https://www.linkedin.com/in/pietheinstrengholt/

Twitter: @phstrengholt / https://twitter.com/phstrengholt

Scott interviews Piethein Strengholt, Senior Cloud Solution Architect at Microsoft and author of the O'Reilly book Data Management at Scale.

Piethein shares his tips and tricks for how to approach Domain Driven Design (DDD) for data. There are puts and takes to each approach so unfortunately for those looking for an easy button, there is a lot to consider. When starting, Piethein recommends looking at your applications and deciding if there is a logical mapping to a single domain or if the application is shared across domains. As you learn more about DDD you can start to approach your domains from multiple other angles to find the best solution for your org.

There is a ton of really great advice, too much to sum up well here. Scott highly recommends reading this article on data domains by Piethein before jumping in to the podcast episode.

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn...

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Bente's LinkedIn: https://www.linkedin.com/in/bentebusch/

Scott interviews Bente Busch, Director of Teams Service Platform - essentially the applications platformm design system, and data platform - at Norwegian government entity NAV (Norwegian Labor and Welfare Department).

Bente talked about some really interesting aspects of NAV's data mesh implementation including:

• The unique challenges of having producers more excited to share the data than consumers demanding more data
• Changing the culture from doing "projects" to building products
• How the on-prem enterprise data warehouse just didn't offer them the same agility they need
• Where they are in their data mesh journey so far
• And much more

It's a very interesting look into how data mesh could be applied in government with eventual plans to share information outside of the organization.

One very interesting insight is that including those building the application platform in the data mesh self-serve platform build out has been a big win - they are already familiar with application developer workflows and how they think so they are better able to anticipate needs when it comes to building the data platform.

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Molly's LinkedIn: https://www.linkedin.com/in/vorwerck/

Monte Carlo's blog: https://www.montecarlodata.com/blog/

In this episode, Scott interviews Molly Vorwerck, head of content and communications at data observability vendor Monte Carlo. Scott asked Molly to be on as she is a great community member in general and as Monte Carlo is well known in the data space for putting out high quality content on hot-button issues. The idea is to take how Molly and team get their ideas and apply that process to figuring out major pain points internally and creating a cohesive strategy to at least start discussing them.

Molly recommends to constantly be interviewing stakeholders. She talks about interviewing people from multiple sides of a challenge, e.g. not just the data consumers but the data producers and the data engineering teams re data challenges. She tries to give people the space to tell their story and asks open-ended questions to truly get their perspective, not arrive at a pre-specified answer.

Molly talks about ways to make the other person feel valued by active listening and making the conversation mutually beneficial. It may be a person you want to interview again so building the relationship is crucial, not just extracting info in a one-time manner.

Scott and Molly dig a bit into the idea of blameless post mortems and how valuable they can be for doing data mesh, especially to figure out what happened to cause data downtime and how to prevent the same issue in the future.

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado,

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

In this somewhat controversial episode, part 1 of 2, Scott covers topics re is data mesh right for your organization. This should only be used as a jumping off point for discussions.

The 4 segment titles are:

Data quality challenges don't necessarily mean it's time for a data mesh

Data Mesh Lite

Building on a solid foundation

How 'bout now, how 'bout right now? ("Patience you must have, my young Padawan")

Take it with a grain of salt!

Mentioned content links:

Webinar with Zhamak and Sina Jahan - lessons from the trenches with data mesh

Zhamak podcast interview with Barry O'Reilly

Flexport Data Mesh Learning meetup

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Chad's contact info:

LinkedIn: https://www.linkedin.com/in/chad-sanderson/

csanderson.data at gmail.com

In this episode, Scott interviews Chad Sanderson, Head of Product: Data Platform at Convoy. This episode is part of our continuing series on data contracts and related topics. Chad covers a lot of the challenges relative to data quality, both in maintaining quality and in the challenges poor quality data can cause a company that is heavily reliant on data.

Chad also shares his tale of trying to implement data mesh at Convoy via a large-scale inverse Conway Maneuver.

Chad covered 4 categories of data quality pain, which he calls the "4 Horseman of Data Quality" in this post:

Omission - metadata is missing; no tool out today that solves the omission problem, so users have to bounce between too many tools to try to figure out data specifics like where it came from, the specific meaning, what it's trying to convey, etc.

Waste: growth of unused, unmaintained, or duplicated data; waste happens when the cost of creating new data is less than using something already created

Divergence: the growing divide between what's going on in "the real world" and what's happening in your data warehouse; your business logic, unless it is constantly maintained and updated, starts to diverge from what is happening to your business so what you show on dashboards and reports no longer matches business reality

Downtime: periods of time where the data is missing, wrong, late, etc.; traditionally what most people think of regarding data quality issues

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman):

Quick "episode"

1. EXPLICIT PERMISSION: skip episodes you don't think will be useful - that is why the BLUFs exist
2. In general, we will publish interview episodes on Tuesday and Friday morning. If there is a mesh musing, it will be published Wednesday morning
3. Starting "takeover weeks" with conferences. Look for one soon. If you have a conference that should do one, get in touch: scott at datastax.com
4. Be a guest. Seriously. This is not an "experts only" podcast! It's a learning out loud one.
5. Data contracts deep dive continues. 3-4 more episodes lined up by end of Feb. Next 2 deep dives are 1) Domain Driven Design for Data + Data-Centric Application Development and 2) Your Data Mesh Proof of Concept
6. I need guests and I need topics. Stop being shy. Do not expect an explicit invitation, step up.
7. New podcast from me coming in early to mid-Feb. This Week I Learned in Data. Be a contributor sharing what you learned (not looking for contributions every week!) or suggest contributors.

P.S. Please do rate and review the podcast. It honestly really does help for visibility. I don't care about the ratings, I just want to make sure if it's a resource, people can find it!

Scott's contact info: scott at datastax.com

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts (most interviews from #32 on) here

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Links from show:

Benn's Twitter: @bennstancil / https://twitter.com/bennstancil

Benn's Substack: https://benn.substack.com/

Benn's post on entities: https://benn.substack.com/p/metadata-money-corporation

Starship Technologies Data Mesh Post: https://medium.com/starshiptechnologies/dodging-the-data-bottleneck-data-mesh-at-starship-5925a2de45e6

In this episode Scott interviewed Benn Stancil, co-founder and Chief Analytics Officer at Mode about his emerging concept of global definitions that he is calling an "entity" - essentially, how are people defining terms like customer and what does that mean in each instance so people are on the same page. Data mesh also requires some clear collaboration on definitions, whether centralized or decentralized, in the federated governance pillar. This interview was generated from a Twitter conversation here.

Benn then went into some details about how he views the importance of having visibility into how data "breaks" so it is much easier to identify and fix and that limiting custom integration is crucial so when you fix at the source, it properly propagates downstream. They wrapped up discussing the need to make data producers' lives easier while simultaneously doing the same with data consumers.

Overall, there are some agreements and disagreements and Scott came out thinking more about what are the real causes of pain that would make a full journey to data mesh make sense. It's a good episode to see some of the challenges people are trying to tackle outside of the data mesh community.

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him at community at datameshlearning.com or on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm,

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts (most interviews from #32 on) here

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Juan's contact info and related links:

Email: juan at data.world

Twitter: @juansequeda / https://twitter.com/juansequeda

LinkedIn: https://www.linkedin.com/in/juansequeda/

Catalog & Cocktails Podcast: https://data.world/podcasts/

Juan's post about Zhamak's appearance on the Data Engineering Podcast: https://www.linkedin.com/pulse/my-takeaways-data-engineering-podcast-episode-mesh-zhamak-sequeda/

Juan's post about knowledge first: https://www.linkedin.com/feed/update/urn:li:activity:6884179569277059072/

Standards related links:

Dublin Core Metadata Initiative: https://dublincore.org/

RDF (Resoruce Description Framework): https://www.w3.org/2001/sw/wiki/RDF

OWL (Web Ontology Language): https://www.w3.org/OWL/

PROV-O: The PROV Ontology: https://www.w3.org/TR/prov-o/

In this episode, Scott interviews Juan Sequeda, Principal Scientist at data.world and co-host of the Catalog and Cocktails podcast. They discussed Juan's knowledge first approach: putting the meaning and value of the data first instead of focusing on the amount of data we are handling/producing. Knowledge first has 3 components, 1) context, 2) people, and 3) relationships. Juan is a big proponent of knowledge graphs and the relationships side is one many people miss.

Juan also gave some thoughts on what his approach to data mesh hinges on: treating data as a product and finding a balance between centralization and decentralization for all the aspects of building out an implementation. Juan mentioned Intuit's approach of fixed, flexible/extensible, or customizable as a good general tool and to look for (and embrace) what he calls intellectual friction.

Lastly, Juan and Scott talked about the general drive to reduce toil, of reinventing the wheel re data interoperability and standard schemas in data mesh. Juan points to a lot of existing research and standards - e.g. RDF, OWL, and many more (see below) - as a starting point.

Data Mesh Radio is...

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Peter's contact info and relevant links:

Email: peter at cloudshuttle.com.au

LinkedIn: https://www.linkedin.com/in/peterhanssens/

Twitter: @petehanssens / https://twitter.com/petehanssens

Cloud Shuttle website: https://www.cloudshuttle.com.au/

Sydney Data Engineering Community: https://sydneydataengineers.github.io/

DataEngBytes Conference: https://dataengconf.com.au/

Article mentioned by Francois Nguyen (CIO of L'Oreal): https://francois-nguyen.blog/2021/03/07/towards-a-data-mesh-part-1-data-domains-and-teams-topologies/

In this episode, Scott interviews Peter Hanssens, Founder and Solutions Architect at cloud/serverless consulting company Cloud Shuttle. Peter also runs a few large data engineering focused communities (meetups, Slack, conferences) in Australia. Peter had reached out about how can startups and SMEs (small to medium enterprises) get the benefits of data mesh without the costs of building a solution fit for a 10,000+ employee company. Peter does a great job seeking information on behalf of his constituents :)

We covered a number of topics including:

• The needs for cultural change as technology will only get you so far
• The beauty of pay-per-use solutions for startups/SMEs - and where there are still gaps in the market
• The challenges of not having a large pool of data engineers to build and manage a platform - or to help domains model their data
• The benefits of centralization until it starts to cause bottlenecks
• The importance of tracking lineage for upstream producers/domains - to see who is using your data and why/how
• And much, much more

I think you will really enjoy Peter's perspectives and there are some useful conclusions if not a perfect blueprint for startups.

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

If you...