Data Mesh Radio

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here and their great data mesh resource center here. You can download their Data Mesh for Dummies e-book (info gated) here.

Kiran's LinkedIn: https://www.linkedin.com/in/kiran-prakash/

Kiran's article on the 'Curse of the Data Lake Monster': https://www.thoughtworks.com/insights/blog/curse-data-lake-monster

In this episode, Scott interviewed Kiran Prakash, Principal Engineer at Thoughtworks.

Some key takeaways/thoughts from Kiran's point of view:

1. ?Controversial?: You MUST have exec sponsorship to move forward with your data mesh implementation. You need the top-down push for necessary reorganization when the time comes. Scott note: only kinda controversial, really more often ignored :D
2. ?Controversial?: Data mesh, if done well, doesn't need to have a huge barrier to entry. That's a misconception. If you think about gradual improvement/evolution, you'll be on the right track.
3. "The Curse of the Data Lake Monster" was like the data field of dreams - there was expectation that if you build a great data lake, value will just happen. If you ingest and process as much as you can, the use cases will just happen. And it really wasn't the case. So we should apply product thinking to data to focus on what matters.
4. The 'Curse' was a manifestation of Conway's Law - the strong separation between IT and the business led to mismatched goals and subpar outcomes. With microservices, that started to be much less of an issue on the operational plane so why not try with data?
5. It's easy to lose sight of Conway's Law and aim for distributed architecture first but the organizations doing data mesh well are changing their architectural and cultural approaches and patterns together. Don't...

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here and their great data mesh resource center here. You can download their Data Mesh for Dummies e-book (info gated) here.

Vanya's LinkedIn: https://www.linkedin.com/in/vanyaseth1809/

In this episode, Scott interviewed Vanya Seth, Head of Technology for Thoughtworks India and Global 'Data Mesh Guild' Lead for Thoughtworks. To be clear, Vanya was only representing her own views on the episode.

Some key takeaways/thoughts from Vanya's point of view:

1. Data mesh is at a similar inflection point to where microservices was a decade ago. Let's not relearn all the hard lessons they already learned. We should adapt/contextualize to data of course but we can skip a lot of the anti-patterns.
2. Similarly, many people are stuck thinking "there's no way that could work" regarding data mesh like they were when people suggested development and operations be combined in DevOps. It's understandable - it's hard to imagine a post monolithic world when all you've known is monoliths.
3. ?Controversial?: We should try hard to prevent creating the fear of missing out (FOMO) for those not doing data mesh. If data mesh isn't right for your org, especially if it isn't right at this time, that's perfectly okay. Don't take on the overhead cost of data mesh if it won't bring more value than cost. Scott note: PREACH!
4. ?Controversial?: Some CDOs or CAOs, their organizations don't really get the value of data so they are implementing data mesh to try to prove out value and make their mark. That can obviously create issues if their organizations aren't ready.
5. A few indicators an org is ready for data mesh (see below for expanded context): A) data/AI investments are not delivering the promised/expected returns and/or it's hard to point to the value delivered in general from data/AI investments; B) the organization is attempting...

Sponsored by NextData, Zhamak's company that is helping ease data product creation.

For more great content from Zhamak, check out her book on data mesh, a book she collaborated on, her LinkedIn, and her Twitter.

This episode is part of the greater AI/ML conversation I had with Zhamak but it's super important to emphasize the importance of trust - enough so that I created a separate quick episode on it. Not just trust in the data itself but that there is easy access and there will be going forward. A lot of the things we have done in data historically has been defensive in nature - especially grabbing a copy of the data now because who knows when you'll get access to it again.

What if we can implicitly trust that there has been care and foresight in preparation of the data I find, that there is an owner I can ask if I'm confused or curious, that my access won't suddenly go away or that what's there won't suddenly change without my knowledge? In ML/AI, the data scientists have done things in ways that made sense to their situation and challenges. What happens when we make trust inherent? What incremental value does that drive?

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Data Mesh Radio episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Sponsored by NextData, Zhamak's company that is helping ease data product creation.

For more great content from Zhamak, check out her book on data mesh, a book she collaborated on, her LinkedIn, and her Twitter.

This episode is part of the greater AI/ML conversation I had with Zhamak. To start, Zhamak recognizes we aren't where we want to be in terms of capabilities - ways of working or tooling - to make this a reality just yet. But, if we can make it so data scientists can trust and easily consume from data products - that we create data products that don't care what use case type - regular analytics or AI/ML - can we remove a lot of the complexity they face? Do they need feature stores for data they aren't transforming? If they can get continued access and know the quality, why create a separate process that has fragility instead of trust the data product owners upstream?

I wasn’t smart enough in the moment to talk about do we need to have a copy of the training data itself for reproducibility but folks smarter on ML than I am can answer that one, probably in the affirmative. But overall, there is a lot of complexity in the way we do AI/ML because data scientists can't trust the sources of their data and they feel the need to take control because if they don't, their models break. So we need to earn their trust and show them a better way. But again, we aren't there yet, so let's work to make this a reality in the future.

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Data Mesh Radio episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here and their great data mesh resource center here. You can download their Data Mesh for Dummies e-book (info gated) here.

Brent's website and book: https://www.effectivedatastorytelling.com/

Brent's LinkedIn: https://www.linkedin.com/in/brentdykes/

Brent's Data Analytics Marathon Forbes article: https://www.forbes.com/sites/brentdykes/2022/01/12/data-analytics-marathon-why-your-organization-must-focus-on-the-finish/?sh=2af698743c3b

In this episode, Scott interviewed Brent Dykes, Chief of Data Storytelling at his own firm, AnalyticsHero. Scott asked Brent to be on after João Sousa pointed him to Brent's content.

Some key takeaways/thoughts from Brent's point of view:

1. Focus on the so-what, what should people take away and do from the insights, not the sausage making of the insights. Execs want to eat the dang cake, not hear about how you made it!
2. !Controversial!: You want to get to a place where you remain more neutral until the data informs your view. It can cause more cognitive load to update our views instead of waiting for the data to speak first.
3. Many organizations lose steam in actually driving action on analytics, they don't drive change with data - they fail at least one of the following: generating actual insights, communicating their insights well enough to drive action, and/or actually acting on the insights.
4. The analytics marathon: data collection -> data processing -> data visualization/reporting -> data analysis -> insight communication -> take action.
5. Many...

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here and their great data mesh resource center here. You can download their Data Mesh for Dummies e-book (info gated) here.

Ananth's LinkedIn: https://www.linkedin.com/in/ananthdurai/

Schemata: https://schemata.app/

Data Engineering Weekly newsletter: https://www.dataengineeringweekly.com/

In this episode, Scott interviewed Ananth Packkildurai, Author of Data Engineering Weekly and the creator of Schemata.

Scott note: we discuss Schemata quite a bit in this episode but it's an open source offering that I think can fill in some of the major gaps in our tooling and even ways of working collaboratively around data.

Some key takeaways/thoughts from Ananth's point of view:

1. !Important!: Collaboration around data is crucial. The best way to get people bought in on collaboration around data is to integrate into their workflow, not to create yet another one-off tool in yet another pane of glass.
2. ?Controversial?: There is so much friction between initial data producers - the domain developers - and data consumers because they are constantly speaking past each other. The data consumers have to learn too much about the domain and the data producers rarely really understand the context of most analytical asks.
3. Data creation is a human-in-the-loop problem. Autonomous data creation is not likely to create significant value because the systems can't understand the context well enough right now.
4. As Zhamak has also pointed out, there is far too much tool fragmentation. It made sense with lots of readily available VC money and finding how to approach things with cloud but we need holistic approaches, not spot approaches to things...

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

All about why we need more explorers in our data mesh implementations and that it's too early for experts :)

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here and their great data mesh resource center here. You can download their Data Mesh for Dummies e-book (info gated) here.

Andrew's LinkedIn: https://www.linkedin.com/in/andrewpease123/

In this episode, Scott interviewed Andrew Pease, Field CTO of North Europe at Salesforce. To be clear, he was only representing his own views on the episode.

Some key takeaways/thoughts from Andrew's point of view (mostly written by him):

1. Sensitizing people to data and improving their data fluency can be a challenge. Lots of people have had some less than perfect past experiences - perhaps a dry, abstract class has given them "statistics trauma". It's important to make it digestible for them to get started.
2. Organizations typically evolve into silos so IT systems/approaches often evolve into silos too - Conway's Law. The bigger those organizations and silos are, the harder they are to bridge / the deeper the divides.
3. Much as we'd like one, there is not a single silver bullet architecture for all organizations to overcome these silos.
4. Without relevant IT architectures and processes, it can be challenging to put relevant and timely data and actionable insights into the business people's workflows. You won't get it "perfect" the first time, but get started and learn to improve through experience.
5. You should reiterate to people that data is there to augment their role, not to replace it. It's there to help them be more efficient and successful in their work. That's a key part of data fluency, not just understanding how to use data but where data can help.
6. Feedback loops are very important to increase data quality levels and data value. It's important to build in these loops to make end-users feel like they are a part of a constant and never-ending improvement exercise. It shouldn't be a big burden but...