Data Mesh Radio

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here (info gated)

Gøran's Twitter: @gorzan / https://twitter.com/gorzan

Audun's Twitter: @audunstrand / https://twitter.com/audunstrand

Gøran's LinkedIn: https://www.linkedin.com/in/g%C3%B8ran-berntsen-66066517/

Audun's LinkedIn: https://www.linkedin.com/in/audunstrand/

NAIS Platform Website: https://nais.io/

In this episode, Scott interviews Audun Fauchald Strand and Gøran Berntsen of NAV. Audun is the Principal Engineer and Gøran is the Product Manager for NAV's NAIS application platform as well as their emerging self-serve data platform for data mesh called NADA.

They covered a lot of different topics including: 1) building out the platform; 2) working with consumers to set expectations for common data products; 3) definition of a data product - and how it will evolve; 4) setting the frameworks for producer teams and allowing them to own the production; 5) communicating across teams; and 6) Cake! No really, a secret to success is cake.

While NAV is early days in building out their data platform for data mesh, they are taking an interesting approach: work with the developers to set data product expectations and then see how the developers would go about creating those data products. Then, the data platform team will build the platform out to make developer workflows much easier. While Gøran, with a background as a data person, feels the pull to make the self-serve platform as data-centric as possible, he understands the need to make it developer friendly from his time building the application platform with those from a developer background like Audun.

They both talked about reducing friction, including via sensible defaults, as a big part of their path forward. Stop trying to make developers come up with everything themselves. While they are still early days on developing those defaults, they are comfortable in their process to get there. And working with developers along the way is key.

To start, NAV's definition of a data product is a single table or view. It...

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here (info gated)

Semantic Arts website: https://www.semanticarts.com/

Dave's book: https://www.semanticarts.com/software-wasteland/

Contact Dave: https://www.semanticarts.com/contact-us/

Dave's LinkedIn: https://www.linkedin.com/in/davemccomb/

In this episode, Scott interviewed Dave McComb, the President and Co-Founder of Semantic Arts. Scott asked Dave on as part of the continuing deep dive into Domain Driven Design for Data and Data-Centric Application Development as Dave wrote the book on Data-Centric Application Development - literally.

Dave's overall argument is that most businesses really have very few "business events", ~500-2000 for even the largest companies. Those large enterprises may have 10K+ applications, each with their own data model and application model, leading to possibly 100M+ data attributes. All that leads to far more complexity than is necessary if companies just focused on building applications from the business events side.

They discussed the amount of work an application developer would need to learn to be able to do data-centric application development; while it is mostly about learning data modeling, especially for graph databases, Dave has seen the application developers really not want to move to this model. This has meant a slower roll-out at a number of clients than if they were embracing it.

Scott asked about the user experience (UX) in data-centric application development, both for the data producer and data consumer. Per Dave, the UX is pretty lacking, especially on the data producer side so there seems to be a need for better developer tooling for graph databases. Despite the "crude" UX, Dave says he sees data consumers really loving consuming data from a graph.

The overall goal of data-centric application development is to provide simplicity and flexibility to organizations as most applications are too rigid for Dave and the system integration is even worse.

As mentioned, the first 3 people who fill out a Contact Us on the Semantic Arts website and mention Data...

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Scott shares his emerging data mesh anti-patterns.

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here (info gated)

José's LinkedIn: https://www.linkedin.com/in/jecabeda/

José's Twitter: @jecabeda / https://twitter.com/jecabeda

In this episode, Scott interviewed José Cabeda, Data Engineer at Call-center-as-a-service provider Talkdesk. They talked about Talkdesk's start to their data mesh journey and progress so far.

When José came across Zhamak's original post, it spoke to a number of the challenges Talkdesk was facing, checking many of the boxes to where they wanted to head. The team started from a single data product and iterated from there. While they are still relatively early in their journey, like every company, they have advanced far past their initial use case.

At Talkdesk, a data product is typically a single table or view in Snowflake but the company's North Star is event streaming as their key information storage and sharing mechanism. However, it was sometimes difficult to train people to understand the difference between a business event - something that occurred in the real world - and an event streaming event.

José had a few key takeaways and recommendations for those implementing data mesh:

1. Change will be constant in a data mesh implementation so it is best to standardize the way people and systems will interact as much as possible. Define expectations!

2. Be open to new ideas, there are many challenges ahead so it's best to face them together.

3. Use a single universal ID for major concepts like account or business events to make interoperability easier / possible.

4. Don't be afraid to slice your data in different ways to serve different use cases.

5. To drive buy-in, start with a single use case, whether that is a data product or multiple data products - most people recommend 2-3 data products in your PoC - so you can show why data mesh is a good idea.

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

If you want to be a...

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here (info gated)

Twitter: @thinh_ha / https://twitter.com/thinh_ha

LinkedIn: https://www.linkedin.com/in/%E2%98%81%EF%B8%8F-thinh-ha-58945969/

Medium post: https://medium.com/google-cloud/10-reasons-why-you-should-not-adopt-data-mesh-7a0b045ea40f

In this episode, Scott interviewed Thinh Ha, Strategic Cloud Engineer at Google Cloud Professional Services. To be clear, Thinh was only representing his own views and was not representing Google/GCP in any way.

Scott had asked Thinh to be on after Thinh wrote a post on Medium called 10 Reasons Why You Should Not Adopt Data Mesh (later changed to 10 Reasons Why You Are Not Ready to Adopt Data Mesh). While Thinh is a self-professed "believer" in data mesh, he brings up a number of very reasonable checklist/self-check reasons you wouldn't be ready to move towards data mesh yet.

Scott and Thinh go down each of the 10 objections/reasons through the episode and it is advisable to read the article before proceeding. You can see the high level reasons below. There are a lot of very valuable insights into each of the reasons that could make this a 5 page summary so just listen to the episode instead ;)

1. You are not operating at a scale where decentralization makes sense

2. You do not have a strong business-case for how adopting Data Mesh will deliver business value for individual business units

3. You treat Data Mesh as a technical solution with a fixed target rather than an operating model that continuously evolves over time

4. Your organizational culture does not empower bottom-up decision-making

5. You do not have clearly established roles & responsibilities and incentive structure for distributed data teams

6. You do not have a critical mass of data talent

7. Your data teams have low engineering maturity

8. You expect to find off-the-shelf software to help you adopt Data Mesh

9. You do not have buy-in to “shift-left” security, privacy, and compliance

10. You do not consider Data Governance to be a core activity to be prioritized against other activities in every data team’s backlog

Data Mesh Radio is hosted by Scott Hirleman. If you want to

4 quick things:

1. 4 episodes this week; interviews with Thinh Ha (re 10 Reasons You Aren't Ready for Data Mesh article), José Cabeda (Talkdesk's User Journey), and Dave McComb (more on Data-centric Application Design) and one mesh musing on anti-patterns.
2. Still having issues getting transcripts done for episodes. If you are at a vendor that wants to sponsor transcripts, please let me know.
3. Patreon to launch soon, probably Friday.
4. Starting work on the getting started / proof of concept guide very soon.

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here (info gated)

Azmath's LinkedIn: https://www.linkedin.com/in/azmathpasha/

In this episode, Scott interviews Azmath Pasha, member of the Forbes Technology Council, who has 25+ years in implementing large-scale IT projects including at CapGemini and Paradigm Technology.

Azmath gave his 3 key measures for data value: cost savings, business value (e.g. driving new initiatives), and data reuse. For data mesh, the long-term value is in the second two but for Azmath, a PoC could be better served focusing on cost savings as it is easier to track and faster to realize.

They dove into the concept of data discovery with human interaction, not purely an online experience. Similar to event storming for discovering your domain events (see DDD for Data episodes), discovery as a purely tool-based experience is always likely to be somewhat lacking. Scott was intrigued about this as that aspect of data discovery hasn't been widely discussed.

To Azmath, the data product experience, part of what Zhamak calls 'the experience plane', is crucial. It is much harder to drive buy-in if your product is hard to use / has a bad user experience.

Azmath's other crucial aspects to getting a data mesh (or any large scale data project) implementation right included: staying tool agnostic so you can remain "future proof"; supporting data producers to reduce time to delivery, especially initial delivery; and looking at your architecture and tool investments over a 5 year time horizon, not just for the short to medium-term.

Azmath wrapped up by saying we are entering a new era of using data, we must democratize the data and also look to new metrics for evaluating the business value of data.

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Application Strangler pattern (recently renamed Strangler Fig Application pattern): https://martinfowler.com/bliki/StranglerFigApplication.html

CQRS: https://www.martinfowler.com/bliki/CQRS.html

Kurt's LinkedIn: https://www.linkedin.com/in/kugardiner/

nib Group careers page: https://nib.wd3.myworkdayjobs.com/careers

In this episode, Scott interviews Kurt Gardiner, Engineering Manager of Data Engineering at Australian Insurance company nib Group.

Kurt shared some insights into nib's journey so far, including the search for something like data mesh before Zhamak published, tool choices (Snowflake, dbt, Fivetran, EventBridge, Kinesis), the slow-role approach to replacing legacy implementation (the "application strangler" pattern mentioned), how they got started, and much more.

Much of nib's approach is the small-scale tactical while building incrementally for the bigger strategic focus. E.g. helping teams to design their data products somewhat manually while building the reusable tooling to be far less manual going forward.

Along their journey, there was some internal pushback from data consumers, especially those used to consuming from the data warehouse. To do data mesh right, Kurt and Scott both emphasized the need to set things up so they can evolve. That will frustrate or scare some people and it's important to work with them to see why that matters. There also needs to be a high tolerance for failure - you will NOT get everything right on your first go.

Kurt also waxed poetic (said nice things) about event streaming patterns, especially CQRS - see link below for more info -, for a useful and scalable pattern that is good for both application development and creating a scalable and useful domain data model. But it requires a complete redesign so it is probably something to slowly introduce where it makes sense, if at all.

Some pithy nuggets of wisdom from Kurt that are highly applicable to data mesh:

"The single biggest problem in communication is the illusion that it has taken place"

"Nobody cares what you know until they know that you care"

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here:

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

PredictiveUX website: https://www.predictiveux.com/

PredictiveUX partnered meetup: https://www.meetup.com/hexagon-ux-dc-chapter/

Karen LinkedIn: https://www.linkedin.com/in/karenpassmore/

Karen Email: karen at predictiveux.com

Steve LinkedIn: https://www.linkedin.com/in/stephenstesney/

Steve Email: sstesney at predictiveux.com

In this episode, Scott interviewed Karen Passmore (CEO) and Steve Stesney (Data Product Lead) at consulting firm PredictiveUX. They touched on a lot of different topics - a key theme throughout is the importance of the user experience in data mesh, for both data product producers and consumers.

Karen highlighted some parallels between data mesh and content management projects and how to take some key learnings from the past and apply them to data mesh implementations. They discussed the importance of providing your internal people with the right content at their current point in their learning journey - a successful implementation of data mesh requires making it far easier and more scalable to share incremental work artifacts and knowledge - it also means your crucial company knowledge actually gets documented properly.

Steve talked about some historic challenges he had personally with decentralized teams - if you don't manage the cross domain collaboration, both at the business and the technical implementation levels, it is a major pain to stich your data together from all those sources. So there needs to be good alignment on interoperability. Basically, data mesh without a good interoperability strategy is just high quality data silos.

Karen and Steve both emphasized the importance of UX (user experience) for driving adoption. You can have the best solution in the world but if the users don't want it, it's not going to be successful. So working with them throughout the process is crucial to get to a successful implementation, whether that is data mesh or not.

Karen wrapped up by emphasizing the need to be patient and to not expect the same results or try to copy the exact path of other organizations implementing a data mesh. Every organization is very unique and you need to figure out what might work for your organization. Take learnings, not exact blueprints.

The last key point to extract is the need for multiple communication methods, especially for data requests. There may be some overlap but it's a great way to ensure reliability and scalability of your business processes.

Data Mesh Radio is hosted by Scott...