Data Mesh Radio

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Mohammad's Latest Data Governance article: https://www.linkedin.com/pulse/some-thoughts-data-governance-mohammad-syed/

Book mentioned: Disrupting Data Governance: A Call to Action by Laura Madsen

In this episode, Scott interviewed Mohammad Syed, Lead Strategist - Data at Caruthers and Jackson about how data mesh governance has to be different from what we've done historically.

Per Mohammad, data governance in data mesh is very different to doing governance for either a data lake or a data warehouse. The warehouse has a focus on high-level quality and usability but at the expense of context and agility. Data lake is about metadata and lineage but at the severe expense of usability - schema on query is not fun for consumers - and often quality.

For most data organizations, governance has been very macro focused - governing the data warehouse or lake as a whole. That is part of why data governance has become a major bottleneck - the focus is on the macro but the individual requests are the micro.

In data mesh, governance can shift to being about maximizing the value of the data instead of mostly preventing risk. Of course, there is a balance between local maximization - the value of each data product - and global maximization - the value at the overall data mesh level.

A key focus to data mesh data governance is enabling - especially enabling the domains to govern their data products. Mohammad made the point that you need to enable your domains by creating the technical and business definitions of a "good" data product. Then the governance team needs to teach teams about the quality definitions, e.g. data product consumability. There is a need for policies of course but mostly focus on frameworks to enable policy creation and enforcement - decentralize!

A key point Mohammad made was: governance only works with informed governors - you must teach domains to govern properly. Transparency is key to make data governance work.

Mohammad emphasized the "good" data product definition leads to the separation of data quality and data product quality. A data product might be more valuable for other reasons - or less costly - by having relaxed data quality standards. In a data warehouse implementation, there is really only a single definition of "good" quality, but that just won't work in data mesh. We really need to develop better frameworks for what data quality means at the micro level.

To get data governance right, strategy and maturity are crucial - what are you actually trying to accomplish? Data mesh for the sake of data mesh is...

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Khanh's LinkedIn: https://www.linkedin.com/in/khanhnchau/

In this episode, Scott interviewed Khanh Chau, Lead Architect for the Data Mesh Initiative at Northern Trust.

Khanh believes you have to be passionate about making data better to do a good job implementing data mesh. And it is DEFINITELY a journey so you need patience and vision. Also, each journey is unique, you can't just copy/paste from another organization. You need to make failure okay - but you should look to make it easy to fail fast, measure, and adjust.

Khanh talked about the need for exec buy-in before heading down the data mesh path. They got that exec buy-in by proving that the total cost of ownership of data was quite high as the consumers had to do a LOT of work to get the data to usable.

When speaking internally, the business people were very excited to participate if it meant they could get quality data. Some of the IT/data engineering folks were harder to convince. It was especially hard to get them to shed layers of not-useful technology.

Some IT teams were easier to convince - they had felt the impact of a few too many middle-of-the-night data downtime incidents. Other teams hadn't felt that pain so there were harder to win over. There was also the incentive of additional possibilities - data mesh meant they could do things they couldn't do before.

Khanh talked about making the platform the easy and right path for 80% of use cases. They focused on making things easy to configure; basically: what transformations do you want to do and then it automatically provisions the pipelines. Their goal was to make it easy to make good progress quickly; their time to initial deploy went from 2-3 months per data service to 2-3 weeks per data product and they hope to drive it down further.

Northern Trust has been moving forward with data mesh for about 7 months as part of their high-level digital transformation initiative. On the data side, they had previously focused on data virtualization and data federation but it was not delivering the results they wanted. It was not as scalable as they wanted - it was taking 2-3 months to launch each new data service. They also did not have great information on who was consuming the data and why.

For their data mesh proof of concept, Khanh and team set a timeline of 9 weeks. They needed to prove value by then or data mesh would be a very tough sell internally. Khanh talked about the need to sell data mesh as a paradigm shift in order to get people out of technology-focused thinking.

Northern Trust decided to take a pragmatic approach e.g. not pushing all aspects of data ownership fully left. Khanh and team were focused on finding a "happy balance" on data product SLAs and quality - improvement was necessary but the team preferred done to perfect.

A big...

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Books/posts/papers mentioned:

Blameless PostMortems and a Just Culture by John Allspaw - Link

The Theory of Graceful Extensibility: Basic rules that govern adaptive systems by David D Woods - Link

The Field Guide to Understanding 'Human Error' by Sidney Dekker - Link

In this episode, Scott interviewed Tim Tischler, Principal Engineer at Wayfair. Prior to Wayfair, Tim worked as a Site Reliability Champion at New Relic and is well known in the "human factors" and resilience engineering space.

Per Tim, our current work culture is overly action-item driven - every meeting must have a set of agenda items generated from it. This prevents people from having learning-focused meetings exclusively designed for context sharing. Humans' brains work differently between learning and fixing mode and we ask totally different questions. To be able to scale our knowledge sharing, we need to have the space to have learning-focused meetings.

A good way to center learning-focused meetings, be they "show and tell" or event storming sessions, is via sharing stories - human communication is founded on story sharing through the millennia. Tim's "show and tell" and event storming sessions at Wayfair have had extremely positive reviews so far.

Tim sees ticket-based interactions - just throwing requirements on someone's JIRA backlog or similar - as fundamentally flawed. If Team A gives Team B requirements, Team B just looks to close the ticket versus getting both sides in the room to exchange context and have a negotiation. Tim prefers two modes of interactions over ticket systems: #1 - no human-touch, automated interactions, e.g. an API; and #2 - high touch, high context sharing interactions.

For resilience engineering specifically, you should apply learnings to each data product AND the mesh as a whole. Part of that is a broad acceptance that you are in a highly dynamic and highly changing org - there will be changes! A few anti-patterns to resilience engineering that apply to data mesh are: 1) a hub and spoke relationship model where one person is the key glue - this is bad at a human level and even worse at a technical level :); 2) business leaders pushing for metrics without sharing the specific context as the results end up as completely empty and useless things you are tracking; and 3) not embedding people building platforms into the teams they are building the platform for - they must really understand the workflows.

Data Mesh Radio...

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Ust's LinkedIn: https://www.linkedin.com/in/ust-oldfield/

Ust's Twitter: @UstDoesTech / https://twitter.com/UstDoesTech

In this episode, Scott interviewed Ust Oldfield, Principal Consultant at Advancing Analytics. They covered the concept of self-serve from a consumer standpoint in data mesh, and some ideas around how to get it right.

According to Ust, the overall data and analytics industry is just starting to move from data consumers only consuming what others have prepared towards self-serve data consumption. But, it is important to still provide those prepared reports so 1) people are working from the same info / on the same page and 2) you give people an easy - and maintained - path to important business information.

Ust also mentioned one key to getting self-serve right is to not just enable consumers to get to the data they want, they really need to be able to understand what they are seeing so documentation, sample queries, and other similar tactics are very crucial. Consumers also need training on how to use the platform - in general, training for self-serve data consumption is very lacking across the industry right now.

How do we share information at scale? Forums? "Show and Tell"? Office hours? Neither Ust nor Scott had great answers just yet. Time will tell.

Ust finished with a recommendation for those building out their self-serve platforms for data consumption: spend a lot of time interviewing your data consumers to figure out what will empower them rather than just trying to deliver what you would want. Also, make sure to enable those who just want to consume data as prepared - those who want to be spoon-fed the info, that's fine, allow them to self-select as that is a valid approach to leveraging data.

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on LinkedIn: https://www.linkedin.com/in/scotthirleman/

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman):

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here (info gated)

Jessitron pithy tweet: https://twitter.com/jessitron/status/1471352190149087233

Jessitron Twitter: @jessitron / https://twitter.com/jessitron

Jessitron emails: jessitron @ gmail.com OR jessitron @ honeycomb.io

Jessitron Presentation: Principles of Collaborative Automation: https://jessitron.com/2020/08/03/talk-principles-of-collaborative-automation/

Jessitron blog post: To share the work, share the decisions: https://jessitron.com/2022/02/01/to-share-the-work-share/

Book mentioned: Why Information Grows: The Evolution of Order, from Atoms to Economies by César A. Hidalgo

In this episode, Scott interviewed Jessica Kerr more widely known as Jessitron who is a Principal Developer Advocate at Honeycomb.io. Or as many affectionately refer to her, the Empress of Software. You're probably gonna enjoy this one :)

Scott asked Jessitron on because she had a very pithy tweet about data mesh being "conscious design for unexpected use" and because she knows the developer mindset extremely well and many folks are having trouble working with the developers to get them bought in to sharing their data well.

Jessitron started off by discussing one of the big issues with application development: despite the tooling and process advancements of the last 20 years, it's all somehow only made application development harder. So the starting advice is don't just add more to their plate. It probably won't go well!

Her biggest point was giving application developers agency in how they share their data is key. Autonomy is just passing the responsibility over without the help. Application developers want guidance/direction to the target outcome, but they want to make the choices on how to achieve said target outcome while being given the resources to do so. The information and capability to do their job is key.

For driving buy-in, start with the why, not the ask. Let them know why their data is valuable. And BE SPECIFIC! The

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here (info gated)

Xavier's Twitter: @xgumara / https://twitter.com/xgumara

Xavier's LinkedIn: https://www.linkedin.com/in/xgumara/

Adevinta meetup presentation: https://www.youtube.com/watch?v=av6cT_r4orQ

Xavier's Medium Articles:

https://medium.com/adevinta-tech-blog/building-a-data-mesh-to-support-an-ecosystem-of-data-products-at-adevinta-4c057d06824d

https://medium.com/adevinta-tech-blog/treating-data-as-a-product-at-adevinta-c1dce5d394c5

https://towardsdatascience.com/data-as-a-product-vs-data-products-what-are-the-differences-b43ddbb0f123

Scott interviewed Xavier Gumara Rigol who has been helping lead Adevinta's data mesh implementation as Area Manager for Experimentation and Analytics Enablement. The discussed the data as a product concept and learnings from Adevinta's journey thus far. Xavi has put out some great articles and did a Data Mesh Learning meetup that are linked below.

One key aspect to data as a product is to understand the need for data product evolution, both relative to maturity and to what is consumed. This is a common theme in many data mesh conversations as historically, data consumption has resisted evolution and change. Consumers need to really understand that the business is evolving so what they consume will too. If you manage data products well, it won't be a sudden change but if we are trying to share insights into a domain, those insights will change. When thinking about data product maturity, it's totally okay to start by thinking of a data product as a single table or view.

Xavi also mentioned some pitfalls to forced data product evolution - e.g. getting it wrong as changes can be quite costly to backfill. Adding new attributes is easy but computing something for 3 to 6 months in hindsight can cost a

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Transcript provided by Scott Hirleman here

An episode all about driving buy-in by different personas for data mesh.

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/

All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf

Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/

Please Rate and Review us on your podcast app of choice!

If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here

Episode list and links to all available episode transcripts here.

Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.

Transcript for this episode (link) provided by Starburst. See their Data Mesh Summit recordings here (info gated)

Eric's LinkedIn: https://www.linkedin.com/in/ericbroda/

Eric's Twitter: @ericmbroda / https://twitter.com/ericmbroda

Eric's Medium (multiple posts on data mesh): https://medium.com/@ericbroda

In this episode, Scott interviews Eric Broda, an Executive Consultant in the financial services space. Eric shared his learnings from aiding a large financial services firm to implement data mesh from the infancy of the project.

Eric's big thesis for companies looking to be data-driven is to think of themselves as a platform for connecting supply and demand. The internal company may be the supplier, e.g. if a bank is lending money directly, but more often it is about being the platform - so here to match investors to consumers looking to take out a loan. Lowering the friction between both sides of your constituency is crucial and getting really good at data can help there.

To Eric, the technology is like plumbing - you expect it to work but most businesses at the high level don't care about it as long as it works. You don't buy a house for the plumbing.

Eric's big point of advice is that you shouldn't underestimate the organizational change required to do something like data mesh right. Plan for the change and don't try to skip the necessary change, that will lead to disaster. Speaking of organizational structure, Eric firmly believes that centralization of data ownership fails Conway's Law. While companies can overcome that with a LOT of effort, most don't get there due to fatigue.

When developing a new data product, Eric recommends to first start with expected usage patterns pretty explicitly via a 1:1 relationship model; at least early in a data product's life, the data produced needs to explicitly match the needs of the first target data consumer. This is a departure from data mesh recommended practice but seems to be somewhat of a common emerging pattern, at least in financial services.

Eric also stated his belief that master data management - or MDM - "is dead", especially in data mesh. It hasn't ever really worked and it's not worth trying to do it with data mesh. Time will tell on that one.

Data Mesh Radio is hosted by Scott Hirleman. If you want to connect with Scott, reach out to him on...