Experiencing Data w/ Brian T. O’Neill

Today I’m chatting with Osian Jones, Head of Product for the Data Platform at Stuart. Osian describes how impact and ROI can be difficult metrics to measure in a data platform, and how the team at Stuart has sought to answer this challenge. He also reveals how user experience is intrinsically linked to adoption and the technical problems that data platforms seek to solve. Throughout our conversation, Osian shares a holistic overview of what it was like to design a data platform from scratch, the lessons he’s learned along the way, and the advice he’d give to other data product managers taking on similar projects. 

Highlights/ Skip to:

• Osian describes his role at Stuart (01:36)

• “Analytics has been treated very much as a technical problem, and very much so on the data platform side, which is more on the infrastructure and the tooling to enable analytics to take place. And so, viewing that purely as a technical problem left us at odds in a way, compared to [teams that had] a product leader, where the user was the focus [and] the user experience was very much driving a lot of what was roadmap.” — Osian Jones (03:15)
  

• Stuart: https://stuart.com/

Today I’m chatting with Josh Noble, Principal User Researcher at TruEra. TruEra is working to improve AI quality by developing products that help data scientists and machine learning engineers improve their AI/ML models by combatting things like bias and improving explainability. Throughout our conversation, Josh—who also used to work as a Design Lead at IDEO.org—explains the unique challenges and importance of doing design and user research, even for technical users such as data scientists. He also shares tangible insights on what informs his product design strategy, the importance of measuring product success accurately, and the importance of understanding the current state of a solution when trying to improve it.

Highlights/ Skip to:

• Josh introduces himself and explains why it’s important to do design and user research work for technical tools used by data scientists (00:43)

• “We want to make machine learning better by testing it, helping people analyze it, helping people monitor models. Bias and fairness is an important part of that, as is accuracy, as is explainability, and as is more broadly AI quality.” — Josh Noble (05:13)
  

• Truera: https://truera.com/

Today I’m chatting with Cole Swain, VP of Product at Tomorrow.io. Tomorrow.io is an untraditional weather company that creates data products to deliver relevant business insights to their customers. Together, Cole and I explore the challenges and opportunities that come with building an untraditional data product. Cole describes some of the practical strategies he’s developed for collecting and implementing qualitative data from customers, as well as why he feels rapport-building with users is a critical skill for product managers. Cole also reveals how scientists are part of the fold when developing products at Tomorrow.io, and the impact that their product has on decision-making across multiple industries. 

Highlights/ Skip to:

• Cole describes what Tomorrow.io does (00:56)

• “[Customers aren't] just going to listen to that objective summary and go do the action. It really has to be supplied with a tremendous amount of information around it in a concise way. ... The assumption upfront was just, if we give you a recommendation, you’ll be able to go ahead and go do that. But it’s just not the case.” – Cole Swain (13:40)
  

• “Our sales team is just always finding new use cases. And we have to continue to say no and we have to continue to be disciplined in this arena. But I’d be lying to tell you if that didn’t keep me up at night when I hear about this opportunity of this solution we could build, and I know it can be done in a matter of X amount of time. But the risk of doing that is just too high, sometimes.” – Cole Swain (35:42)

• Company website: https://Tomorrow.io

Today I’m chatting with Samir Sharma, CEO of datazuum. Samir is passionate about developing data strategies that drive business outcomes, and shares valuable insights into how problem framing and research can be done effectively from both the data and business side. Samir also provides his definition of a data strategy, and why it can be complicated to uncover whose job it is to create one. Throughout the conversation, Samir and I uncover the value of including different perspectives when implementing a data strategy and discuss solutions to various communication barriers. Of course, dashboards and data products also popped up in this episode as well! 

Highlights/ Skip to:

• How Samir defines a data strategy and whose job it is to create one (01:39)

• “I don’t think business teams really care how you do it. If you can get an outcome—even if it’s quick and dirty. We’re not supposed to be doing these things for months on end. We’re supposed to be iterating quickly to start to show that result and add value and then building on top of that to show more value, more results.” — Samir Sharma (07:29)
  

• “I will continuously talk about processes because there’s not enough done actually understanding processes and how data is an event that occurs when a process is kicked off. … If you don’t understand the process and how data is enabling that process, or how data is being generated and the trigger points, then you’re just building something without really understanding where I need to fit that product in or where I need to fit that workflow in.” – Samir Sharma (11:46)

• “But I start with asking clear questions about if I built you this dashboard, what is the decision you’re going to make off the back of it? Nine times out of ten, that question isn’t asked, if I build you this widget on this dashboard, what decision or action are you going to make or take? And how is that going to be linked back to the map that strategic objective? And if you can ask that question, you can build with purpose.” – Samir Sharma (19:27)

• “You show [users] a bit of value, you show them what they’ve been dying to have, you give them a little bit extra in that so they can really optimize their decisions, and suddenly, you’ve got both sides now speaking a language that is really based on business outcomes and results.” – Samir Sharma (32:38)

•  “If the people in that conversation are the developers on one side, the business team, and they’re starting to see a new narrative, even the developers will start to say, “Oh! Now, I know exactly why I’m doing this. Now, I know why I’m building it.” So, they’re also starting to learn about the business, about what impacts sales, and maybe how marketing then intertwines into that. It’s important that that is done, but not enough time has been taken on that approach.” – Samir Sharma (24:05)
  

• datazuum: https://datazuum.com/

Today I’m chatting with Yuval Gonczarowski, Founder & CEO of the startup, Akooda. Yuval is a self-described “socially capable nerd” who has learned how to understand and meet the needs of his customers outside of a purely data-driven lens. Yuval describes how Akooda is able to solve a universal data challenge for leaders who don’t have complete visibility into how their teams are working, and also explains why it’s important that Akooda provide those data insights without bias. Yuval and I also explore why it’s so challenging to find great product leaders and his rule for getting useful feedback from customers and stakeholders. 

Highlights/ Skip to:

• Yuval describes what Akooda does (00:35)

• “Re: moving from a technical to product role: My first inclination would be straight up talk about the how, but that’s not necessarily my job anymore. We want to talk about the why and how does the customer perceive things, how do they look at things, how would they experience this new feature? And in a sense, [that’s] my biggest change in the way I see the world.” — Yuval Gonczarowski (03:01)
  

• “We are a very data-driven organization. Part of it is our DNA, my own background. When you first start a company and you’re into your first handful of customers, a lot of decisions have to be made based on gut feelings, sort of hypotheses, scenarios… I’ve lived through this pain.” — Yuval Gonczarowski (09:43)

• “I don’t believe I will get honest feedback from a customer if I don’t hurt their pocket. If you want honest feedback [from customers], you got to charge.” — Yuval Gonczarowski (11:38)
  

• When selling a data product: “If you don’t build the right collateral and the right approach and mindset to the fact that it’s not enough when the contract is signed, it’s actually these three sales cycles of making sure that customer adoption is done properly, then you haven’t finished selling. Contract is step one, installation is step two, usage is step three. Until step three is done, haven’t really sold the product.” — Yuval Gonczarowski (16:59)

• “By definition, all products are too complex. And it’s always tempting to add another button, another feature, another toggle. Let’s see what we can remove to make it easier.” – Yuval Gonczarowski (26:35)

• Akooda: https://akooda.co/

Today I’m chatting with Dr. Sebastian Klapdor, Chief Data Officer for Vista. Sebastian has developed and grown a successful Data Product Management team at Vista, and it all began with selling his vision to the rest of the executive leadership. In this episode, Sebastian explains what that process was like and what he learned. Sebastian shares valuable insights on how he implemented a data product orientation at Vista, what makes a good data product manager, and why technology usage isn’t the only metric that matters when measuring success. He also shares what he would do differently if he had to do it all over again.

Highlights/ Skip to:

• How Sebastian defines a data product (01:48)

• “You need to establish a culture, and that’s often the hardest part that takes the longest -  to treat data as an asset, and not to treat it as a byproduct, but to treat it as a product and treat it as a valuable thing.” – Sebastian Klapdor (07:56)
  

• “We went out there and we were hiring people in the market who were experienced [Product Managers]. But we also see internal people, actually grooming and growing into all of these roles, both from these 80 folks who have been around before, but also from other areas of Vista.” – Sebastian Klapdor (20:28)

• “[Being a good Product Manager] comes back to the good old classics of collaborating, of being empathetic to where other people are at, their priorities, and understanding where [our] priorities fit into their bigger piece, and jointly aligning on what is valuable for Vista.” – Sebastian Klapdor (22:27)

• “I think there’s nothing more detrimental than saying, ‘Yeah, sure, we can deliver things, and with data, it can do everything.’ And then you disappoint people and you don’t stick to your promises. … If you don’t stick to your promise, it will hurt you.” – Sebastian Klapdor (23:04)
  

• “That was the biggest learning … whom do we staff as data product managers? And what do we expect of a good data product manager? How does a career path look like? That took us a really long time to figure out.” – Sebastian Klapdor (30:18)
  

• Vista: https://vista.io

Today I’m chatting with Bob Mason, Managing Partner at Argon Ventures. Bob is a VC who seeks out early-stage founders in the ML/AI space and helps them inform their go-to-market, product, and design strategies. In this episode, Bob reveals what he looks for in early-stage data and intelligence startups who are trying to leverage ML/AI. He goes on to explain why it’s important to identify what your strengths are and what you enjoy doing so you can surround yourself with the right team. Bob also shares valuable insight into how to earn trust with potential customers as an early-stage startup, how design impacts a product’s success, and his strategy for differentiating yourself and creating a valuable product outside of the ubiquitous “platform play.” 

Highlights/ Skip to:

• Bob explains why and how Argon Ventures focuses their investments in intelligent industry companies (00:53)

• “In a former life, I was a software engineer, founder, and CTO myself, so I have to watch myself to not just geek out on the technology itself because the most important element when you’re determining if you want to move forward with investment or not, is this: is there a real problem here to be solved or is this technology in search of a problem?” — Bob Mason (01:51)
  

• “I don’t think the technical folks in an early-stage startup absolve themselves of not being really intimately involved with their go-to-market and who they’re ultimately creating value for.” – Bob Mason (07:07)

• “When we’re making an investment decision, startups don’t generally have any customers, and so we don’t necessarily use the signal of long-term customer adoption as a driver for our initial investment decision. But it’s very much top of mind after investment and as we’re trying to build and bring the first version of the product to market. Being very thoughtful and mindful of sort of customer experience and long-term adoption is absolutely critical.” – Bob Mason (11:23)

• “If you’re a scientist, the way you’re presenting both raw data and sort of summaries of data could be quite different than if you’re working with a business analyst that’s a few years out of college with a liberal arts degree. How you interpret results and then present those results, I think, is actually a very interesting design problem.” – Bob Mason (18:40)

• “I think initially, a lot of early AI startups just kind of assumed that customers would be patient and let the system run, [waiting] 3, 6, 9, 12 months [to get this] magical ROI, and that’s just not how people (buyers) operate.” – Bob Mason (21:00)

• “Re: platform plays: Obviously, you could still create a tremendous platform that’s very broad, but we think if you focus on the business problem of that particular vertical or domain, that actually creates a really powerful wedge so you can increase your value proposition. You could always increase the breadth of a platform over time. But if you’re not solving that intrinsic problem at the very beginning, you may never get the chance to survive.” – Bob Mason (28:24)

• Argon Ventures: https://argon.vc/

Today I’m chatting with Bruno Aziza, Head of Data & Analytics at Google Cloud. Bruno leads a team of outbound product managers in charge of BigQuery, Dataproc, Dataflow and Looker and we dive deep on what Bruno looks for in terms of skills for these leaders. Bruno describes the three patterns of operational alignment he’s observed in data product management, as well as why he feels ownership and customer obsession are two of the most important qualities a good product manager can have. Bruno and I also dive into how to effectively abstract the core problem you’re solving, as well as how to determine whether a problem might be solved in a better way. 

Highlights / Skip to:

• Bruno introduces himself and explains how he created his “CarCast” podcast (00:45)

• “What’s happening in the industry is really interesting. For people that are running data teams today and listening to us, the makeup of their teams is starting to look more like what we do [in] product management.” — Bruno Aziza (04:29)
  

• “As a product manager, yes, we’re in the business of software, but in fact, I think you’re in the career management business. Your job is to make sure that whatever your customer’s job is that you’re making it so much easier that they, in fact, get so much more done, and by doing so they will get promoted, get the next job.” – Bruno Aziza (15:41)

• “I think that is the task of any technology company, of any product manager that’s helping these technology companies: don’t be building a product that’s looking for a problem. Just start with the problem back and solution from that. Just make sure you understand the problem very well.” (19:52)

• “If you’re a data product manager today, you look at your data estate and you ask yourself, ‘What am I building to save money? When am I building to make money?’ If you can do both, that’s absolutely awesome. And so, the data product is an asset that has been built repeatedly by a team and generates value out of data.” – Bruno Aziza (23:12)

• “[Machine learning is] hard because multiple teams have to work together, right? You got your business analyst over here, you’ve got your data scientists over there, they’re not even the same team. And so, sometimes you’re struggling with just the human aspect of it.” (30:30)

• “As a data leader, an IT leader, you got to think about those soft ways to accomplish the stuff that’s binary, that’s the hard [stuff], right? I always joke, the hard stuff is the soft stuff for people like us because we think about data, we think about logic, we think, ‘Okay if it makes sense, it will be implemented.’ For most of us, getting stuff done is through people. And people are emotional, how can you express the feeling of achieving that goal in emotional value?” – Bruno Aziza (37:36)

• As referenced by Bruno, “Good Product Manager/Bad Product Manager”: https://a16z.com/2012/06/15/good-product-managerbad-product-manager/

Today I’m chatting with returning guest Tom Davenport, who is a Distinguished Professor at Babson College, a Visiting Professor at Oxford, a Research Fellow at MIT, and a Senior Advisor to Deloitte’s AI practice. He is also the author of three new books (!) on AI and in this episode, we’re discussing the role of product orientation in enterprise data science teams, the skills required, what he’s seeing in the wild in terms of teams adopting this approach, and the value it can create. Back in episode 26, Tom was a guest on my show and he gave the data science/analytics industry an approximate “2 out of 10” rating in terms of its ability to generate value with data. So, naturally, I asked him for an update on that rating, and he kindly obliged. How are you all doing? Listen in to find out!

Highlights / Skip to:

• Tom provides an updated rating (between 1-10) as to how well he thinks data science and analytics teams are doing these days at creating economic value (00:44)

• “Data science in general, I think has been focused heavily on motivation to fit lines and curves to data points, and that particular motivation certainly isn’t enough in that even if you create a good model that fits the data, it doesn’t mean at all that is going to produce any economic value.” – Tom Davenport  (03:05)
  

• “[PM is] a hard thing, even for people in non-technical roles, because product management has always been a sort of ‘minister without portfolio’ sort of job, and you know, influence without formal authority, where you are responsible for a lot of things happening, but the people don’t report to you, generally.” – Tom Davenport (22:03)

• “This collaboration between a human being making a decision and an AI system that might in some cases come up with a different decision but can’t explain itself, that’s a really tough thing to do [well].” – Tom Davenport (28:04)
  

• “I think it’s unlikely that [organizational gaps] are going to be successfully addressed by merging everybody together in one organization. I think that’s what product managers do is they try to address those gaps in the organization and develop a process that makes coordination at least possible, if not true, all the time.” – Tom Davenport (36:49)

• Tom’s LinkedIn: https://www.linkedin.com/in/davenporttom/

Today I’m chatting with former-analyst-turned-design-educator Jeremy Utley of the Stanford d.school and co-author of Ideaflow. Jeremy reveals the psychology behind great innovation, and the importance of creating psychological safety for a team to generate what they may view as bad ideas. Jeremy speaks to the critical collision of unrelated frames of reference when problem-solving, as well as why creativity is actually more of a numbers game than awaiting that singular stroke of genius. Listen as Jeremy gives real-world examples of how to practice and measure (!) your innovation efforts and apply them to data products.

Highlights/ Skip to:

• Jeremy explains the methodology of thinking he’s adopted after moving from highly analytical roles to the role he’s in now (01:38)

• “I’m in spreadsheets every day to this day, but I recognize that there’s a time and place when that’s the tool that’s needed, and then specifically, there’s a time and a place where that’s not going to help me and the answer is not going to be found in the spreadsheet.” – Jeremy Utley (03:13)
  

• “I think a vendetta that I have is against this wrong placement of—this exaltation of efficiency is the end-all, be-all. Innovation is not efficient. And the question is not how can I be efficient. It’s what is effective. And effectiveness, oftentimes when it comes to innovation and breaking through, doesn’t feel efficient.” – Jeremy Utley (09:17)
  

• “The way the brain works, we actually understand it. The way breakthroughs work we actually understand them. The difficulty is it challenges our definitions of efficiency and professionalism and all of these things.” – Jeremy Utley (15:13)

• “What’s the a priori probability that any solution is the right solution? Or any idea is a good idea? It’s exceptionally low. You have to be exceptionally arrogant to think that most of your ideas are good. They’re not. That’s fine, we don’t mind because then what’s efficient is actually to generate a lot.” – Jeremy Utley (26:20)

• “If you don’t learn that nothing happens when the ball hits the floor, you can never learn how to juggle. And to me, it’s a really good metaphor. The teams that don’t learn nothing happens when they have a bad idea. Literally, the world does not end. They don’t get fired. They don’t get ridiculed. Now, if they do get fired or ridiculed, that’s a leadership problem.” – Jeremy Utley (35:59)

• [The following] is an essential question for a team leader to ask. Do people on my team have the freedom, at least with me, to share what they truly fear could be an incredibly stupid idea?” – Jeremy Utley (41:52)

• Ideaflow: https://www.amazon.com/Ideaflow-Only-Business-Metric-Matters-ebook/dp/B09R6M3292