Experiencing Data w/ Brian T. O’Neill

Simon Buckingham Shum is Professor of Learning Informatics at Australia’s University of Technology Sydney (UTS) and Director of the Connected Intelligence Centre (CIC)—an innovation center where students and staff can explore education data science applications. Simon holds a Ph.D from the University of York, and is known for bringing a human-centered approach to analytics and development. He also co-founded the Society for Learning Analytics Research (SoLAR), which is committed to advancing learning through ethical, educationally sound data science.

In this episode, Simon and I discuss the state of education technology (edtech), privacy, human-centered design in the context of using AI in higher ed, and the numerous technological advancements that are re-shaping the higher level education landscape.

Our conversation covered:

• How the hype cycle around big data and analytics is starting to pervade education

Designing for Analytics

simon.buckinghamshum.net

Simon on LinkedIn

#experiencingdata

Designing for Analytics Podcast

“We are seeing AI products coming out. Some of them are great, and are making a huge difference for learning STEM type subjects— science, tech, engineering, and medicine. But some of them are not getting the balance right.” — Simon

“The trust break-down will come, and has already come in certain situations, when students feel they’re being tracked…” — Simon, on students perceiving BI solutions as surveillance tools instead of beneficial

“Increasingly, it’s great to see so many people asking critical questions about the biases that you can get in training data, and in algorithms as well. We want to ask questions about whether people are trusting this technology. It’s all very well to talk about big data and AI, etc., but ultimately, no one’s going to use this stuff if they don’t trust it.” — Simon

“I’m always asking what’s the user experience going to be? How are we actually going to put something in front of people that they’re going to understand…” — Simon

“There are lots of success stories, and there are lots of failure stories. And that’s just what you expect when you’ve got edtech companies moving at high speed.” — Simon

“We’re dealing, on the one hand, with poor products that give the whole field a bad name, but on the other hand, there are some really great products out there that are making a tangible difference, and teachers are extremely enthusiastic about.” — Simon

“There’s good evidence now, about the impact that some of these tools can have on learning. Teachers can give some homework out, and the next morning, they can see on their dashboard which questions were the students really struggling with.” — Simon

“The area that we’re getting more and more interested in, and which educators are getting more and more interested in, are the kinds of skills and competencies you need for a very complex future workplace.” — Simon

“We obviously want the students’ voice in the design process. But that has to be balanced with all the other voices are there as well, like the educators’ voice, as well as the technologists, and the interaction designers and so forth.” — Simon on the nuance of UX considerations for students

“…you have to balance satisfying the stakeholder with actually what is needed.” — Brian

“…we’re really at the mercy of behavior. We have to try and infer, from behavior or traces, what’s going on in the mind, of the humans we are studying.” — Simon

“We might say, “Well, if we see a student writing like this, using these kinds of textual features that we can pick up using natural language processing, and they revise their draft writing in response to feedback that we’ve provided automatically, well, that looks like progress. It looks like they’re thinking more critically, or it looks like they’re reflecting more deeply on an experience they’ve had, for example, like a work placement.” — Simon

“They’re in products already, and when they’re used well, they can be effective. But they can also be sort of weapon of mass destruction if you use them badly.” — Simon, on predictive models

Cennydd Bowles is a London-based digital product designer and futurist, with almost two decades of consulting experience working with some of the largest and most influential brands in the world. Cennydd has earned a reputation as a trusted guide, helping companies navigate complex issues related to design, technology, and ethics. He’s also the author of Future Ethics, a book which outlines key ethical principles and methods for constructing a fairer future.

In this episode, Cennydd and I explore the role that ethics plays in design and innovation, and why so many companies today—in Silicon Valley and beyond—are failing to recognize the human element of their technological pursuits. Cennydd offers his unique perspective, along with some practical tips that technologists can use to design with greater mindfulness and consideration for others.

In our chat, we covered topics from Cennydd’s book and expertise including:

• Why there is growing resentment towards the tech industry and the reason all companies and innovators need to pay attention to ethics

Cennydd Bowles

Future Ethics (book)

Design for Real Life 

The Trouble with Bias

Twitter: @cennydd

“There ought to be a clearer relationship between innovation and its social impacts.” — Cennydd

“I wouldn’t be doing this if I didn’t think there was a strong upside to technology, or if I didn’t think it couldn’t advance the species.” — Cennydd

“I think as our power has grown, we have failed to use that power responsibly, and so it’s absolutely fair that we be held to account for those mistakes.” — Cennydd

“I like to assume most creators and data people are trying to do good work. They’re not trying to do ethically wrong things. They just lack the experience or tools and methods to design with intent.” — Brian

“Ethics is about discussion and it’s about decisions; it’s not about abstract theory.” — Cennydd

“I have seen many times diversity act as an ethical early warning system [where] people who firmly believe the solution they’re about to put out into the world is, if not flawless, pretty damn close.” — Cennydd

“The ethical questions around the misapplication or the abuse of data are strong and prominent, and actually have achieved maybe even more recognition than other forms of harm that I talk about.” — Cennydd

“There aren’t a whole lot of ethical issues that are black and white.” — Cennydd

“When you never talk to a customer or user, it’s really easy to make choices that can screw them at the benefit of increasing some KPI or business metric.” — Brian

“I think there’s really talented people in the data space who actually understand bias really well, but when they think about bias, they think they’re thinking more about, ‘how is it going to skew the insight from the data?’ Not the human impact.” — Brian

“I think every business has almost a moral duty to take their consequences seriously.” — Cennydd

Eric Siegel, Ph.D. is founder of the Predictive Analytics World and Deep Learning World conference series, executive editor of “The Predictive Analytics Times,” and author of “Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die.” A former Columbia University professor and host of the Dr. Data Show web series, Siegel is a renowned speaker and educator who has been commissioned for more than 100 keynote addresses across multiple industries. Eric is best known for making the “how” and “why” of predictive analytics (aka machine learning) understandable and captivating to his audiences.

In our chat, we covered:

• The value of defining business outcomes and end user’s needs prior to starting the technical work of predictive modeling, algorithms, or software design.

Machine Learning Week

#experiencingdata

PredictiveAnalyticsWorld.com

ThePredictionBook.com

Dr. Data Show

Twitter: @predictanalytic

“The greatest pitfall that hinders analytics is not to properly plan for its deployment.” — Brian, quoting Eric

“You don’t jump to number crunching. You start [by asking], ‘Hey, how is this thing going to actually improve business?’ “ — Eric

“You can do some preliminary number crunching, but don’t greenlight, trigger, and go ahead with the whole machine learning project until you’ve planned accordingly, and iterated. It’s a collaborative effort to design, target, define scope, and ultimately greenlight and execute on a full-scale machine learning project.” — Eric

“If you’re listening to this interview, it’s your responsibility.” — Eric, commenting on whose job it is to define the business objective of a project.

“Yeah, so in terms of if 10 were the highest potential [score], in the sort of ideal world where it was really being used to its fullest potential, I don’t know, I guess I would give us a score of [listen to find out!]. Is that what Tom [Davenport] gave!?” — Eric, when asked to rate the analytics community on its ability to deliver value with data

“We really need to get past our outputs, and the things that we make, the artifacts and those types of software, whatever it may be, and really try to focus on the downstream outcome, which is sometimes harder to manage, or measure … but ultimately, that’s where the value is created.” — Brian

“Whatever the deployment is, whatever the change from the current champion method, and now this is the challenger method, you don’t have to jump entirely from one to the other. You can incrementally deploy it. So start by saying well, 10 percent of the time we’ll use the new method which is driven by a predictive model, or by a better predictive model, or some kind of change. So in the change in the transition, you sort of do it incrementally, and you mitigate your risk in that way.”— Eric

Greg Nelson is VP of data analytics at Vidant Health, as well as an adjunct faculty member at Duke University. He is also the author of the  “Analytics Lifecycle Toolkit,” which is a manual for integrating data management technologies. A data evangelist with over 20 years of experience in analytics and advisory, Nelson is widely known for his human-centered approach to analytics. In this episode, Greg and I explore what makes a data product or decision support application indispensable, specifically in the complex world of healthcare. In our chat, we covered:

• Seeing through the noise and identifying what really matters when designing data products

Vidant Health Analytics Lifecycle Toolkit Greg Nelson’s article “Bias in Artificial Intelligence”  Greg Nelson on LinkedIn Twitter: @GregorySNelson Video: Tuning a card deck for human-centered co-design of Learning Analytics

“We'd rather do fewer things and do them well than do lots of things and fail.”— Greg

“In a world of limited resources, our job is to make sure we're actually building the things that matter and that will get used. Product management focuses the light on use case-centered approaches and design thinking to actually come up with and craft the right data products that start with empathy.”— Greg

“I talk a lot about whole-brain thinking and whole-problem thinking. And when we understand the whole problem, the whole ‘why’ about someone's job, we recognize pretty quickly why Apple was so successful with their initial iPod.”— Greg

“The technical people have to get better [...] at extracting needs in a way that is understandable, interpretable, and really actionable, from a technology perspective. It's like teaching someone a language they never knew they needed. There's a lot of resistance to it.” — Greg

“I think deep down inside, the smart executive knows that you don’t bat .900 when you're doing innovation.” —  Brian

“We can use design thinking to help us fail a little bit earlier, and to know what we learned from it, and then push it forward so that people understand why this is not working. And then you can factor what you learned into the next pass.” — Brian

“If there's one thing that I've heard from most of the leaders in the data and analytics space, with regards particularly to data scientists, it’s [the importance of] finding this “other” missing skill set, which is not the technical skillset. It's understanding the human behavioral piece and really being able to connect the fact that your technical work does have this soft skill stuff.” — Brian

“At the end of the day, I tell people our mission is to deliver data that people can trust in a way that's usable and actionable, built on a foundation of data literacy and dexterity. That trust in the first part of our core mission is essential.”— Greg

Nancy Duarte is a communication expert and the leader of the largest design firm in Silicon Valley, Duarte, Inc. She has more than 30 years of experience working with global companies and counts eight of the top ten Fortune 500 brands in her clientele. She is the author of six books, and her work as appeared in Fortune, Time Magazine, Forbes, Wired, Wall Street Journal, New York Times, Los Angeles Times, Cosmopolitan Magazine, and CNN.

In this episode, Nancy and I discussed some of the reasons analytics and data experts fail to effectively communicate the insights and value around data. She drew from her key findings in her work as a communication expert that she details in her new book, Data Story, and the importance of communicating data through the natural structure of storytelling.

In our chat, we covered:

• How empathy is tied to effective communication.

Nancy Duarte on LinkedIn

Twitter: @nancyduarte

Slidedocs

Duarte DataStory

“I think the biggest struggle for analysts is they see a lot of data.”  —Nancy

“In a business context, the goal is not to do perfect research most of the time. It’s actually to probably help inform someone else’s decision-making.” —Nancy

“Really understand empathy, become a bit of a student of story, and when you start to apply.”  these, you’ll see a lot of traction around your ideas.” — Nancy

“We’ve gone so heavily rewarded the analytical mindset that now we can’t back out of that and be dual-modal about being an analytical mindset and then also really having discipline around a creative mindset.” — Nancy

“There’s a bunch of supporting data, but there’s also all this intuition and other stuff that goes into it. And so I think just learning to accept the ambiguity as part of that human experience, even in business.”  — Brian

“If your software application doesn’t produce meaningful decision support, then you didn’t do anything. The data is just sitting there and it’s not actually activating.”  — Brian

“People can’t draw a direct line from what art class or band does for you, and it’s the first thing that gets cut. Then we complain on the backend when people are working in professional settings that they can’t talk to us.”  — Brian