Thunder Nerds

283 – 🔊 Voice Content And Usability with Preston So

Listen Later

In this episode, we get to talk with Preston So, Senior Director of Product Strategy at Oracle. We talk to Preston about his new book VOICE CONTENT AND USABILITY. We discuss the concepts of building conversational designs that are ethical, accessible, and usable.

✨ Episode Sponsor
  • Auth0: https://auth0.com
  • Auth0 on YouTube: https://www.youtube.com/auth0
  • Auth0 on Twitch: https://www.twitch.tv/auth0
  • Auth0 Avocado Labs online meetup events: https://avocadolabs.dev/
🔗 Episode Links
  • Preston’s new book – Voice Content And Usability: https://abookapart.com/products/voice-content-and-usability
  • Publisher: https://abookapart.com/
  • Preston on Twitter: https://twitter.com/prestonso
  • Preston’s Website: https://preston.so/
  • Preston on LinkedIn: https://www.linkedin.com/in/prestonso/
  • Oracle: https://www.oracle.com/
  • Previous episode – 🪓 Headless CMS, Decoupling Drupal with Gatsby, & Conversational Design with Preston So https://www.thundernerds.io/2020/06/headless-cms-decoupling-drupal-w-gatsby-conversational-design-w-preston-so/
  • Ask GeorgiaGov: https://georgia.gov/chat
  • Google Cloud Dialogflow: https://cloud.google.com/dialogflow
  • Diglossia: https://en.wikipedia.org/wiki/Diglossia
  • Word by Word: The Secret Life of Dictionaries: https://www.amazon.com/Word-Secret-Life-Dictionaries/dp/110187094X
  • Conversations with Things: UX Design for Chat and Voice: https://www.amazon.com/Conversations-Things-Design-Chat-Voice/dp/1933820268/ref=sr_1_1
  • Invisible Man: https://www.amazon.com/Invisible-Man-Ralph-Ellison/dp/0679732764
  • Gatsby: The Definitive Guide: https://preston.so/books/gatsby/
  • Hosts: 
    • Frederick Weiss: https://twitter.com/FrederickWeiss
    • Brian Hinton: https://twitter.com/mrbrianhinton
📜 Transcript

Brian Hinton: [00:00:00] I’m Brian Hinton.

Frederick Weiss: and I’m Frederick Philip von Weiss.

And thank you so much for consuming the Thunder Nerds, a conversation with the people behind the technology
that love what they do

[00:00:46] Brian Hinton: [00:00:46] and do

tech good.

[00:00:52] Frederick Weiss: Yeah, thanks

everybody for watching the show. If you can please go to the notification bell and subscribe.

Brian Hinton: We’d like to thank Auth0, Auth0 is

this season’s sponsor. They make it easy for developers to build a custom secure and standards-based
login, a unified login and authentication as a service, to try them out, go to Auth0.com today. Also check
out their YouTube and Twitch under the username, Auth0 with some great developer resources and streams, and
last but not least is our avocado labs.

[00:01:43] I love that name. An online destination that their developer advocates run

organizing some great meetups. Thank you Auth0.

[00:01:52] Frederick Weiss: [00:01:52] Yes. Thanks

Auth0! Let’s go ahead and welcome our guest.

[00:01:50] Frederick Weiss: [00:01:50] Thanks so much,

Brian. So with that being said, and  without any dues being further, let's go ahead and get to our
guest and welcome him back. We have the author of the new book, VOICE
CONTENT AND USABILITY, senior director product strategy at Oracle, speaker,
Preston So. Preston, welcome back to the show!

[00:02:17] Preston So: [00:02:17] Hey Frederick.

Hey Brian. Thanks so much for having me back on Thunder Nerds. Might I say it’s a real pleasure to be
back here one more time to talk about my new book. Thanks for having me.

[00:02:26] Frederick Weiss: [00:02:26] I

appreciate it. And we started a little late and you have an event that you were just doing. Do you mind
telling us a little bit about that event?

[00:02:32] Preston So: [00:02:32] What that. I

will. And  my first and foremost dear apologies to everyone who was waiting for this live stream. I had
the misfortune of forgetting getting, send out a confirmation email and an email that actually I had,
let's say, Hey, this event is happening today. So we started a bit late and we ended a bit late.

[00:02:51] It was my launch event for my new book, which is here, voice content and

usability. And  we had a great time doing some discussion about the implications of voice interfaces
for those of us who work with the web, which is, I think a lot of us in the funder nerds audience, as well
as the implications of voice on our society.

[00:03:13] And of course, The vaunted and traditional book cake, which is something

that everyone at a book apart, my publisher has to unveil  as part of the process of launching a new
book. It was a  very interesting  process, but  very sorry to those who were waiting on this
YouTube person.

[00:03:33] Frederick Weiss: [00:03:33] Oh,

sorry. Did you say a book cake?

[00:03:35] Like

[00:03:35] Preston So: [00:03:35] utterly a

cake? Yeah. Book cake.  Maybe I'm saying too much. I don't know how, like it should be cake.
Gotcha. Yeah.  If she, yeah.   Not like everything is cake,  oh, it's all cake that
 yes. It's all cake as well, but a book cake, because basically  oh, you're supposed to
have a cake that looks like your book and    represents your book.

[00:03:55] Yeah.  So it was a great launch event and  it was a real

pleasure to  share a little bit about the process. I went through writing the book and some of the
really exciting things that  I taught. Love that.

[00:04:09] Frederick Weiss: [00:04:09] And

speaking of the book, we're going to be giving away three copies of the ebook courtesy of a book apart
today.

[00:04:16]     If you can just chat with us, ask us your questions. Maybe

tell us you want a book we're going to randomly give away some books. So we'll be doing that as the
show progresses on.  Preston first, let me talk to you a little bit about  you being with us last
time, promoting your last book.

[00:04:35] Decoupling Drupal. Am I saying that correct?

[00:04:38] Preston So: [00:04:38] Yes. Decoupled

Drupal in Practice.

[00:04:42]    Frederick Weiss: [00:04:44] How was the success of that and how did that  prompt you to  start writing a

new book?  You just wrote that book not too long ago and all of a sudden you have another book.

[00:04:54] So I see a pattern every year, a new book,  I wish I could come out

with a new book every year. Like someone    would say RL Stein of goosebumps or something like
that. But  This has been a really interesting process because my books tend to be very focused on
 really technical aspects of the ways in which we work with our content and the ways in which we work
on the web.

[00:05:19] Preston So: [00:05:19] The first book

I wrote was back in 2018, a couple of Drupal and practice. And I think one question I get a lot  and
 definitely happy to answer for some of those on the call or  those in the audience.
 What's it, what is it like as a technologist to write a book?  Especially for those who are
developers or designers.

[00:05:38] So this book is actually my first book that is not a coding book, not a

technical book. It doesn't have any code snippets in it. Couple of code formatted sections that are
really tiny, but it doesn't really have any sort of tutorials as to how to spin up a command line
interface or things like that.

[00:05:58] It's really focused on the user experience and design audience and the

accessibility audience, which is a very different audience from the audiences that I'm used to writing
for. What's interesting is that decoupled, Drupal and practice is about the architectural underpinnings
or the foundation of how you can deploy content.

[00:06:16] That's oriented towards things like JavaScript applications or other

sorts of environments, like voice interfaces.  But it really dives into the Navy. Voice content
usability. However, is really unlike that because it really focuses on how we as designers, as user
experience professionals who are working on usability testing or usability research  can really engage
with this new field that is emerging around voice interface design, and specifically around things like
voice content strategy and voice content design.

[00:06:49] But the other thing I will say is that I actually made the mistake. I had

the privilege, or some would say the misfortune of writing two books at the same time, over the past year
and a half. And the other book that I've got coming out this fall is Gatsby, the definitive guide, which
is about Gatsby JS, the static site framework.

[00:07:08] So right back in the other direction.

[00:07:10] Brian Hinton: [00:07:10] So

you're going to write

[00:07:12] Frederick Weiss: [00:07:12] every

other year,

[00:07:14] Brian Hinton: [00:07:14] three

minutes a year

[00:07:15] Preston So: [00:07:15] I was thinking

of more Fibonacci sequence, actually, Brian  like I think I should write five and then eight and then
13  Yeah, they might get a little shorter and they might be filled with some more memes.

[00:07:25] So why is voice content usability? Like, why did you're like, okay,

now I really think I need to write this.

[00:07:33] Yeah. Yeah. Specifically too, if I

[00:07:34] Frederick Weiss: [00:07:34] could

append to that point, Brian  Y   you said yourself, like you moved away from  like a coding
kind of thing. Like why go that way into

[00:07:42] Preston So: [00:07:42] the

accessibility?

[00:07:44] So I've always been  really into web development, but my real

core interest and passion has always been for design and user experience. I started out as a web designer. I
started out as a print designer. I actually also did computer programming back in the  back in those
days and got into web development that way.

[00:08:02] But it really wasn't necessarily something that was  an itch. I

got to scratch very much this aspect of design and user experience that is beyond the web. And I've
always been interested, not only in how we can serve some of the users who are interacting with some of the
content that we produce or some of the experiences that we create in  terms of technology beyond the
web.

[00:08:27] I was also really interested in how we can actually best serve. Users that

already exist and users that are already within the demographics of the audiences that we're trying to
serve.  I've always been interested in web accessibility first and foremost, as well as some of the
aspects of how accessible it really changes the ways that we think about other user interfaces that might
not have gotten and so much attention from the standpoint of how they can better serve disabled users and
those  who might be elderly and have a little bit more trouble  for example,  using a mouse
or typing on a keyboard and those two audiences, specifically the elderly and  disabled communities
around   the U S we're communities that we aim to serve with the first ever voice interface for
residents of the state of Georgia.

[00:09:15] I worked to ask Georgia gov, which had the specific goal of really

focusing on. How we can serve residents of the state of Georgia who want to be able to find out things like
registering the vote or how they can get a small business loan or how they can renew their fishing license
without necessarily having to incur the cognitive costs of either interacting with a screen reader driven
website or interacting with, let's say somebody in person at an agency office.

[00:09:45] And I think one of the really interesting insights that we found is that

 I think really unexpectedly is that a lot of the websites that we build, obviously we think. Nowadays
because so many people use the web because disabled folks use screen readers because so many people now are
used to the paradigm of the web.

[00:10:04] The website is really  the gospel of how people should now consume

content and how people do consume content. But I think one of the things that's been born out by this
project is that the kinds of things that people would ask an Amazon Alexa sitting in their own home about
the state of Georgia and the government capabilities that are available to them were completely
different.

[00:10:26] And in some cases, diametrically opposed to the sorts of queries and

things that people would search for on the georgia.gov website, which is the ultimate source of all of the
information that we used. And that really illuminates a little bit of this.  I would say a little bit
of this hidden bias that we have.

[00:10:43] Towards the website as the primary conduit for information, when in some

ways it really should be just considered one facet of a wide variety of ways to access our content
equitably. So then what do we

[00:10:56] Frederick Weiss: [00:10:56] do?

 Are we expected to  have multiple locations for our content, like specifically. I'm going to
build content for voice, or I'm going to build content for a website, and I'm going to build content
that goes into an application.

[00:11:14] Or am I  or does it behoove us to write content that is a uniform and

maybe in a specific way, and possibly you might answer in what way that, that might be as one source of
truth.

[00:11:31] Preston So: [00:11:31] That's a

really challenging question. And obviously I shouldn't really go too far here without saying that some
of those questions are answered in my book, voice, content, and usability as a book of parts.

[00:11:42] Please don't give everything away just a little bit. Could you read

the whole book out loud, please?  That'd  be here all day. Yeah.    Yeah, we do
have, so what I will say is that  this is the perennial debate, right? I think one of the things that
we as designers struggle with as we really deal with this exploding, Kind of menagerie of user experiences
that we increasingly have to deal with is  what do we do with our data?

[00:12:10] What do we do with all of these things that we've built that are in

some ways, very much oriented towards, or very focused on the audiences that we've cultivated over time,
namely our websites and mobile applications being for these very visually rooted experiences and
demographics that are used to these visual experiences, the things that are really problematic about some of
the approaches that were characteristic of the early days of voice content.

[00:12:37] Let's say when people were experimenting with voice interfaces or chat

bots, as a means to deliver a certain type of content, you would have a parallel version of the information
that was already housed in your website. And  those of us who are content designers or content straps,
Can really feel the pain that comes from the notion of having a set of content over here in one silo,
that's destined for the website and another piece of content over here, that's destined for a voice
interface.

[00:13:04] How do you keep those two things in sync? And now that we have regulations

like GDPR and HIPAA, for example, that are really obligatory, that content stays current, or that content
stays up to date with what we need. How do we actually make sure that all of this content stays up to date
without having it be in a single source of truth for content?

[00:13:24] Now, my book definitely doesn't make any prescriptions about going in

one direction or the other where, oh yeah, you must do it this way. Or you must do it that way because there
are exceptions to everything and  nothing is ever cut and dry. However, I generally err on the side of
saying that  look at the case of what we did with the state of Georgia, georgia.gov, they insisted
actually that we use one single source of truth for content that was going to be an omni-channel or channel
agnostic source of truth for content because ultimately a lot of us don't have the luxury to maintain
multiple versions of content that are destined for multiple conduits of content.

[00:14:00] So we ended up keeping it all in one source and we ended up maintaining it

all in one. And  having both voice and web versions of the content pull from the exact same repository
of content, which ends up being more scalable in the long run, especially now that Georgia has built an
additional chat bot that is a written chat bot, a textual chat bot, but also pulls from the same
content.

[00:14:25] I'm curious, there

[00:14:25] Brian Hinton: [00:14:25] was a course

of your research and writing of this book. Was there anything that shocked you or surprised you that you
didn't like?

[00:14:33] Preston So: [00:14:33] Didn't

immediately realize. Yeah.  It's a great question, Brian. I there's a    there's
too many to list because I think one of the things that's really one of the things that's really
 tough about voice interfaces is that up until recently, it's been really challenging for a lot of
those who are not computational linguists or machine learning engineers or people who are really deeply
involved in some of these very low level technologies to really get involved with voice.

[00:15:08] However, one of the things I will share is that in some ways there's

really interesting emergencies of some of the foibles in voice interface design. When you start working with
this technology that is very reminiscent of back in the day  and  those of us who were listening
to Thunderbirds.

[00:15:27] Have worked in the web for a while, will recognize, for example, the

things that we used to deal with in the early two thousands or mid two thousands, like quirks mode
compatibility, or  some of the really odd browser hacks that we had to do with CSS. And there's
weird things like that in voice interfaces.

[00:15:46] One example of this that I'll share and I'll keep it just to one

is when we build, ask Georgia gov, which of course is that voice interface for the residents of the state of
Georgia. There was a situation where we  had a retrospective. And one of the things that we did for
Georgia was they wanted to have the ability to administer and manage all this content in one single
place.

[00:16:08] And we had a parallel set of logs and reports that would sit right next to

the logs and reports for the website. So whenever somebody would hit a 4 0 4 error on the website  they
could compare and see. How many times did this piece of content also air out, for example, for  the
voice interface for Alexa, were there situations where the search return, the results or where it triggered
4 0 4 errors on the content management system that we were using to serve both the website and the voice
interface.

[00:16:39] So we had this retrospective about eight months after the launch of the

interface, which was  in 2017. And we had a discussion about some of the logs and we kind of leaf
through them and said, okay, what are some of the errors that we're seeing? And what are some of the
things that we can do to either adjust the content or maybe even do some debugging of the interface
itself?

[00:17:00] There was this one result that kept on coming up over and over again, this

one error, this 404 error, basically a search that somebody conducted that returned no results, no content.
And it was the word Lawson's L a w S O N apostrophe S. And this kept on popping up  over and over
again. It was about 16 times.

[00:17:20] If I remember correctly in the log and we thought. Who is searching, who

wants to search for this, like proper now this brand name  this person named Lawson    did
they get this confused with the different kind of application on their Alexa that they're trying to use?
And we sat there and scratched our heads for a few minutes.

[00:17:38] And one of the native Georgians in the room suddenly perked up and she

said, you know what? I think it's somebody who is from Georgia, who has a Southern drawl, who is trying
to say the word license as in driver's license or nursing license or fishing license, and sure
enough.

[00:17:57] That was exactly what happened. And this is one of those situations where,

Hey  you can do the best designed application that adheres to the latest and greatest standards and
specifications like we did back in those days with CSS and come within an inch of perfection when it comes
to these voice interfaces that we build custom.

[00:18:17] But ultimately it's in the hands of people like Amazon or Google,

whether or not they can actually understand the kaleidoscope of American English dialects that we have in
this country. And that we really should be able to understand. And I think it's a really good sign that
yeah, these voice assistants are really good.

[00:18:35] But they're not yet at that point where they can beat us at our own

game of human conversation. Yeah.

[00:18:40] Frederick Weiss: [00:18:40] This

brings me  if you don't mind really quick, Brian, this is something that  Todd Libby wrote
here, and he  he also appended to his question  where their edge and he wrote challenging where
they're challenging edge cases with respect to a 11 Y that you ran into the Georgia project.

[00:19:00]     Preston So: [00:19:01]

Yeah. Great question, Todd.  And when it comes to the work that we did  on accessibility, on
 Astoria gov    in terms of edge cases, I will share that. I think one of the big challenges,
there were several challenges, right?   And one I think     is one of the  one of those
challenges that's inherent to.

[00:19:20] Voice interfaces  that are pure voice interfaces, which I, and others

define as basically a voice interface that lacks a screen. So there's no visual component, no tactical
 or physical cues on it.  Yeah  not a gooey. You're basically just interacting with
somebody through the spoken word.

[00:19:37] And I think this is not really an edge case  so I don't wanna say

that this answers the question, but one of the things that I think a lot of people forget, and I think is
really important to keep in mind when working with voice interfaces, when it comes to extending the
accessibility of your content on a website or your web properties, is the fact that pure voice interfaces
 that are  lacking   in a visual or physical component  are actually not accessible to
certain disabled people, namely those who are deaf or those who are deaf blind.

[00:20:10] And the notion that I think a lot of people have today, Is  

 voice interfaces can solve a lot of cases for accessibility, but that's really not the case
because when it comes to  so many of the demographics that we need to serve in the disabled community,
there are  certain solutions that only go part of the way there and we're going to do that.

[00:20:34] Yup. And yeah, so that's  yep. Yeah.    That's

exactly right.   How do we also make sure that we can serve content on a mobile? Consumable way to
refreshable braille displays that are maybe not necessarily the same thing as the kind of  let's
say screaming and experience.

[00:20:52] That's very rooted in the visual structure of a webpage it's very

early days still in    this, the sort of notion of multimodal accessibility or how to really make
sure that a lot of the user interfaces that we have are not actually stepping on the toes of other folks who
are accessing content in particular ways.

[00:21:12] The edge case, however, that I will share is I think a lot of people also

make the assumption that these voice interfaces and voice assistants can be.       The
ultimate solution for a lot of folks who are blind or have low vision, but that's really a tough sell in
some ways, because I think one of the things that's really important to recognize about these peer voice
interfaces like Alexa, is that they have a learning curve too.

[00:21:38] We know this web meter and  some of these  browsers or three

meters like  ChromeVox  or Jaws have issues that require people to ascend a very steep learning
curve to use them in an effective and efficient way. And voice interfaces are very much the same way. So one
of the things that we encountered during our usability testing was.

[00:22:01] Just  one of those things that we didn't necessarily expect,

which is that a lot of people that we  had come in and  worked with and went through our usability
study, really had very little experience with Alexa devices. And I think for  those who are looking at
voice interfaces as a means to    be a compelling potential sidelong alternative to swim meters,
 that might necessarily,  that might potentially be a little bit problematic and how they  
 efficiently guide users to their content as  as the voice interface designer, Chris Mari writes
 it is something to think about, which is  there is still a learning curve.

[00:22:41] And how do you actually address that learning curve in a way that makes

sense to those users that you need to.

[00:22:47] Brian Hinton: [00:22:47] Yeah,

I'm curious in the sense of Georgia, where we're at my current role, we're working on a chat
bot. And one thing that we've found most difficult is I think it's called semantic, parsing a word
 converts that conversation into what logically makes sense.

[00:23:03] What are they asking? And it's like the difference, like the capital

of Georgia, someone's saying capital of Georgia and that's all they say, or what's the capital
of Georgia or Georgia Capitol is like  did you encounter anything weird in that sense or any  
 cases.

[00:23:19] Preston So: [00:23:19] Yeah.  I

talk about this a lot in chapter three of my book, which is about writing those conversational dialogues
that really are the lattice work of the voice interfaces that we produce.

[00:23:32] And it's a really challenging kind of thing because a lot of these

questions, Brian are really rooted in the technology that you're using.  Because some voice
ecosystems or conversational ecosystems are better equipped to deal with. Let's say variations, like the
ones that you mentioned just now, than others are.

[00:23:49] But there is a lot of work being done to improve the situation. So back in

the day  in 2016, when we worked on Astoria gov and in the grand scheme of voice interfaces and the
history of conversation design five years ago is a long time ago.  We might as well be talking about
clay tablets and abacuses at this point, because that was an era where a lot of those utterances that people
would state  in order to do a process of what's.

[00:24:17] Intent identification where the user interface is able to piece together a

sense of what the user actually wants to achieve, which is much easier said than done.  That's a
process that used to be very much a sort of manually driven process. For example, let's say that
you're trying to identify  a yeah.

[00:24:37] You're trying to identify a question like    what is the

capital of Georgia? It has to be phrased like a question, let's say. And one of the things that I think
is really challenging for a lot of people who are just getting started with voice interfaces is that in some
of these ecosystems, some of these technologies obligate you to be very clear about defining how the user
has to respond.

[00:24:56] And as we know, as users. The ways that we actually respond to some of

these questions and the ways in which we actually say some of these things can be phrased completely
differently from the ways in which we've actually coded  the voice interfaces or conversational
interfaces or chatbots to consider.

[00:25:14] And whenever we have, what's called a, an out of domain error where

the  chat bot or the conversational interface or voice interface, isn't able to actually understand
what you're saying, because the way that you phrased it, even though it's a perfectly logical thing
isn't accounted for within the context of what the voice interfaces in  is able to understand
through its  programming  is a  very big problem.

[00:25:39] So I'd talk about intent identification   and  the problems

that occur when you have these  very dedicated slots or tokens or some of these  No, basically
this teasing out process that you have to do with intern identification that really relies on some of these
boilerplate templates that users have to use to say these things, but that's not how we speak.

[00:26:00] That's not natural, right?  Nobody really wants to have to say

things the same way. Over and over again, to be understood by a voice interface. Although there is usability
research evidence that suggests that some users do prefer that. But there are some ecosystems now, like
dialogue flow, for example  or    some of the major new  conversational  tools that
are out there are getting better at  understanding, let's say all the different variations that you
could possibly have and being able to intelligently parse through that and say, okay, this is the intent of
what the user is trying to do.

[00:26:36] Even though this person might have said something that's very remote

from the, let's say a normal way or the default way that we would expect.

[00:26:45] Brian Hinton: [00:26:45] Yeah. My

favorite, like real life scenario of beating my brain, being the AI, trying to understand is when I,
somewhere, I can't remember where it was, Midwest that they asked  what Coke do you want?

[00:26:56] And I said, Coke. And they're like, I'm sorry. Is that okay? Yeah.

[00:27:04] Preston So: [00:27:04] That's

what they call it.

[00:27:07] Brian Hinton: [00:27:07] I can't

imagine dealing with that sort of a scenario, isn't it? AI type?    Yeah.  That's
funny too, cause it could be something where  if you're trying to  communicate something out
   to the bot or the voice technology, you got to think about the context of the personification
of this voice   or  the overall brand.  If I'm interacting with a hospital, I don't
want the voice to sound all silly and goofy. I  I want it to sound like a, just a normal, regular
voice.  There are some kinds of situations   that you might want  or even languages for that
matter.  If I'm somebody in Italy and I'm looking for.

[00:27:48] Frederick Weiss: [00:27:48] A lasagna

recipe and I'm in Italy and I'm looking for a lasagna recipe and I go to, and it sends me to the
food network and it starts reading me like a M roll recipe in  in English. And I don't understand
English.  There's all kinds of interesting  facets

[00:28:01] Preston So: [00:28:01] to this,

 yeah, this really brings up, I think a couple of interesting elements of the ways in which the
conversation design or voice interface design landscape really requires us to think very differently about
some of the things that we usually took for granted.

[00:28:17] And one of those really  is the building blocks of language. And

I'm very lucky in that. Working with voice interfaces over the past five or six years has really allowed
me to scratch my itch when it comes to my academic background, which is actually in linguistics. I have a
degree in linguistics. Not a lot of people know that.

[00:28:34] But the biggest issue, I think a lot of us face is we're moving.

 In several directions at the same time, the first is that we're moving a lot of the ways in which
we use to write user interface, texts, or  content from the written word over into the spoken word,
which is a very different realm from how we normally write UI texts.

[00:28:56] Are we, how are we? Normally I actually write content. And  just one

example to illustrate that is the fact that we don't really say the phrase to whom it may concern when
we actually speak. And we also don't really write the word literally, as often as we say it in
conversation. So a lot of these little nuances are things that can often be missed.

[00:29:17] And there's two ways in which this really. Can be a problem. The first

is that there are certain expectations that users will have that their voice interface reflects the kind of
informal or colloquial conversation that they might have with a friend. And when it doesn't reflect
that, and when the voice interface comes out with this very kind of stilted utterance or something,
that's a very uncanny valley, like I can really interrupt or dislodge the user from what  is
 called habitability  and a voice interface.

[00:29:48] This is something that is  talked about quite a bit in voice

interface literature, where the user has to feel like they're not gonna want to actually tear their hair
out  or what little hair they have in terms of having a conversation with a voice interface. So
that's number one, but I think number two is really interesting given that you alluded to some of the
challenges around multilingualism.

[00:30:09] Types of conversation. And this really comes to, I think, some of the

elements of voice interface design that remain a largely unexplored area and also an area that is very
challenging because of the fact that so much of our conversational technology and voice interface technology
has so far been rooted in the English speaking world.

[00:30:30] And one of those issues is when we think about the ways in which we want

to serve multilingual audiences and international audiences on the web, we just have to provide translatable
strings, right? We just have to provide like these versions of these different pieces of texts that we have
or different pieces of content we have.

[00:30:48] But that is a  very different kind of proposition when it comes to

some of these other languages. And I think one of the biggest issues that we have to focus on. Is the fact
that not all languages work like English, not all languages operate in the same kinds of systems and the
same kinds of assumptions that a lot of us have about English.

[00:31:08] And one of the things that is really interesting to me is that I'm

noticing more and more some of this Anglophone privilege or Anglophone bias in a lot of the voice interfaces
that are coming out that are meant to be multilingual are also direct translations of an English interface
because fundamentally some languages simply do not work the same way as English.

[00:31:28] There's a phenomenon in linguistics called Dyke Glossier. And this is

something I talk about on my blog, Preston dot.  And this notion of glossy is actually a phenomenon. I
studied also when I was in college where the written form of a language is so vastly different from the
spoken form of a language that they might as well be considered two different dialects or two different
vernaculars.

[00:31:50] And in some cases. Like Brazilian Portuguese, for example, you really have

to learn two different grammatical systems and two different  lexicons and two different approaches to
the language in order to make yourself understood. Because if I went out on the street and I started
speaking in the way that I write, I wouldn't actually be necessarily understood.

[00:32:10] It I'd be understood because people would be able to understand, but

it would be a very strange and off-putting conversation. What I find is very interesting with a lot of the
work that conversation designers are doing today is that there's a lot of focus on efficiency and
scalability, where we can build one single conversational agent or one single conversational interface that
manifests as a chat bot as a slack bot, as a WhatsApp bot Facebook messenger bot, and as an Alexa skill and
a Google.

[00:32:36] But there's a big problem with that, because that assumes that the

same kind of conversation you would have with a chatbot is going to be the kind of conversation you have
with a voice interface. And one of the things that we see in linguistics and also in the kinds of
conversations that we have on a daily basis through email and texts and at the delegate.

[00:32:57] It isn't the case that our spoken conversations are word for word or

even letter for letter. Exactly the same as our written conversations. And for those who don't speak
English, for those who are operating in a realm where let's say that  the language  that
they're writing for is not English.

[00:33:16] A lot of those considerations and concerns become a lot more important

than essential when it comes to some of the design that we have to do. And I think this means that we have a
long way to go in the English speaking world to understand how some of these conversational interfaces
 really are rooted in our ways of speaking in ways that might not be so appropriate for the rest of the
world that we need to.

[00:33:38] Brian Hinton: [00:33:38] Yeah, all of

this made me think of a book. I recently read  word by word, the secret life of dictionaries.  And
it's a fantastic book, but it's like the slang too, of how  you mentioned the different
versions of Portuguese, the slang is different  like Mexican slang versus  Spanish, Mexican slang
versus Spain.

[00:34:00] Spanish slang, very different and English slang, different, like someone

said, and also how people will say things like cool versus cool, like completely different.  And
 how to interpret that    yeah, Johnny. Yeah. Tone.

[00:34:15] Preston So: [00:34:15] Yeah. And I

think this really illustrates a couple of different things.

[00:34:18] You've got the subtext that is not something  in UI text or in

web content or in any of the word mediums that we have.  And paralanguage sticks in this realm of,
okay. How are you actually?  Really reflect back the fact that the user or the interface might be
speaking in a sarcastic tone or in a more assigned tone or in a very stilted tone.

[00:34:43] Like those three things can mean very different things, even though they

all use the same single sentence. But the other thing that's really interesting too, Brian, and I think
you raised a really good point there, which is  it's not just the fact that we have all these
differences between languages  and the ways that they operate.

[00:34:58] We also have very important differences. Like I mentioned earlier with

that Lawson's example  around those of us who speak English. And one of the things that worries me
a lot about some of these voice interfaces is first of all, the fact that  we hear fundamentally one
single dialect represented oftentimes in this  realm of voice interfaces.

[00:35:19] And it's very similar in some ways to the ways in which newscasters

and weather forecasters used to have to be obligated. By their organizations to speak using a middle
American or general American dialect. It was unacceptable in certain past decades, in the news media for
somebody to speak with a Southern accent or somebody to speak with a different dialect of American English
on the air.

[00:35:43] And that's something that's represented now in voice interfaces,

in both a very limiting and very pernicious way. Because as we know, from interacting with so many different
people from so many different walks of life, not only do we have examples of people who  might be
bilingual or who might be members of a  queer or trans communities who have to switch between different
modes of speech or those who are bilingual  descendants of immigrant communities who have to be able to
code switch between English and Spanish, why aren't those sorts of interesting toggles and those sorts
of interesting nuances.

[00:36:17] Representative voice interfaces too, because maybe the kind of

conversation that I want to have is the kind of conversation that I would have at home in new Delhi, where
I'm switching in between English and Hindi mid-sentence or I'm switching in between English and what
I think mid sentence. So these sorts of considerations are not only important for those who are users of
English in outside of America, which I think is  one example of the America centric approach that we
often have with technology  all over the place.

[00:36:46] But also the fact that we have been very marginalized and

underrepresented.    Oppressed  groups of people in the United States who speak in certain
ways that are not reflected in how we want voice interfaces to speak as well. And I think one very
compelling example, two very compelling examples of this is  first of all, the fact that  the ways
in which people use  AAV or African American vernacular English is very different from the sorts of
voice interfaces that we interact with.

[00:37:14] For example, why is it that  we can't hear those sorts of

conversations represented in an Alexa device. It has something to do with the intrinsic bias that a lot of
us have for a more middle American or general American approach to the conversations that we have. Of
course, fundamentally and foundationally a white American form of speech.

[00:37:33] And by the same token  we know  that those who identify as LGBTQ

have very different approaches to using certain language.  There's certain code terms. There are
certain colloquialisms that are really not understood by audiences that are outside of that community. And
how do we make sure that voice interfaces can also represent those things?

[00:37:54] And this ties back to one of the things I talked about. In the final

chapter of my book, which really is focused on the problems that surface that we don't consider when we
go Willy nilly into this realm of voice interfaces and serving people through conversation in ways that we
don't expect. And one of those examples is  think about why organizations today and think about why
it is that so many people want to get into voice interfaces and want to get into chat bots in the first
place.

[00:38:20] So many people are doing this because these airlines, hotels, large

companies, corporations, they fundamentally want to be able to reduce the load on their customer service,
frontline agents or those who are cost center staffers. But if you think about it, who are these call center
staffers? Who are these people who answered your recall when you're calling them in the middle of the
night from the airport, screaming about your lost luggage or screaming about your canceled flight.

[00:38:44] It's somebody who might be in the Philippines or somebody who might be

in India or somebody. Might be in the global south, it was a person of color who is from a lower middle of
middle income country who doesn't have the resources necessary to speak in a general American dialect in
the same way that you would expect somebody who's from your own community to speak.

[00:39:03] And this really illustrates a very, I think, big concern in voice

interfaces today, which is. When we begin to sterilize and flatten out all of these rich nuances that make
our conversations with all of these different people and from all of these different lived experiences, so
important to our worldview and to the ways in which we interact with the world.

[00:39:26] What does that do to our future as users? What does that do to our level

of trust in our user interfaces? What does that do to credibility and authority? Of those user interfaces
and the information that they provide, because let me be honest. When I think about the fact that a voice
interface might lead to a Filipino  center worker or somebody who is in Mumbai, who is  in a call
center losing their jobs.

[00:39:52] I'm not so sure that I want that replacement to be this uncanny valley

voice that is very stilted and mechanical and might not necessarily reflect the world that we live in today.
And I think this really ties into a lot of the issues that we face around  misinformation and automated
racism and algorithmic oppression  that we see around machine vision and so on and so forth, voice
interfaces and voice technology and conversational technology.

[00:40:18] These are also domains that are not exempt from the issues that we have in

society. Yeah,

[00:40:24] Frederick Weiss: [00:40:24] we start

losing the quality of humanity and what you Manatee is, but is there anything  I know you were talking
a lot about in chapter six, about, about the future. Are there any brighter  notes that you could
 no.

[00:40:41] Frederick, there's not, yeah. I don't want to go down the matrix

road, but  are there any like cool new things is that we could be looking forward to or  things
that we could start thinking about now that would be advantageous for us to go, oh, you know what, let me
next year start thinking about this so I could get my projects.

[00:41:01] Preston So: [00:41:01] Yeah,

absolutely.  There's so much to think about. And obviously I wouldn't have written this book if
I thought it was going to be a dystopian nightmare and the next few years, or next few decades  because
voice technology really does have a lot of  illuminating and very interesting    prospects
that I think there's really important things to call out there.

[00:41:19] Not just the facts. And this is not something I mentioned very much in my

book, but I do mention it very briefly in my  Alyssa part, article usability testing for voice content,
which is that there aren't a lot of people who I really appreciate waist interfaces for one unexposed.
And that is that I think, as we all know, a lot of us, especially over the course of the last year and a
half.

[00:41:42] And  I do want to make sure to hold space for those who are

 still  dealing with grief or suffering right now from the consequences of the coronavirus
pandemic. Especially of course in India and Australia currently going through a very severe lockdown
 and the third wave ongoing in Africa  Voice interfaces have been shown to stave off loneliness
for a lot of people.

[00:42:05] There is research that suggests that having a voice interface that is

there to have a conversation with  is something that could be very beneficial for mental health. And in
the future, as these conversations become better and better as voice interfaces, get to the point where they
can do much better, small talk than  these really simplistic, let's say gimmicky responses that
they often issue.

[00:42:28] I think we can really look forward to a lot of interesting, let's say

social  benefits from voice interfaces. The other one though, I think is also the fact that  there
is going to be more efficiency when it comes to content delivery and information delivery. There's a.
Futurists named Mark Curtis, who refers to what's called the conversational singularity.

[00:42:47] And we know about the kind of tech or AI singularity, the conversational

singularity is along the same lines, which is this notion that as we move further and further into the
future, there's going to be a point in time where conversational interfaces will be indistinguishable
from other humans when it comes to the kind of conversation that we three are having right now.

[00:43:09] And one of the things that I think is important to call out, of course, as

well. Okay. That's a great kind of future, but conversational singularity is going to be
indistinguishable, but for whom, right?  Whose conversations are going to be indistinguishable.
 As I was just saying earlier, but I think one of the really interesting things about the
conversational singularity and some of them.

[00:43:27] Let's say conversations, centric, approaches that are coming out,

which wash away some of the weird distortions that we have today, some of these arbitrary lines in the stand
that we have, where you talk with a certain Alexa skill or a certain Google assistant, and they can only
help you with this one, certain task.

[00:43:43] They can only help you order a pizza, but they can't help you book a

flight.  These sorts of interactions will soon become smoother because you know what, maybe I do want
to go directly into just like I would with a hotel concierge.  Actually have a conversation that moves
directly into ordering a pizza.

[00:43:58] With extra pineapple as it should be. And then directly into booking a

flight  over to    my favorite vacation destination. So a lot of these efficiencies are going
to become very important in the future. And I think what's going to happen in the next few decades is
we'll start to see ways in which, okay.

[00:44:16] Yeah. Some of these  issues that we have with how conversational

interfaces work or reflect the world that we live in back at us  are going to become better in terms of
the efficiency and ultimately the performance of user interfaces in the same way as that websites and mobile
applications have become much more efficient and much more able to get us over to the things that we want to
do.

[00:44:41] Frederick Weiss: [00:44:41] I

remember at a Google IO, they had a, what was the one assistant that called to book a hair appointment for
somebody. And they were like, oh yeah  it's completely indistinguishable from a person that's
wrong.

[00:44:53] Preston So: [00:44:53] I can totally

tell I'm saying,

[00:44:57] Frederick Weiss: [00:44:57] yeah, I,

yeah, I think you could tell, but they said  and if you're on a phone call things  you have
things in the background you're trying to get through things quickly and you're like, yeah,
whatever.

[00:45:09] Yeah. It could work. I'm sure.  One day, like you said, a person

will get like that  that movie, her with Joaquin, Phoenix and Scarlett, Troy.

[00:45:17] Preston So: [00:45:17] Yeah.

 Who among us hasn't accidentally answered an    automated phone call. That sounds
exactly like a conversation. What are those spam calls that were all besieged by lately and answered a
question because it sounded so real or  perish the thought, and this is going to be very
revealing.

[00:45:32] I think we've all done this, you accidentally answer somebody's

voicemail. Automated message saying, Hey, it's Preston. Oh, Hey  I'll leave a message at the
tone. Oh wait. Okay.

[00:45:43] But yeah, I think   it's a really exciting time and I do think

that I think one of the things  that's important, and I think this book is very timely, right?
Because one of the things I will admit is that when this book first was being    germinated as an
idea, I thought  it might be a little early because this project that we did for Georgia  was very
early in its time.

[00:46:03] It's one of the first ever content driven information driven voice

interfaces. It's also really one of the first, very few examples of state governments and local
governments doing this kind of work at the time, too.  But now I think it's very timely because
 one of the things that we've seen over the course of the past year and a half is smart speakers,
smart home systems.

[00:46:24] Everyone's buying them, they're flying off the shelves and

increasingly here as we re-answer the world or  live with the virus as it continues to be a problem for
so many of us in the world, Just start getting used to some of these other ways of interacting with content.
Other ways of interacting with information, with use cases and applications that we need to actually go
through.

[00:46:48] And voice is just one of those. And  I think we're going to see a

lot more investment and a lot more care from the user experience side, not just the developer side  
 in this realm of, okay, we've done this for the web and the web has served us really well for the
last few decades, but how do we actually make sure that some of these more multimodal approaches, as we
mentioned earlier on accessibility or some of these more interesting immersive or voiced an oral  and
immersive approaches can be things that will be compelling for users and designers and practitioners in the
future as well.

[00:47:25] Frederick Weiss: [00:47:25] Makes

sense? What do you think Brian? Or should we go to the lightning round? Yeah. Yeah.

[00:47:31] We're

[00:47:31] Brian Hinton: [00:47:31] getting

close to the end here. So we're

[00:47:33] Frederick Weiss: [00:47:33] flying

rats on. I've got my gloves on. Let's go ahead.

[00:47:37] Brian Hinton: [00:47:37] Yeah. So

we're each gonna ask you a question, answer yours, and  one at a time.  And I'll go first.
So would you rather be able to run at a hundred miles per hour or fly at 10?

[00:47:49] Preston So: [00:47:49] I have to

think about this one.  Probably fly and it's. Yeah. It's because you can see more. Yeah.
That's fair.

[00:47:59] Frederick Weiss: [00:47:59] Preston.

What is your favorite thing about yourself?

[00:48:04] Preston So: [00:48:04] Oh my gosh.

  Oh my gosh. These are some questions y'all really, I don't remember the last lightning round
being like this. I think my favorite aspect about myself is that I have learned a lot and I've had the
privilege of living in many different countries, which not everybody has the privilege to say.

[00:48:28] And that's given me a lot of good perspective. I'll say that.

   Would you rather live where it snows all the time or where the temperature never falls below a
hundred

[00:48:39] degrees? Wow. This is like Snowpiercer versus thread 3d or something like

that. Def. So  I'm somebody who needs, so right now I am in an air conditioned room, even though
it's actually not that hard of a day here in New York city, I need the cold, I cannot deal with the
heat.

[00:48:57] And  so yeah, it's definitely snowing all the time. I could

probably be okay. In, in, in Antarctica actually, I would say, okay,

[00:49:06] Frederick Weiss: [00:49:06] Preston,

what book are you yourself reading?  To  to learn from currently that you're

[00:49:12] Preston So: [00:49:12] enjoying. All

right.    I'm currently reading three different books.  Not really making much progress
in either of those; it's  like the Fibonacci sequence of reading books  and increasing those
every year.

[00:49:28] One book that I'm reading, which I will share, which is a very

 esoteric book right now is  Bosnian Croatian and Serbian a textbook because I'm learning
Serbo Croatian at the moment as a language, but I'm also reading two other books that are really
interesting.  The first is conversations with things which is a book written by Rebecca Ivanhoe.

[00:49:49] And I forget the co authors name. I have it right here. I should look at

it.  As well as  Margot Bloomstein book trustworthy, which is a book about  how brands can be
more authentic in how they operate in terms of content strategy.

[00:50:09] Cool.

[00:50:09]   Brian Hinton: [00:50:11] What

current fact about your life  would most impress your five-year-old self?

[00:50:19] Preston So: [00:50:19] Oh my God.

Wow. My five-year-old self. Got it. I thought that was an easy question. You answered it last time. Did I
really? Oh my gosh.  Let me think.      The fact about myself, that people  I think
the fact that my five-year-old self would most be impressed by is  the fact that, oh my gosh  
 after the fact.

[00:50:48] Frederick Weiss: [00:50:48] I

remember last time you said moving to New York and working in New York was one of your childhood dreams,

[00:50:54] Preston So: [00:50:54] giving them

the answers. That's really funny, cause that's not what I, that's not what I would say to myself
actually. That's really interesting.    You know what I'll say is this actually I think
this is an interesting one because  just to get a little personal here  when I was, and  a
lot of us dealt with this when we were younger  a lot of us as children, as young toddlers are as
young.

[00:51:14] Kids, we deal with  speech impediments or other issues with

 let's say  pronouncing words correctly, or  doing those sorts of things. And I grew up
with a speech impediment, which makes also some of the voice technology kind of things, really poignant.
 So what I would say is my five-year-old self would definitely be very proud of me for the fact that I
 can basically go on stage in front of 3000 people and not break a sweat  or have this live stream
with also 3000 people.

[00:51:43] Of course, there's 3000 people listening to this right now.  And

not break a sweat either.      Yeah.  With a personal note there.

[00:51:52] Frederick Weiss: [00:51:52] Nice.

What is the most interesting thing that you learned in the process of writing

[00:51:58] Preston So: [00:51:58] this book?

Most interesting thing that I learned in the process of writing this book, the most interesting thing I
learned in the process of writing this book is probably the.

[00:52:10] Unexpected applications of accessibility. And unexpected challenges around

accessibility that occur with voice interfaces, especially given the fact that I think a lot of us are
accessibility   efficient autos or those who are really  passionate about accessibility. I think
we often forget that.

[00:52:31] Not only are there    so many different types of interfaces that

we need to consider  the interface that has  become the most important one today, which is that
the screen reader for websites is actually not necessarily the most optimal or pleasant experience. And
 I already did have a sense of this because I  do a lot of        and this is
one thing I think everybody should do is you should always take.

[00:52:57] Sort of user interface, you're building and using it from the

perspective of somebody who's using a screen reader or somebody who's using  an assistive
interface, because it is very important to understand  how  people work from that perspective.
 But one of the things, so I already knew that  screen meters were really very tough, but I guess
one of the things that I didn't necessarily realize is  just how much people actually really
don't like the screen reader sometimes, and really see it as an obstacle to  getting to what they
need.

[00:53:28] That was a very long answer.

[00:53:31] Frederick Weiss: [00:53:31]

That's okay.

[00:53:35] Brian Hinton: [00:53:35] What book

 has made you cry?

[00:53:41] Preston So: [00:53:41] What book has

made me cry?  Gosh. Yeah, that's a really interesting question. Wow.    
 There's been, there's definitely been    many books that have made me cry. I would
say  the book that both made me cry and made the deepest impact  on me is probably, oh my gosh.
I'm just trying to think about this now because  yeah  the, what I will say is  the book
that has definitely made the biggest impact on me and made me cry.

[00:54:22] Both of those were probably invisible men.  Which is a book that I

recommend everybody read.  It's one of those books that you read in high school   or college
English class, but    it's a very important book and something that I think everybody should
definitely read. Let me

[00:54:40] Frederick Weiss: [00:54:40] I'm

out of lightning round questions.

[00:54:43] Brian, do you have anything else on that? Oh, no, I think we're good.

Great.  Let's get to our final topic here at the end Preston. We like to ask our guests for parting
words of wisdom, any kind of  things that you'd like to    tell our audience at the end.

[00:55:01] Preston So: [00:55:01] Yeah.

 Great question.  I think my biggest parting words of advice for everyone, and this is not just
those who are in the design field or who are  in the technology world.

[00:55:16] But I think one of the things that I would recommend for everyone who is

watching this or listening to this, or will watch or listen to this  is that it's really important
to really listen to. And uplift and amplify and also hear and take into account in your own day-to-day work
and your own day-to-day life.

[00:55:41] The lived experiences of those who are completely unlike you. And by

completely unlike you  all of those people who face a multiple axes of marginalization or oppression,
or who faced very deep obstacles in our world today, who might be disabled, might be women or femmes might
be people who are queer or trans might be people who are of color, who are black or indigenous.

[00:56:09] And I think one thing that is really important to me, and one thing

that's very  important to the way I live my life is. So I really deeply understand where everyone
is coming from in terms of their context and in terms of how they have come to be the person that they are
today. Because ultimately as practitioners of technology, as those who work on technology, the ultimate
reason we're doing this is to help everybody who is our audience, succeed with what they're
doing.

[00:56:42] And there's no way to do that unless you really deeply understand and

take the time to learn about and  comprehend what it is that your audience goes through in any field
that we work as people in this  world that we live in.

[00:57:02] Frederick Weiss: [00:57:02] Very well

said. Thank you, Preston.  Again all your social links we have at Preston So.

[00:57:07] So on Twitter, your website is Preston So. Presence on LinkedIn. And of course the

new book, Voice
Content and Usability by Preston So. Get it
there today. Preston again, thank you so much for being on the show. We  really appreciate it.

[00:57:32] Brian Hinton: [00:57:32] No, thank

you for taking the time.

[00:57:35] Preston So: [00:57:35] Thank you both

so much. It was such a pleasure to be here on Thunder Nerds again, and I'd love to come back sometime.
 Maybe I'll rehearse some lightning talk or lightning question responses for next time, but thanks
so much for having me. I appreciate it. Yep.

[00:57:47] Frederick Weiss: [00:57:47] Thank

you!

[00:57:48] Yeah, for the next book. We'll see you then. Thanks everybody. Oh,

hold on. I got one last comment. Let's see.  Thank you for all. Todd- “Thank you for all the
phenomenal conversation”. Thank you so much, Todd. Thank you everybody for watching. Really appreciate
it. Take care everyone.

If you have questions, or suggestions to modify the transcript, PLEASE let us know at

[email protected]

...more
View all episodesView all episodes
Download on the App Store
Thunder NerdsBy Frederick Philip Von Weiss, and Brian Hinton
  • 4.9
  • 4.9
  • 4.9
  • 4.9
  • 4.9

4.9

22 ratings