Human-Centered Security

“Technical people need to better understand the laws and regulations and lawyers need to better understand the technology and processes in place. When that happens, when those worlds come together, that’s where you can meaningfully make things happen.” -Justine Phillips, Partner at Baker McKenzie

In this episode, we talk about:

• Essential questions product teams should ask legal experts when integrating AI into new products and features.
• In particular, why it’s important for designers and engineers to question the source of the data they are using for AI-powered products and features.
• The need to anticipate international security and privacy regulations, which are constantly changing, including emerging regulations that could impact companies developing IoT devices.

Justine Phillips is a Partner at Baker McKenzie, where she is co-chair of data+cyber for the Americas. She is the author of Data Privacy Program Guide: How to Build a Privacy Program That Inspires Trust.

What do CISOs have to say about the security tools their teams use?:

In this episode, we talk to cybsecurity leaders Bill Bonney, Gary Hayslip, and Matt Stamper about:

• The ever-evolving role of the CISO and what CISOs care about most.
• What product teams designing security software need to understand:
  • Security tools need to operate across varied ecosystems (which means your product team needs to understand those ecosystems).
  • Complexity is the enemy of security. Yes, UX matters.
  • Context-switching means security teams waste time. Instead, security tools need to present the right information at the right time.
  • Why CISOs are excited to leverage AI in security tools—and what concerns them the most.

Bill Bonney, Gary Hayslip, and Matt Stamper are seasoned CISOs and cybersecurity leaders. They are co-founders of the CISO Desk Reference Guide—a series of books including topics such as security policy, third-party risk, privacy, and incident response—which provide actionable insights for security leaders.

In this episode, we talk about: 

• Security tools don’t get a free pass when it comes to involving end users as part of the design process. 
• People studying and building ML-based security tools make a lot of assumptions. Instead of wasting time on assumptions, why not learn from security practitioners directly?
• Businesses (and academia) are investing a great deal in building ML-based security tools. But are those tools actually useful? Are they introducing problems you didn’t anticipate? And even if they are useful, how do you know security practitioners will adopt them?
• Why are adversarial machine learning defenses outlined in academic research not being put into practice? Jaron outlines three places where there are significant roadblocks: First, there are barriers to developers being aware of these defenses in the first place. Second, developers need to understand how the threats impact their systems. And third, they need to know how to effectively implement the defenses (and, importantly, be incentivized to do so).

Jaron Mink is an Assistant Professor in the School of Computing and Augmented Intelligence at Arizona State University focused on the intersection of usable security, machine learning, and system security. 

In this episode, we highlight two of Jaron’s papers:

• “Everybody’s Got ML, Tell Me What Else Do You Have”: Practitioners’ Perception of ML-Based Security Tools and Explanations.”
• “Security is not my field, I’m a stats guy”: A Qualitative Root Cause Analysis of Barriers to Adversarial Machine Learning Defenses in Industry

In this episode, we talk about:

• The role misaligned incentives play in security behaviors.
• How Serge and his team approach security-focused UX research. 
• Looking upstream at the security decisions made by software engineers and, in turn, the situations they are often placed in due to resource constraints and competing priorities at their organizations.
• Learning from other industries with highly-skilled professionals (shout-out to the humble check list!)
• Regulations and policy changes will likely place greater liability on the organizations shipping software.

Serge Egelman is the Founder and Chief Scientist at AppCensus and Research Director at International Computer Science Institute (ICSI). He’s written countless research papers on usable security and privacy. Most recently, his research centers around improving the user experience for users who are responsible for safeguarding their customer’s data (such as software engineers).

Shante Perrin, a cybersecurity leader, and her team use cybersecurity software to not only to detect and respond to cybersecurity threats but also, as Shante describes, to help paint a picture for their customers:

“We like to build a timeline of events to build that picture, create that story so we can deliver it to the customer and explain why we felt it is suspicious. In other words, why are we bothering you about this?”

In this episode, we talk about:

• Building stories from data: analysts must translate technical information into clear, understandable narratives for customers.
• If people designing cybersecurity software can design better, more effective experiences for analysts, analysts can do a better job of communicating these narratives to their customers.
• How security analysts at different levels perceive and handle threats differently—and how that changes what they need or expect from cybersecurity software.
• How thinking like an attacker can help security analysts—but only if the tools they use provide them with the right information at the right time. 

Shante Perrin is a cybersecurity leader and is currently the director of a managed services team. She led a cybersecurity team for a Fortune 100 company as an MSSP and has been a security analyst and security operations center (SOC) lead.

In this episode, we talk about: 

• The need for human-centered security—in order for security measures to be effective, they must center around people, making usability as crucial as technology. 
• We explore the gap between research and practice, highlighting the need to bring cybersecurity research into real-world application. Human-centered security research can’t possible be effective if no one knows about it or finds it challenging to implement.
• The importance of collaboration, advocating for more shared spaces where researchers and practitioners can come together to address pressing cybersecurity challenges.

Julie Haney is a Computer Scientist and Human-Centered Security Researcher and program lead at NIST (National Institute of Standards and Technology). She was formerly a Computer Scientist at the United States Department of Defense. In the episode we refer to two of Julie’s publications: “From Ivory Tower to Real World: Building Bridges Between Research and Practice in Human-Centered Cybersecurity” and “Towards Bridging the Research-Practice Gap: Understanding Researcher-Practitioner Interactions and Challenges in Human-Centered Cybersecurity.”

Security analysts respond to security detections and alerts. As part of this, they have to sift through a mountain of data and they have to do it fast. Not in hours, not in days. In minutes.

Tom Harrison, security operations manager at Secureworks, explains it perfectly, “We have a time crunch and it’s exacerbated by the other big issue security analysts have: we have an absolute ton of data that we have to sift through.”

In this episode:

Tom explains that security analysts are forced to go back to a pile of data with each subsequent question in their workflow. That’s a huge waste of time. And a terrible user experience. 

Tom says, “It would lead to better accuracy, faster triage, and a better user experience if you can just take me directly to the answer or at the very least a subsection that has the answer I’m looking for.”

What does this mean for you as a UX designer designing security products? You need a deep understanding of security analyst workflows to help them identify and respond to attacks as quickly as possible.

That way, you can design security products that support users who are under intense pressure to do things quickly. Tom describes how the UX can “guide or complement the workflow.”

Tom talks about what gets him excited about integrating AI into security analyst workflows—and what has him worried, as well.

Tom Harrison is a Security Operations Manager at Secureworks. We dubbed Tom an “ideas machine” and a fierce advocate for the security analyst user experience. In fact, Tom is conducting UX research in the field better than most UX researchers. He’s a passionate teacher and shares his knowledge and resources in a free security reference guide.

“Even though usability and security tradeoffs will always be with us, we can get much smarter. Some of the techniques are really simple. For one, write everything down a user needs to do in order to use your app securely. Yeah, keep writing.”

In this episode, we talk about:

• What is threat modeling and why should product teams and UX designers care about it? (Also check out Adam’s first episode on Human-Centered Security).
• Focus on parts of the user journey where you might gain or lose customers: what tradeoffs between usability and security are you making here?
• Involve a cross-disciplinary team from the very beginning. This is critiical: “How do we get focused on the parts of the problem that matter so we don’t spend forever on the wrong stuff?”

Adam Shostack is an expert on threat modeling, having worked at Microsoft and currently running security consultancy Shostack + Associates. He is the author of The New School of Information Security, Threat Modeling: Designing for Security and Threats: What Every Engineer Should Learn From Star Wars. Adam’s YouTube channel has entertaining videos that are also excellent resources for learning about threat modeling.

“UX design can enhance the overall performance, adoption, and impact in cybersecurity tools that leverage AI, making the tools more accessible to a broader range of users, including those who don’t have deep technical or security knowledge.”

In this episode, Siddharth Hirwani and John Robertson talk about:

• Pressures and challenges security analysts face and how AI can help.
• Moving beyond AI hype and focusing on integrating AI in a way that genuinely addresses security analyst’s needs.
• How UX design can foster trust and adoption of AI tools, while still encouraging analysts to verify AI outputs. 
• John and Siddharth highlight problems like over-reliance and bias and how UX can be leveraged to address these concerns.

Siddharth Hirwani is Senior Principal Product Designer interested in exploring the critical intersection of user experience and cybersecurity.

John Robertson is a researcher interested in the experience of technical users, especially those in cybersecurity. Recently his focus has been understanding workflows of cybersecurity analysts in security operations centers.

Siddharth and John will be presenting their paper “Cybersecurity Analyst’s Perception of AI Security Tools and Practical Implications” at USENIX SOUPS (Symposium on Usable Privacy and Security) in August 2024.

“People try to talk about the technical user experience at too high of a level. You talk about alert fatigue and you kind of understand what alert fatigue is just by the name. Yeah, there’s a lot of alerts. But watching it in action is different.”

In this episode, Heidi interviews John about what he’s learned about designing for security analysts. We talk about:

• The importance of understanding user workflows. “Alert fatigue” is just a saying until you actually observe it in action.
• While trust is hard to measure, it’s critical for improving the security user experience.
• Practical tips on how to promote cross-disciplinary collaboration.

John Robertson is a researcher interested in the experience of technical users, especially those in cybersecurity. Recently his focus has been understanding workflows of Cybersecurity Analysts in Security Operations Centers.