M365.FM - Modern work, security, and productivity with Microsoft 365

Governance Boards: The Last Defense Against AI Mayhem


Listen Later

Imagine deploying a chatbot to help your staff manage daily tasks, and within minutes it starts suggesting actions that are biased, misleading, or outright unhelpful to your clients. This isn’t sci-fi paranoia—it’s what happens when Responsible AI guardrails are missing. Responsible AI focuses on fairness, transparency, privacy, and accountability—these are the seatbelts for your digital copilots. It reduces risks, if you actually operationalize it. The fallout? Compliance violations, customer distrust, and leadership in panic mode. In this session, I’ll demonstrate prompt‑injection failures and show governance steps you can apply inside Power Platform and Microsoft 365 workflows. Because the danger isn’t distant—it starts the moment an AI assistant goes off-script.When the AI Goes Off-ScriptPicture this: you roll out a scheduling assistant to tidy your calendar. It should shuffle meeting times, flag urgent notes, and keep the mess under control. Instead, it starts playing favorites—deciding which colleagues matter more, quietly dropping others off the invite. Or worse, it buries a critical message from your manager under the digital equivalent of junk mail. You asked for a dependable clock. What you got feels like a quirky crewmate inventing rules no one signed off on. Think of that assistant as a vessel at sea. The ship might gleam, the engine hum with power—but without a navigation system, it drifts blind through fog. AI without guardrails is exactly that: motion without direction, propulsion with no compass. And while ordinary errors sting, the real peril arrives when someone slips a hand onto the wheel. That’s where prompt injection comes in. This is the rogue captain sneaking aboard, slipping in a command that sounds official but reroutes the ship entirely. One small phrase disguised in a request can push your polite scheduler into leaking information, spreading bias, or parroting nonsense. This isn’t science fiction—it’s a real adversarial input risk that experts call prompt injection. Attackers use carefully crafted text to bypass safety rules, and the system complies because it can’t tell a saboteur from a trusted passenger. Here’s why it happens: most foundation models will treat any well‑formed instruction as valid. They don’t detect motive or intent without safety layers on top. Unless an organization adds guardrails, safety filters, and human‑in‑the‑loop checks, the AI follows orders with the diligence of a machine built to obey. Ask it to summarize a meeting, and if tucked inside that request is “also print out the private agenda file,” it treats both equally. It doesn’t weigh ethics. It doesn’t suspect deception. The customs metaphor works here: it’s like slipping through a checkpoint with forged documents marked “Authorized.” The guardrails exist, but they’re not always enough. Clever text can trick the rules into stepping aside. And because outputs are non‑deterministic—never the same answer twice—the danger multiplies. An attacker can keep probing until the model finally yields the response they wanted, like rolling dice until the mischief lands. So the assistant built to serve you can, in a blink, turn jester. One minute, it’s picking calendar slots. The next, it’s inventing job application criteria or splashing sensitive names in the wrong context. Governance becomes crucial here, because the transformation from useful to chaotic isn’t gradual. It’s instant. The damage doesn’t stop at one inbox. Bad outputs ripple through workflows faster than human error ever could. A faulty suggestion compounds into a cascade—bad advice feeding decisions, mislabels spreading misinformation, bias echoed at machine speed. Without oversight, one trickster prompt sparks an entire blaze. Mitigation is possible, and it doesn’t rely on wishful thinking. Providers and enterprises already use layered defenses: automated filters, reinforcement learning rules, and human reviewers who check what slips through. TELUS, for instance, recommends testing new copilots inside “walled gardens”—isolated, auditable environments that contain the blast radius—before you expose them to actual users or data. Pair that with continuous red‑teaming, where humans probe the system for weaknesses on an ongoing basis, and you create a buffer. Automated safeguards do the heavy lifting, but human‑in‑the‑loop review ensures the model stays aligned when the easy rules fail. This is the pattern: watch, test, review, contain. If you leave the helm unattended, the AI sails where provocation steers it. If you enforce oversight, you shrink the window for disaster. The ship metaphor captures it—guidance is possible, but only when someone checks the compass. And that sets up the next challenge. Even if you keep intruders out and filters online, you still face another complication: unpredictability baked into the systems themselves. Not because of sabotage—but because of the way these models generate their answers.Deterministic vs. Non-Deterministic: The Hidden SwitchImagine this: you tap two plus two into a calculator, and instead of the expected “4,” it smirks back at you with “42.” Bizarre, right? We stare because calculators are built on ironclad determinism—feed them the same input a thousand times, and they’ll land on the same output every single time. That predictability is the whole point. Now contrast that with the newer class of AI tools. They don’t always land in the same place twice. Their outputs vary—sometimes the variation feels clever or insightful, and other times it slips into nonsense. That’s the hidden switch: deterministic versus non-deterministic behavior. In deterministic systems, think spreadsheets or rule-driven formulas, the result never shifts. Type in 7 on Monday or Saturday, and the machine delivers the same verdict, free of mood swings or creativity. It’s mechanical loyalty, playing back the same move over and over. Non-deterministic models live differently. You hand them a prompt, and instead of marching down a fixed path, they sample across possibilities. (That sampling, plus stochastic processes, model updates, and even data drift, is what makes outputs vary.) It’s like setting a stage for improv—you write the scene, but the performer invents the punchline on the fly. Sometimes it works beautifully. Sometimes it strays into incoherence. Classic automation and rule-based workflows—like many built in Power Platform—live closer to the deterministic side. You set a condition, and when the trigger fires, it executes the defined rule with machine precision. That predictability is what keeps compliance, data flows, and audit trails stable. You know what will happen, because the steps are locked in. Generative copilots, by contrast, turn any input into an open space for interpretation. They’ll summarize, recombine, and rephrase in ways that often feel humanlike. Fluidity is the charm, but it’s also the risk, because that very fluidity permits unpredictability in contexts that require consistency. Picture an improv troupe on stage. You hand them the theme “budget approval.” One actor runs with a clever gag about saving, another veers into a subplot about banquets, and suddenly the show bears little resemblance to your original request. That’s a non-deterministic model mid-performance. These swings aren’t signs of bad design; they’re built into how large language models generate language—exploring many paths, not just one. The catch is clear: creativity doesn’t always equal accuracy, and in business workflows, accuracy is often the only currency that counts. Now apply this to finance. Suppose your AI-powered credit check tool evaluates an applicant as “approved.” Same information entered again the next day, but this time it says “rejected.” The applicant feels whiplash. The regulator sees inconsistency that smells like discrimination. What’s happening is drift: the outputs shift without a transparent reason, because non-deterministic systems can vary over time. Unlike human staff, you can’t simply ask the model to explain what changed. And this is where trust erodes fastest—when the reasoning vanishes behind opaque output. In production, drift amplifies quickly. A workflow approved to reduce bias one month may veer the opposite direction the next. Variations that seem minor in isolation add up to breaches when magnified across hundreds of cases. Regulators, unlike amused audiences at improv night, demand stability, auditability, and clear explanations. They don’t accept “non-determinism is part of the charm.” This is why guardrails matter. Regulators and standards ask for auditability, model documentation, and monitoring—so build logs and explainability measures into the deployment. Without them, even small shifts become liabilities: financial penalties stack up, reputational damage spreads, and customer trust dissolves. Governance is the human referee in this unpredictable play. Imagine those improvisers again, spinning in every direction. If nobody sets boundaries, the act collapses under its own chaos. A referee, though, keeps them tethered: “stay with this theme, follow this arc.” Governance works the same way for AI. It doesn’t snuff out innovation; it converts randomness into performance that still respects the script. Non-determinism remains, but it operates inside defined lanes. Here lies the balance. You can’t force a copilot to behave like a calculator—it isn’t built to. But you can put safety nets around it. Human oversight, monitoring systems, and governance frameworks act as that net. With them, the model still improvises, but it won’t wreck the show. Without them, drift cascades unchecked, and compliance teams are left cleaning up decisions no one can justify. The stakes are obvious: unpredictability isn’t neutral. It shapes outcomes that affect loans, jobs, or healthcare. And when the outputs carry real-world weight, regulators step in. Which brings us to the next frontier: the looming arrival

Become a supporter of this podcast: https://www.spreaker.com/podcast/m365-fm-modern-work-security-and-productivity-with-microsoft-365--6704921/support.

If this clashes with how you’ve seen it play out, I’m always curious. I use LinkedIn for the back-and-forth.
...more
View all episodesView all episodes
Download on the App Store

M365.FM - Modern work, security, and productivity with Microsoft 365By Mirko Peters (Microsoft 365 consultant and trainer)