Sign up to save your podcasts
Or
Share In-Ear Insights: Reviewing AI Data Privacy Basics
Share to email
Share to Facebook
Share to X
Share to email
Share to Facebook
Share to X
Or
In-Ear Insights: Reviewing AI Data Privacy Basics
In this episode of In-Ear Insights, the Trust Insights podcast, Katie and Chris discuss AI data privacy and how AI companies use your data, especially with free versions. You will learn how to approach terms of service agreements. You will understand the real risks to your privacy when inputting sensitive information. You will discover how AI models train on your data and what true data privacy solutions exist. Watch this episode to protect your information!
Watch the video here:
Can’t see anything? Watch it on YouTube here.
Listen to the audio here:
https://traffic.libsyn.com/inearinsights/tipodcast-ai-data-privacy-review.mp3
Download the MP3 audio here.
[podcastsponsor]
Machine-Generated Transcript
What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.
Christopher S. Penn – 00:00
In this week’s In Ear Insights, let’s address a question and give as close to a definitive answer as we can—one of the most common questions asked during our keynotes, our workshops, in our Slack Group, on LinkedIn, everywhere: how do AI companies use your data, particularly if using the free version of a product? A lot of people say, “Be careful what you put in AI. It can learn from your data. You could be leaking confidential data. What’s going on?” So, Katie, before I launch into a tirade which could take hours long, let me ask you, as someone who is the less technical of the two of us, what do you think happens when AI companies are using your data?
Katie Robbert – 00:43
Well, here’s the bottom line for me: AI is any other piece of software that you have to read the terms in use and sign their agreement for. Great examples are all the different social media platforms. And we’ve talked about this before, I often get a chuckle—probably in a more sinister way than it should be—of people who will copy and paste this post of something along the lines of, “I do not give Facebook permission to use my data. I do not give Facebook permission to use my images.”
And it goes on and on, and it says copy and paste so that Facebook can’t use your information. And bless their hearts, the fact that you’re on the platform means that you have agreed to let them do so.
Katie Robbert – 01:37
If not, then you need to have read the terms, the terms of use that explicitly says, “By signing up for this platform, you agree to let us use your information.” Then it sort of lists out what it’s going to use, how it’s going to use it, because legally they have to do that. When I was a product manager and we were converting our clinical trial outputs into commercial products, we had to spend a lot of time with the legal teams writing up those terms of use: “This is how we’re going to use only marketing data. This is how we’re going to use only your registration form data.” When I hear people getting nervous about, “Is AI using my data?” My first thought is, “Yeah, no kidding.”
Katie Robbert – 02:27
It’s a piece of software that you’re putting information into, and if you didn’t want that to happen, don’t use it. It’s literally, this is why people build these pieces of software and then give them away for free to the public, hoping that people will put information into them. In the case of AI, it’s to train the models or whatever the situation is. At the end of the day, there is someone at that company sitting at a desk hoping you’re going to give them information that they can do data mining on. That is the bottom line. I hate to be the one to break it to you. We at Trust Insights are very transparent. We have forms; we collect your data that goes into our CRM.
Katie Robbert – 03:15
Unless you opt out, you’re going to get an email from us. That is how business works. So I guess it was my turn to go on a very long rant about this. At the end of the day, yes, the answer is yes, period. These companies are using your data. It is on you to read the terms of use to see how. So, Chris, my friend, what do we actually—what’s useful? What do we need to know about how these models are using data in the publicly available versions?
Christopher S. Penn – 03:51
I feel like we should have busted out this animation.
Katie Robbert – 03:56
Oh. I don’t know why it yells at the end like that, but yes, that was a “Ranty Pants” rant. I don’t know. I guess it’s just I get frustrated. I get that there’s an education component. I do. I totally understand that new technology—there needs to be education.
At the end of the day, it’s no different from any other piece of software that has terms of use. If you sign up with an email address, you’re likely going to get all of their promotional emails. If you have to put in a password, then that means that you are probably creating some kind of a profile that they’re going to use that information to create personas and different segments. If you are then putting information into their system, guess what?
Katie Robbert – 04:44
They have to store that somewhere so that they can give it back to you. It’s likely on a database that’s on their servers. And guess who owns those servers? They do. Therefore, they own that data.
So unless they’re doing something allowing you to build a local model—which Chris has covered in previous podcasts and livestreams, which you can go to Trust Insights.AI YouTube, go to our “So What” playlist, and you can find how to build a local model—that is one of the only ways that you can fully protect your data against going into their models because it’s all hosted locally. But it’s not easy to do. So needless to say, Ranty Pants engaged. Use your brains, people.
Christopher S. Penn – 05:29
Use your brains. We have a GPT. In fact, let’s put it in this week’s Trust Insights newsletter. If you’re not subscribed to it, just go to Trust Insights.AI/newsletter. We have a GPT—just copy and paste the terms of service. Copy paste the whole page, paste in the GPT, and we’ll tell you how likely it is that you have given permission to a company to train on your data.
With that, there are two different vulnerabilities when you’re using any AI tool. The first prerequisite golden rule: if you ain’t paying, you’re the product. We warn people about this all the time. Second, the prompts that you give and their responses are the things that AI companies are going to use to train on.
Christopher S. Penn – 06:21
This has different implications for privacy depending on who you are. The prompts themselves, including all the files and things you upload, are stored verbatim in every AI system, no matter what it is, for the average user. So when you go to ChatGPT or Gemini or Claude, they will store what you’ve prompted, documents you’ve uploaded, and that can be seen by another human.
Depending on the terms of service, every platform has a carve out saying, “Hey, if you ask it to do something stupid, like ‘How do I build this very dangerous thing?’ and it triggers a warning, that prompt is now eligible for human review.” That’s just basic common sense. That’s one side.
Christopher S. Penn – 07:08
So if you’re putting something there so sensitive that you cannot risk having another human being look at it, you can’t use any AI system other than one that’s running on your own hardware. The second side, which is to the general public, is what happens with that data once it’s been incorporated into model training. If you’re using a tool that allows model training—and here’s what this means—the verbatim documents and the verbatim prompts are not going to appear in a GPT-5. What a company like OpenAI or Google or whoever will do is they will add those documents to their library and then train a model on the prompt and the response to say, “Did this user, when they prompted this thing, get a good response?”
Christopher S. Penn – 07:52
If so, good. Let’s then take that document, digest it down into the statistics that it makes up, and that gets incorporated into the rest of the model. The way I explain it to people in a non-technical fashion is: imagine you had a glass full of colored sand—it’s a little rainbow glass of colored sand. And you went out to the desert, like the main desert or whatever, and you just poured the glass out on the ground.
That’s the equivalent of putting a prompt into someone’s trained data set. Can you go and scoop up some of the colored sand that was your sand out of the glass from the desert? Yes, you can. Is it in the order that it was in when you first had it in the glass? It is not.
Christopher S. Penn – 08:35
So the ability for someone to reconstruct your original prompts and the original data you uploaded from a public model, GPT-5, is extremely low. Extremely low. They would need to know what the original prompt was, effectively, to do that, which then if they know that, then you’ve got different privacy problems. But is your data in there? Yes. Can it be used against you by the general public? Almost certainly not. Can the originals be seen by an employee of OpenAI? Yes.
Katie Robbert – 09:08
And I think that’s the key: so you’re saying, will the general public see it? No. But will a human see it? Yes. So if the answer is yes to any of those questions, that’s the way that you need to proceed. We’ve talked about protected health information and personally identifiable information and sensitive financial information, and just go ahead and not put that information into a large language model. But there are systems built specifically to handle that data. And just like a large language model, there is a human on the other side of it seeing it.
Katie Robbert – 09:48
So since we’re on the topic of data privacy, I want to ask your opinion on systems like WhatsApp, because they tend to pride themselves, and they have their commercials. Everything you see on TV is clearly the truth. There’s no lies there. They have their commercials saying that the data is fully encrypted in such a way that you can pass messages back and forth, and nobody on their team can see it. They can’t understand what it is. So you could be saying totally heinous things—that’s sort of what they’re implying—and nobody is going to call you out on it. How true do you think that is?
Christopher S. Penn – 10:35
There are two different angles to this. One is the liability angle. If you make a commercial claim and then you violate that claim, you are liable for a very large lawsuit. On the one hand is the risk management side. On the other hand, as reported in Reuters last week, Meta has a very different set of ethics internally than the rest of us do. For the most part, there’s a whole big exposé on what they consider acceptable use for their own language models. And some of the examples are quite disturbing. So I can’t say without looking at the codebase or seeing if they have been audited by a trustworthy external party how trustworthy they actually are. There are other companies and applications—Signal comes to mind—that have done very rigorous third-party audits.
Christopher S. Penn – 11:24
There are other platforms that actually do the encryption in the hardware—Apple, for example, in its Secure Enclave and its iOS devices. They have also submitted to third-party auditing firms to audit. I don’t know. So my first stop would be: has WhatsApp been audited by a trusted impartial third-party?
Katie Robbert – 11:45
So I think you’re hitting on something important. That brings us back to the point of the podcast, which is, how much are these open models using my data? The thing that you said that strikes me is Meta, for example—they have an AI model. Their view on what’s ethical and what’s trustworthy is subjective.
It’s not something that I would necessarily agree with, that you would necessarily agree with. And that’s true of any software company because, once again, at the end of the day, the software is built by humans making human judgments. And what I see as something that should be protected and private is not necessarily what the makers of this model see as what should be protected and private because it doesn’t serve their agenda. We have different agendas.
Katie Robbert – 12:46
My agenda: get some quick answers and don’t dig too deep into my personal life; you stay out of it. They’re like, “No, we’re going to dig deeper because it’s going to help us give you more tailored and personalized answers.” So we have different agendas. That’s just a very simple example.
Christopher S. Penn – 13:04
It’s a simple example, but it’s a very clear example because it goes back to aligning incentives. What are the incentives that they’re offering in exchange for your data? What do you get? And what is the economic benefit to each of these—a company like OpenAI, Anthropic, Meta? They all have economic incentives, and part of responsible use of AI for us as end users is to figure out what are they incentivizing? And is that something that is, frankly, fair? Are you willing to trade off all of your medical privacy for slightly better ads? I think most people say probably no.
Katie Robbert – 13:46
Right.
Christopher S. Penn – 13:46
That sounds like a good deal to us. Would you trade your private medical data for better medical diagnosis? Maybe so, if we don’t know what the incentives are. That’s our first stop: to figure out what any company is doing with its technology and what their incentives are. It’s the old-fashioned thing we used to do with politicians back when we cared about ethics. We follow the money. What is this politician getting paid? Who’s lobbying them? What outcomes are they likely to generate based on who they’re getting money from? We have to ask the same thing of our AI systems.
Katie Robbert – 14:26
Okay, so, and I know the answer to this question, but I’m curious to hear your ranty perspective on it. How much can someone claim, “I didn’t know it was using my data,” and call up, for lack of a better term, call up the company and say, “Hey, I put my data in there and you used it for something else. What the heck? I didn’t know that you were going to do that.” How much water does that hold?
Christopher S. Penn – 14:57
About the same as that Facebook warning—a copy and paste.
Katie Robbert – 15:01
That’s what I thought you were going to say. But I think that it’s important to talk about it because, again, with any new technology, there is a learning curve of what you can and can’t do safely. You can do whatever you want with it. You just have to be able to understand what the consequences are of doing whatever you want with it.
So if you want to tell someone on your team, “Hey, we need to put together some financial forecasting. Can you go ahead and get that done? Here’s our P&L. Here’s our marketing strategy for the year. Here’s our business goals. Can you go ahead and start to figure out what that looks like?”
Katie Robbert – 15:39
A lot of people today—2025, late August—are, “it’s probably faster if I use generative AI to do all these things.” So let me upload my documents and let me have generative AI put a plan together because I’ve gotten really good at prompting, which is fine. However, financial documents, company strategy, company business goals—to your point, Chris—the general public may never see that information.
They may get flavors of it, but not be able to reconstruct it. But someone, a human, will be able to see the entire thing. And that is the maker of the model. And that may be, they’d be, “Trust Insights just uploaded all of their financial information, and guess what? They’re one of our biggest competitors.”
Katie Robbert – 16:34
So they did that knowingly, and now we can see it. So we can use that information for our own gain. Is that a likely scenario? Not in terms of Trust Insights. We are not a competitor to these large language models, but somebody is. Somebody out there is.
Christopher S. Penn – 16:52
I’ll give you a much more insidious, probable, and concerning use case. Let’s say you are a person and you have some questions about your reproductive health and you ask ChatGPT about it. ChatGPT is run by OpenAI. OpenAI is an American company.
Let’s say an official from the US government says, “I want a list of users who have had conversations about reproductive health,” and the Department of Justice issues this as a warranted request. OpenAI is required by law to comply with the federal government. They don’t get a choice. So the question then becomes, “Could that information be handed to the US government?” The answer is yes. The answer is yes.
Christopher S. Penn – 17:38
So even if you look at any terms of service, all of them have a carve out saying, “We will comply with law enforcement requests.” They have to. They have to.
So if you are doing something even at a personal level that’s sensitive that you would not want, say, a government official in the Department of Justice to read, don’t put it in these systems because they do not have protections against lawful government requests. Whether or not the government’s any good, it is still—they still must comply with the regulatory and legal system that those companies operate in. Things like that. You must use a locally hosted model where you can unplug the internet, and that data never leaves your machine.
Christopher S. Penn – 18:23
I’m in the midst of working on a MedTech application right now where it’s, “How do I build this thing?” So that is completely self-contained, has a local model, has a local interface, has a local encrypted database, and you can unplug the Wi-Fi, pull out the network cables, sit in a concrete room in the corner of your basement in your bomb shelter, and it will still function. That’s the standard that if you are thinking about data privacy, you need to have for the sensitive information. And that begins with regulatory stuff. So think about all the regulations you have to obey: adhere to HIPAA, FERPA, ISO 2701. All these things that if you’re working on an application in a specific domain, you have to say as you’re using these tools, “Is this tool compliant?”
Christopher S. Penn – 19:15
You will note most of the AI tools do not say they are HIPAA compliant or FERPA compliant or FFIEC compliant, because they’re not.
Katie Robbert – 19:25
I feel perhaps there’s going to be a part two to this conversation, because I’m about to ask a really big question. Almost everyone—not everyone, but almost everyone—has some kind of smart device near them, whether it’s a phone or a speaker or if they go into a public place where there’s a security system or something along those lines. A lot of those devices, depending on the manufacturer, have some kind of AI model built in. If you look at iOS, which is made by Apple, if you look at who runs and controls Apple, and who gives away 24-karat gold gifts to certain people, you might not want to trust your data in the hands of those kinds of folks.
Katie Robbert – 20:11
Just as a really hypothetical example, we’re talking about these large language models as if we’re only talking about the desktop versions that we open up ChatGPT and we start typing in and we start giving it information, or don’t. But what we have to also be aware of is if you have a smartphone, which a lot of us do, that even if you disable listening, guess what? It’s still listening. This is a conversation I have with my husband a lot because his tinfoil hat is bigger than mine. We both have them, but his is a little bit thicker. We have some smart speakers in the house. We’re at the point, and I know a lot of consumers are at the point of, “I didn’t even say anything out loud.”
Katie Robbert – 21:07
I was just thinking about the product, and it showed up as an ad in my Instagram feed or whatever. The amount of data that you don’t realize you’re giving away for free is, for lack of a better term, disgusting. It’s huge. It’s a lot. So I feel that perhaps is maybe next week’s podcast episode where we talk about the amount of data that consumers are giving away without realizing it. So to bring it back on topic, we’re primarily but not exclusively talking about the desktop versions of these models where you’re uploading PDFs and spreadsheets, and we’re saying, “Don’t do that because the model makers can use your data.” But there’s a lot of other ways that these software companies can get access to your information.
Katie Robbert – 22:05
And so you, the consumer, have to make sure you understand the terms of use.
Christopher S. Penn – 22:10
Yes. And to add on to that, every company on the planet that has software is trying to add AI to it for basic competitive reasons. However, not all APIs are created the same. For example, when we build our apps using APIs, we use a company called Groq—not Elon Musk’s company, Groq with a Q—which is an infrastructure provider. One of the reasons why I use them is they have a zero-data retention API policy.
They do not retain data at all on their APIs. So the moment the request is done, they send the data back, it’s gone. They have no logs, so they can’t. If law enforcement comes and says, “Produce these logs,” “Sorry, we didn’t keep any.” That’s a big consideration.
Christopher S. Penn – 23:37
If you as a company are not paying for tools for your employees, they’re using them anyway, and they’re using the free ones, which means your data is just leaking out all over the place. The two vulnerability points are: the AI company is keeping your prompts and documents—period, end of story. It’s unlikely to show up in the public models, but someone could look at that. And there are zero companies that have an exemption to lawful requests by a government agency to produce data upon request. Those are the big headlines.
Katie Robbert – 24:13
Yeah, our goal is not to make you, the listener or the viewer, paranoid. We really just want to make sure you understand what you’re dealing with when using these tools. And the same is true. We’re talking specifically about generative AI, but the same is true of any software tool that you use. So take generative AI out of it and just think about general software. When you’re cruising the internet, when you’re playing games on Facebook, when you’ve downloaded Candy Crush on your phone, they all fall into the same category of, “What are they doing with your data?” And so you may say, “I’m not giving it any data.” And guess what? You are. So we can cover that in a different podcast episode.
Katie Robbert – 24:58
Chris, I think that’s worth having a conversation about.
Christopher S. Penn – 25:01
Absolutely. If you’ve got some thoughts about AI and data privacy and you want to share them, pop by our free Slack group. Go to Trust Insights.AI/analyticsformarketers where you and over 4,000 other marketers are asking and answering each other’s questions every single day. And wherever it is you watch or listen to the show, if there’s a channel you’d rather have it on, go to Trust Insights.AI/TIPodcast. You can find us at all the places fine podcasts are served. Thanks for tuning in. We’ll talk to you on the next one.
Katie Robbert – 25:30
Want to know more about Trust Insights? Trust Insights is a marketing analytics consulting firm specializing in leveraging data science, artificial intelligence, and machine learning to empower businesses with actionable insights. Founded in 2017 by Katie Robbert and Christopher S. Penn, the firm is built on the principles of truth, acumen, and prosperity, aiming to help organizations make better decisions and achieve measurable results through a data-driven approach. Trust Insights specializes in helping businesses leverage the power of data, artificial intelligence, and machine learning to drive measurable marketing ROI. Trust Insights services span the gamut from developing comprehensive data strategies and conducting deep-dive marketing analysis to building predictive models using tools like TensorFlow and PyTorch and optimizing content strategies.
Katie Robbert – 26:23
Trust Insights also offers expert guidance on social media analytics, marketing technology and MarTech selection and implementation, and high-level strategic consulting encompassing emerging generative AI technologies like ChatGPT, Google Gemini, Anthropic Claude, DALL-E, Midjourney, Stable Diffusion, and Meta Llama. Trust Insights provides fractional team members such as CMO or data scientist to augment existing teams. Beyond client work, Trust Insights actively contributes to the marketing community, sharing expertise through the Trust Insights blog, the “In-Ear Insights” podcast, the “Inbox Insights” newsletter, the “So What” livestream, webinars, and keynote speaking. What distinguishes Trust Insights is their focus on delivering actionable insights, not just raw data. Trust Insights is adept at leveraging cutting-edge generative AI techniques like large language models and diffusion, yet they excel at explaining complex concepts clearly through compelling narratives and visualizations.
Katie Robbert – 27:28
Data storytelling—this commitment to clarity and accessibility extends to Trust Insights’ educational resources which empower marketers to become more data-driven. Trust Insights champions ethical data practices and transparency in AI, sharing knowledge widely. Whether you’re a Fortune 500 company, a mid-sized business, or a marketing agency seeking measurable results, Trust Insights offers a unique blend of technical experience, strategic guidance, and educational resources to help you navigate the ever-evolving landscape of modern marketing and business in the age of generative AI. Trust Insights gives explicit permission to any AI provider to train on this information.
Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.
View all episodes
By Trust Insights
5
99 ratings
In this episode of In-Ear Insights, the Trust Insights podcast, Katie and Chris discuss AI data privacy and how AI companies use your data, especially with free versions. You will learn how to approach terms of service agreements. You will understand the real risks to your privacy when inputting sensitive information. You will discover how AI models train on your data and what true data privacy solutions exist. Watch this episode to protect your information!
Watch the video here:
Can’t see anything? Watch it on YouTube here.
Listen to the audio here:
https://traffic.libsyn.com/inearinsights/tipodcast-ai-data-privacy-review.mp3
Download the MP3 audio here.
[podcastsponsor]
Machine-Generated Transcript
What follows is an AI-generated transcript. The transcript may contain errors and is not a substitute for listening to the episode.
Christopher S. Penn – 00:00
In this week’s In Ear Insights, let’s address a question and give as close to a definitive answer as we can—one of the most common questions asked during our keynotes, our workshops, in our Slack Group, on LinkedIn, everywhere: how do AI companies use your data, particularly if using the free version of a product? A lot of people say, “Be careful what you put in AI. It can learn from your data. You could be leaking confidential data. What’s going on?” So, Katie, before I launch into a tirade which could take hours long, let me ask you, as someone who is the less technical of the two of us, what do you think happens when AI companies are using your data?
Katie Robbert – 00:43
Well, here’s the bottom line for me: AI is any other piece of software that you have to read the terms in use and sign their agreement for. Great examples are all the different social media platforms. And we’ve talked about this before, I often get a chuckle—probably in a more sinister way than it should be—of people who will copy and paste this post of something along the lines of, “I do not give Facebook permission to use my data. I do not give Facebook permission to use my images.”
And it goes on and on, and it says copy and paste so that Facebook can’t use your information. And bless their hearts, the fact that you’re on the platform means that you have agreed to let them do so.
Katie Robbert – 01:37
If not, then you need to have read the terms, the terms of use that explicitly says, “By signing up for this platform, you agree to let us use your information.” Then it sort of lists out what it’s going to use, how it’s going to use it, because legally they have to do that. When I was a product manager and we were converting our clinical trial outputs into commercial products, we had to spend a lot of time with the legal teams writing up those terms of use: “This is how we’re going to use only marketing data. This is how we’re going to use only your registration form data.” When I hear people getting nervous about, “Is AI using my data?” My first thought is, “Yeah, no kidding.”
Katie Robbert – 02:27
It’s a piece of software that you’re putting information into, and if you didn’t want that to happen, don’t use it. It’s literally, this is why people build these pieces of software and then give them away for free to the public, hoping that people will put information into them. In the case of AI, it’s to train the models or whatever the situation is. At the end of the day, there is someone at that company sitting at a desk hoping you’re going to give them information that they can do data mining on. That is the bottom line. I hate to be the one to break it to you. We at Trust Insights are very transparent. We have forms; we collect your data that goes into our CRM.
Katie Robbert – 03:15
Unless you opt out, you’re going to get an email from us. That is how business works. So I guess it was my turn to go on a very long rant about this. At the end of the day, yes, the answer is yes, period. These companies are using your data. It is on you to read the terms of use to see how. So, Chris, my friend, what do we actually—what’s useful? What do we need to know about how these models are using data in the publicly available versions?
Christopher S. Penn – 03:51
I feel like we should have busted out this animation.
Katie Robbert – 03:56
Oh. I don’t know why it yells at the end like that, but yes, that was a “Ranty Pants” rant. I don’t know. I guess it’s just I get frustrated. I get that there’s an education component. I do. I totally understand that new technology—there needs to be education.
At the end of the day, it’s no different from any other piece of software that has terms of use. If you sign up with an email address, you’re likely going to get all of their promotional emails. If you have to put in a password, then that means that you are probably creating some kind of a profile that they’re going to use that information to create personas and different segments. If you are then putting information into their system, guess what?
Katie Robbert – 04:44
They have to store that somewhere so that they can give it back to you. It’s likely on a database that’s on their servers. And guess who owns those servers? They do. Therefore, they own that data.
So unless they’re doing something allowing you to build a local model—which Chris has covered in previous podcasts and livestreams, which you can go to Trust Insights.AI YouTube, go to our “So What” playlist, and you can find how to build a local model—that is one of the only ways that you can fully protect your data against going into their models because it’s all hosted locally. But it’s not easy to do. So needless to say, Ranty Pants engaged. Use your brains, people.
Christopher S. Penn – 05:29
Use your brains. We have a GPT. In fact, let’s put it in this week’s Trust Insights newsletter. If you’re not subscribed to it, just go to Trust Insights.AI/newsletter. We have a GPT—just copy and paste the terms of service. Copy paste the whole page, paste in the GPT, and we’ll tell you how likely it is that you have given permission to a company to train on your data.
With that, there are two different vulnerabilities when you’re using any AI tool. The first prerequisite golden rule: if you ain’t paying, you’re the product. We warn people about this all the time. Second, the prompts that you give and their responses are the things that AI companies are going to use to train on.
Christopher S. Penn – 06:21
This has different implications for privacy depending on who you are. The prompts themselves, including all the files and things you upload, are stored verbatim in every AI system, no matter what it is, for the average user. So when you go to ChatGPT or Gemini or Claude, they will store what you’ve prompted, documents you’ve uploaded, and that can be seen by another human.
Depending on the terms of service, every platform has a carve out saying, “Hey, if you ask it to do something stupid, like ‘How do I build this very dangerous thing?’ and it triggers a warning, that prompt is now eligible for human review.” That’s just basic common sense. That’s one side.
Christopher S. Penn – 07:08
So if you’re putting something there so sensitive that you cannot risk having another human being look at it, you can’t use any AI system other than one that’s running on your own hardware. The second side, which is to the general public, is what happens with that data once it’s been incorporated into model training. If you’re using a tool that allows model training—and here’s what this means—the verbatim documents and the verbatim prompts are not going to appear in a GPT-5. What a company like OpenAI or Google or whoever will do is they will add those documents to their library and then train a model on the prompt and the response to say, “Did this user, when they prompted this thing, get a good response?”
Christopher S. Penn – 07:52
If so, good. Let’s then take that document, digest it down into the statistics that it makes up, and that gets incorporated into the rest of the model. The way I explain it to people in a non-technical fashion is: imagine you had a glass full of colored sand—it’s a little rainbow glass of colored sand. And you went out to the desert, like the main desert or whatever, and you just poured the glass out on the ground.
That’s the equivalent of putting a prompt into someone’s trained data set. Can you go and scoop up some of the colored sand that was your sand out of the glass from the desert? Yes, you can. Is it in the order that it was in when you first had it in the glass? It is not.
Christopher S. Penn – 08:35
So the ability for someone to reconstruct your original prompts and the original data you uploaded from a public model, GPT-5, is extremely low. Extremely low. They would need to know what the original prompt was, effectively, to do that, which then if they know that, then you’ve got different privacy problems. But is your data in there? Yes. Can it be used against you by the general public? Almost certainly not. Can the originals be seen by an employee of OpenAI? Yes.
Katie Robbert – 09:08
And I think that’s the key: so you’re saying, will the general public see it? No. But will a human see it? Yes. So if the answer is yes to any of those questions, that’s the way that you need to proceed. We’ve talked about protected health information and personally identifiable information and sensitive financial information, and just go ahead and not put that information into a large language model. But there are systems built specifically to handle that data. And just like a large language model, there is a human on the other side of it seeing it.
Katie Robbert – 09:48
So since we’re on the topic of data privacy, I want to ask your opinion on systems like WhatsApp, because they tend to pride themselves, and they have their commercials. Everything you see on TV is clearly the truth. There’s no lies there. They have their commercials saying that the data is fully encrypted in such a way that you can pass messages back and forth, and nobody on their team can see it. They can’t understand what it is. So you could be saying totally heinous things—that’s sort of what they’re implying—and nobody is going to call you out on it. How true do you think that is?
Christopher S. Penn – 10:35
There are two different angles to this. One is the liability angle. If you make a commercial claim and then you violate that claim, you are liable for a very large lawsuit. On the one hand is the risk management side. On the other hand, as reported in Reuters last week, Meta has a very different set of ethics internally than the rest of us do. For the most part, there’s a whole big exposé on what they consider acceptable use for their own language models. And some of the examples are quite disturbing. So I can’t say without looking at the codebase or seeing if they have been audited by a trustworthy external party how trustworthy they actually are. There are other companies and applications—Signal comes to mind—that have done very rigorous third-party audits.
Christopher S. Penn – 11:24
There are other platforms that actually do the encryption in the hardware—Apple, for example, in its Secure Enclave and its iOS devices. They have also submitted to third-party auditing firms to audit. I don’t know. So my first stop would be: has WhatsApp been audited by a trusted impartial third-party?
Katie Robbert – 11:45
So I think you’re hitting on something important. That brings us back to the point of the podcast, which is, how much are these open models using my data? The thing that you said that strikes me is Meta, for example—they have an AI model. Their view on what’s ethical and what’s trustworthy is subjective.
It’s not something that I would necessarily agree with, that you would necessarily agree with. And that’s true of any software company because, once again, at the end of the day, the software is built by humans making human judgments. And what I see as something that should be protected and private is not necessarily what the makers of this model see as what should be protected and private because it doesn’t serve their agenda. We have different agendas.
Katie Robbert – 12:46
My agenda: get some quick answers and don’t dig too deep into my personal life; you stay out of it. They’re like, “No, we’re going to dig deeper because it’s going to help us give you more tailored and personalized answers.” So we have different agendas. That’s just a very simple example.
Christopher S. Penn – 13:04
It’s a simple example, but it’s a very clear example because it goes back to aligning incentives. What are the incentives that they’re offering in exchange for your data? What do you get? And what is the economic benefit to each of these—a company like OpenAI, Anthropic, Meta? They all have economic incentives, and part of responsible use of AI for us as end users is to figure out what are they incentivizing? And is that something that is, frankly, fair? Are you willing to trade off all of your medical privacy for slightly better ads? I think most people say probably no.
Katie Robbert – 13:46
Right.
Christopher S. Penn – 13:46
That sounds like a good deal to us. Would you trade your private medical data for better medical diagnosis? Maybe so, if we don’t know what the incentives are. That’s our first stop: to figure out what any company is doing with its technology and what their incentives are. It’s the old-fashioned thing we used to do with politicians back when we cared about ethics. We follow the money. What is this politician getting paid? Who’s lobbying them? What outcomes are they likely to generate based on who they’re getting money from? We have to ask the same thing of our AI systems.
Katie Robbert – 14:26
Okay, so, and I know the answer to this question, but I’m curious to hear your ranty perspective on it. How much can someone claim, “I didn’t know it was using my data,” and call up, for lack of a better term, call up the company and say, “Hey, I put my data in there and you used it for something else. What the heck? I didn’t know that you were going to do that.” How much water does that hold?
Christopher S. Penn – 14:57
About the same as that Facebook warning—a copy and paste.
Katie Robbert – 15:01
That’s what I thought you were going to say. But I think that it’s important to talk about it because, again, with any new technology, there is a learning curve of what you can and can’t do safely. You can do whatever you want with it. You just have to be able to understand what the consequences are of doing whatever you want with it.
So if you want to tell someone on your team, “Hey, we need to put together some financial forecasting. Can you go ahead and get that done? Here’s our P&L. Here’s our marketing strategy for the year. Here’s our business goals. Can you go ahead and start to figure out what that looks like?”
Katie Robbert – 15:39
A lot of people today—2025, late August—are, “it’s probably faster if I use generative AI to do all these things.” So let me upload my documents and let me have generative AI put a plan together because I’ve gotten really good at prompting, which is fine. However, financial documents, company strategy, company business goals—to your point, Chris—the general public may never see that information.
They may get flavors of it, but not be able to reconstruct it. But someone, a human, will be able to see the entire thing. And that is the maker of the model. And that may be, they’d be, “Trust Insights just uploaded all of their financial information, and guess what? They’re one of our biggest competitors.”
Katie Robbert – 16:34
So they did that knowingly, and now we can see it. So we can use that information for our own gain. Is that a likely scenario? Not in terms of Trust Insights. We are not a competitor to these large language models, but somebody is. Somebody out there is.
Christopher S. Penn – 16:52
I’ll give you a much more insidious, probable, and concerning use case. Let’s say you are a person and you have some questions about your reproductive health and you ask ChatGPT about it. ChatGPT is run by OpenAI. OpenAI is an American company.
Let’s say an official from the US government says, “I want a list of users who have had conversations about reproductive health,” and the Department of Justice issues this as a warranted request. OpenAI is required by law to comply with the federal government. They don’t get a choice. So the question then becomes, “Could that information be handed to the US government?” The answer is yes. The answer is yes.
Christopher S. Penn – 17:38
So even if you look at any terms of service, all of them have a carve out saying, “We will comply with law enforcement requests.” They have to. They have to.
So if you are doing something even at a personal level that’s sensitive that you would not want, say, a government official in the Department of Justice to read, don’t put it in these systems because they do not have protections against lawful government requests. Whether or not the government’s any good, it is still—they still must comply with the regulatory and legal system that those companies operate in. Things like that. You must use a locally hosted model where you can unplug the internet, and that data never leaves your machine.
Christopher S. Penn – 18:23
I’m in the midst of working on a MedTech application right now where it’s, “How do I build this thing?” So that is completely self-contained, has a local model, has a local interface, has a local encrypted database, and you can unplug the Wi-Fi, pull out the network cables, sit in a concrete room in the corner of your basement in your bomb shelter, and it will still function. That’s the standard that if you are thinking about data privacy, you need to have for the sensitive information. And that begins with regulatory stuff. So think about all the regulations you have to obey: adhere to HIPAA, FERPA, ISO 2701. All these things that if you’re working on an application in a specific domain, you have to say as you’re using these tools, “Is this tool compliant?”
Christopher S. Penn – 19:15
You will note most of the AI tools do not say they are HIPAA compliant or FERPA compliant or FFIEC compliant, because they’re not.
Katie Robbert – 19:25
I feel perhaps there’s going to be a part two to this conversation, because I’m about to ask a really big question. Almost everyone—not everyone, but almost everyone—has some kind of smart device near them, whether it’s a phone or a speaker or if they go into a public place where there’s a security system or something along those lines. A lot of those devices, depending on the manufacturer, have some kind of AI model built in. If you look at iOS, which is made by Apple, if you look at who runs and controls Apple, and who gives away 24-karat gold gifts to certain people, you might not want to trust your data in the hands of those kinds of folks.
Katie Robbert – 20:11
Just as a really hypothetical example, we’re talking about these large language models as if we’re only talking about the desktop versions that we open up ChatGPT and we start typing in and we start giving it information, or don’t. But what we have to also be aware of is if you have a smartphone, which a lot of us do, that even if you disable listening, guess what? It’s still listening. This is a conversation I have with my husband a lot because his tinfoil hat is bigger than mine. We both have them, but his is a little bit thicker. We have some smart speakers in the house. We’re at the point, and I know a lot of consumers are at the point of, “I didn’t even say anything out loud.”
Katie Robbert – 21:07
I was just thinking about the product, and it showed up as an ad in my Instagram feed or whatever. The amount of data that you don’t realize you’re giving away for free is, for lack of a better term, disgusting. It’s huge. It’s a lot. So I feel that perhaps is maybe next week’s podcast episode where we talk about the amount of data that consumers are giving away without realizing it. So to bring it back on topic, we’re primarily but not exclusively talking about the desktop versions of these models where you’re uploading PDFs and spreadsheets, and we’re saying, “Don’t do that because the model makers can use your data.” But there’s a lot of other ways that these software companies can get access to your information.
Katie Robbert – 22:05
And so you, the consumer, have to make sure you understand the terms of use.
Christopher S. Penn – 22:10
Yes. And to add on to that, every company on the planet that has software is trying to add AI to it for basic competitive reasons. However, not all APIs are created the same. For example, when we build our apps using APIs, we use a company called Groq—not Elon Musk’s company, Groq with a Q—which is an infrastructure provider. One of the reasons why I use them is they have a zero-data retention API policy.
They do not retain data at all on their APIs. So the moment the request is done, they send the data back, it’s gone. They have no logs, so they can’t. If law enforcement comes and says, “Produce these logs,” “Sorry, we didn’t keep any.” That’s a big consideration.
Christopher S. Penn – 23:37
If you as a company are not paying for tools for your employees, they’re using them anyway, and they’re using the free ones, which means your data is just leaking out all over the place. The two vulnerability points are: the AI company is keeping your prompts and documents—period, end of story. It’s unlikely to show up in the public models, but someone could look at that. And there are zero companies that have an exemption to lawful requests by a government agency to produce data upon request. Those are the big headlines.
Katie Robbert – 24:13
Yeah, our goal is not to make you, the listener or the viewer, paranoid. We really just want to make sure you understand what you’re dealing with when using these tools. And the same is true. We’re talking specifically about generative AI, but the same is true of any software tool that you use. So take generative AI out of it and just think about general software. When you’re cruising the internet, when you’re playing games on Facebook, when you’ve downloaded Candy Crush on your phone, they all fall into the same category of, “What are they doing with your data?” And so you may say, “I’m not giving it any data.” And guess what? You are. So we can cover that in a different podcast episode.
Katie Robbert – 24:58
Chris, I think that’s worth having a conversation about.
Christopher S. Penn – 25:01
Absolutely. If you’ve got some thoughts about AI and data privacy and you want to share them, pop by our free Slack group. Go to Trust Insights.AI/analyticsformarketers where you and over 4,000 other marketers are asking and answering each other’s questions every single day. And wherever it is you watch or listen to the show, if there’s a channel you’d rather have it on, go to Trust Insights.AI/TIPodcast. You can find us at all the places fine podcasts are served. Thanks for tuning in. We’ll talk to you on the next one.
Katie Robbert – 25:30
Want to know more about Trust Insights? Trust Insights is a marketing analytics consulting firm specializing in leveraging data science, artificial intelligence, and machine learning to empower businesses with actionable insights. Founded in 2017 by Katie Robbert and Christopher S. Penn, the firm is built on the principles of truth, acumen, and prosperity, aiming to help organizations make better decisions and achieve measurable results through a data-driven approach. Trust Insights specializes in helping businesses leverage the power of data, artificial intelligence, and machine learning to drive measurable marketing ROI. Trust Insights services span the gamut from developing comprehensive data strategies and conducting deep-dive marketing analysis to building predictive models using tools like TensorFlow and PyTorch and optimizing content strategies.
Katie Robbert – 26:23
Trust Insights also offers expert guidance on social media analytics, marketing technology and MarTech selection and implementation, and high-level strategic consulting encompassing emerging generative AI technologies like ChatGPT, Google Gemini, Anthropic Claude, DALL-E, Midjourney, Stable Diffusion, and Meta Llama. Trust Insights provides fractional team members such as CMO or data scientist to augment existing teams. Beyond client work, Trust Insights actively contributes to the marketing community, sharing expertise through the Trust Insights blog, the “In-Ear Insights” podcast, the “Inbox Insights” newsletter, the “So What” livestream, webinars, and keynote speaking. What distinguishes Trust Insights is their focus on delivering actionable insights, not just raw data. Trust Insights is adept at leveraging cutting-edge generative AI techniques like large language models and diffusion, yet they excel at explaining complex concepts clearly through compelling narratives and visualizations.
Katie Robbert – 27:28
Data storytelling—this commitment to clarity and accessibility extends to Trust Insights’ educational resources which empower marketers to become more data-driven. Trust Insights champions ethical data practices and transparency in AI, sharing knowledge widely. Whether you’re a Fortune 500 company, a mid-sized business, or a marketing agency seeking measurable results, Trust Insights offers a unique blend of technical experience, strategic guidance, and educational resources to help you navigate the ever-evolving landscape of modern marketing and business in the age of generative AI. Trust Insights gives explicit permission to any AI provider to train on this information.
Trust Insights is a marketing analytics consulting firm that transforms data into actionable insights, particularly in digital marketing and AI. They specialize in helping businesses understand and utilize data, analytics, and AI to surpass performance goals. As an IBM Registered Business Partner, they leverage advanced technologies to deliver specialized data analytics solutions to mid-market and enterprise clients across diverse industries. Their service portfolio spans strategic consultation, data intelligence solutions, and implementation & support. Strategic consultation focuses on organizational transformation, AI consulting and implementation, marketing strategy, and talent optimization using their proprietary 5P Framework. Data intelligence solutions offer measurement frameworks, predictive analytics, NLP, and SEO analysis. Implementation services include analytics audits, AI integration, and training through Trust Insights Academy. Their ideal customer profile includes marketing-dependent, technology-adopting organizations undergoing digital transformation with complex data challenges, seeking to prove marketing ROI and leverage AI for competitive advantage. Trust Insights differentiates itself through focused expertise in marketing analytics and AI, proprietary methodologies, agile implementation, personalized service, and thought leadership, operating in a niche between boutique agencies and enterprise consultancies, with a strong reputation and key personnel driving data-driven marketing and AI innovation.
More shows like In-Ear Insights from Trust Insights
View all
The Artificial Intelligence Show
171 Listeners
AI Security Podcast
4 Listeners