Perla Ni, the founder of GreatNonprofits.org took a risk and organized a Stanford student group called Statistics for Good to figure out if their platform was suffering from a courtesy bias. Sometimes we don’t get the complete truth when we ask people to give us feedback especially when we know we are being judged or if we think we might lose a free service in the future. GreatNonprofits.org aggregates real feedback from the people that nonprofits seek to help, the question is are these people just being nice when leaving reviews and exhibiting a courtesy bias?

Josh Woodward’s song “Tik Tik” was featured in this podcast. Find his ‘pay what you want’ music site: http://www.joshwoodward.com/

Episode 17

Speaker 1: This is “Using The Whole Whale,” a podcast that brings you stories of data and technology in the non-profit world. This is George Weiner, your host and the Chief Whaler of wholewhale.com, thank you for joining us.

So I went to the doctor the other week and just got a check up. Don’t worry, everything is fine. They went through the normal things of testing this and that, and he went through the you know, “Do you smoke?” I was like, “Absolutely not.” And he went through the, “Do you drink? How many drinks a week do you have?” And that’s where you do the like, the quick pause, do the mental math and then just like, take 30% off that number and give them that number. And so what happens there actually happens to a lot of us when we survey our audience and we try to find the true answers and get that honest feedback. And it’s a good thing, I have to say, that doctors are not in charge of doing the average census for how much Americans drink. Because I think it would be woefully under reported.

Welcome to episode 17 of Using The Whole Whale. We are talking with Perla, the founder and CEO of greatnonprofits.org. A Great organization that allows people to leave reviews of non-profits they interact with. And we also have with us on this cast, Xioao, a current student at Stanford and she runs and is part of the Stats for Good program there that uses stats to help non-profits understand their data. So we are going to be diving into an interesting bit of work they did, kind of related to what happens when you’re dealing with the observer effect. Often referred to as the Hawthorne effect. Where in the act of observation, right as an observer, impacts the person who you’re observing. And so this happens, you know, on the surveys like we mentioned, that you’re maybe trying to get honest feedback for.

One of the things and effects that this can cause is what is called a Courtesy Bias. Wherein the person that you are potentially offering a free service to may not say that this was terrible. They don’t want to be mean to someone that just gave them something for free. And it was something that was kind of weighing on Perla’s mind at greatnonprofits.org. Where they’re trying to aggr