Shoshana Grossbard, Economist, Professor, Editor, Becker’s Student

I recently volunteered to teach a new class on the history of economic thought and it has been profoundly rewarding for me. I love economics but I also love the stories of its players, my tribe, the economists. This person critiquing that person, this idea twisting around that other persons idea. The idea I might get paid to study people like Adam Smith and Thomas Malthus? Pinch me — I must be dreaming.

What’s been interesting to me is how much I recognize. Even though I have never read Malthus theory of gluts before, it’s somehow eerily familiar. But it’s more than just seeing the traces of later theories in these early writers. Reading history is also helping reframe the broad story of economics itself — the story of capitalism, its tensions, the conflicts between people, the appropriation and division of surplus, and the organization accomplished through markets and exchange. Reading history helps makes sense of me and others when I see the full sweep of the times and the debates.

But some people are remembered, some people aren’t. Some ideas were dropped and some hung around. Sometimes they were omitted because the theories while useful and intriguing explanations then were not as useful as another explanation that would replace it. That process of what we remember and what we forget happens at all levels, big and small, and it too is part of the story of economics in a way.

This idea that now long gone economists were once in their offices working earnestly on ideas we have collectively chosen to ignore and forget is not itself tragic, though. Nor is what was selected to persist glorious. Both simply are. Most of the things I have chosen or done in fact no one will ever know. Most of the words I have said, no one was present to hear. History has forgotten more than it has remembered.

Nevertheless I want to know. I want to learn all the stories, all the people, all the ideas, all the ways that ideas are forged and changed and kept and passed around. I know that it sounds a bit dramatic to say “My people”. What a strange way to self identify. And yet it is genuine. I am an economist. It will say in my tombstone that I was an economist even. I am more an economist than I am almost any other thing. So perhaps that’s why I care about the stories of the economists — the people and the ideas. I care about the people too. I care especially about the stories that for one reason or another would simply never be told were we not to ask and listen with open mind and accepting hearts, the hallmark of curiosity.

I see my podcast as a way to collect the stories of people whose stories don’t show up in the footnotes of our textbooks. I do it around topics I care about like causal inference, economists in tech, and public policy. And one of the series I have been doing I call “Becker’s students”. I chose Gary Becker because it was Becker’s Nobel prize speech more than any other intellectual experience I had that prompted me to get a PhD in economics. He has cast a massive shadow over me. And my series so far has included interviews with two of his former students from when he was a professor at Columbia University: Robert Michael and Michael Grossman.

But this week I am talking with one of his students from the University of Chicago where he spent the majority of his career. My guest this week is Shoshana Grossbard, professor of economics at San Diego State University, and editor of Review of Economics of the Household.

This interview was one of the best interview experiences I have had yet. Shoshana was honest, warm, and most of all very candid about her career, about the history of household economics, the things that had major impacts on her, but also the discouragements in her career. That she would share both the highs and lows as well as her thoughts about economics as a science and its practitioners so transparently with me of all people was deeply humbling for me.

You will learn that Becker was an important figure for her, not surprisingly as he was her advisor, but like many important people to us who we’ve known for years, the relationship was also a complicated one. His influence cast a long shadow over who she chose to become, and I appreciated that she was so forthright. I hope you like this interview too. Please tell others about it! Don’t forget to subscribe and if you like it, consider supporting it!

Scott's Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

From the earliest days of economics as a formal science, economists have been trying to understand the causes of the wealth of nations. In the field of development economics, the introduction of the randomized controlled trial has become an important tool in the broader toolbox of economists for trying to understand the many things that may cause the improvement of human welfare in lower income countries, too. This work has been ground breaking and recognized by the Nobel committee for its lasting importance. Like many others, I have been inspired by the development economists relentless effort to address poverty through rigorous causal inference and program evaluation.

While most people know that randomization is an important ingredient in causal inference, what is not as widely known is the Stable Unit Treatment Value Assumption, or SUTVA for short. We know that randomized treatments can eliminate selection bias and thus allow us to use realized outcomes in place of potential outcome. SUTVA requires that the potential outcomes themselves be stable and unchanging when treatment assignments of other units change, which brings to mind complex problems with externalities between people and interference when designing experiments, but as Imbens and Rubin note in their 2015 book, SUTVA also requires that treatments not vary unknowingly across units. Such “hidden variation in treatment” can make causal interpretation difficult if not possible.

But SUTVA also brings to mind the problems of external validity. When someone reading of a study’s large gains in a field experiment, they might then decide to roll out such a program at large scale. Assume for simplicity constant treatment effects for a moment — can they expect the same thing to happen in their community what happened in this particular trial? They can insofar as they do not inadvertently change the treatment itself by unknowingly varying the inputs in important ways. If the RCT found large literacy gains using a particular type worker, but to bring it to scale, the policymaker foregoes using those same inputs, then it becomes a new empirical question as to whether the effect found in the lab will in fact scale, even with constant treatment effects. But we have known this for centuries. Concepts like production functions and cost functions directly speak to these very things.

Successful policy requires evidence but also a set of skills that maybe aren’t there when designing or evaluating an RCT. It requires both cognitive skill, and perhaps even moreso non cognitive skill, as often the economist then must wear both a scientist hat, a manager hat, and an entrepreneur hat. And not every economist has those skills, or maybe even is interested in venturing into the messy world of building socially impactful policy. But I have also been inspired by a small group of applied development economists like Noam Angrist, Co-founder and Executive Director of Youth Impact (formerly Young 1ove), and Paul Niehaus at GiveDirectly, who create organizations that try to bring effective programs to larger scale while simultaneously committing themselves to constant evaluation of themselves and their programs. Whether it’s a trend or not, I don’t know, but I have been intrigued.

This week on The Mixtape with Scott, I have the pleasure of interviewing one of these economist entrepreneurs, Noam Angrist. Noam is, as I said, the co-founder and executive director of Youth Impact, a non-profit focused on improving the welfare of young people around the world through education and health programs. I met him for the first time several years ago because I noticed what he was doing and had done and wanted to introduce him to my students. So I wrote and asked him if he would be willing to be the de facto keynote speaker at a conference I was helping organize on causal inference. He graciously agreed and spoke with us about the organization he had helped found and the work they were doing in developing countries scaling rigorously evaluated interventions to reach thousands of youth around the world. I was very intrigued because of the blend of causal inference and economics with such creative entrepreneurial work.

Given the explosive success of the credibility revolution at changing hearts and minds, I suppose it was only a matter of time before economist-entrepreneurs trained in that way of thinking would begin moving outside of academic departments and into other parts of the world. We have seen it with economists in tech. Noam Angrist is an example of the contemporary economist-entrepreneur who works closely with governments, academia and even commerce to bring the best of all things to help achieve the age-old quest of economics of improving the well being of humans alive today.

In this interview, Noam and I talked about growing up in Massachusetts and the looking back serendipitous injury in high school that put him on a path to studying economics at MIT. He’s since embarked on an original career as an economist where his competency as a leader, his creativity and curiosity as a scientist and his innate commitment to public service and community is shaping the type of work and the way that work is done at Youth Impact. I hope you enjoy this interview as much as I did. Please follow, subscribe and consider supporting The Mixtape with Scott!

Scott's Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Who is Leah Boustan?

Leah Boustan is a professor of economics at Princeton University and this week’s guest on The Mixtape with Scott. Her research has to date largely focused on two of the largest demographic events in US history: the Great Migration of African-Americans from the rural South to industrial cities in the North and West in the mid-twentieth century, and a period of mass migration from Europe to the US from 1850-1920. She is author of two books related to both topics:  Competition in the Promised Land (Princeton University Press, 2017) and Streets of Gold: America’s Untold Story of Immigrant Success (PublicAffairs, 2022) with Ran Abramitzy.

Leah’s work with Ran on immigration to the US takes advantage of large digitized records from the Census which they linked together so that they could follow individuals over decades. This allowed them to trace out the fortunes of migrants across multiple waves of the Census to ask and attempt to answer several fundamental questions like:

* Did immigrants of the past pull themselves up “by their bootstraps” as the stories are often told to us and remembered?

* Did the children of immigrants move up America’s economic ladder as fast as their “peers” — children, in other words, of established residents?

* Does assimilation today by immigrants happen at a similar or different speed as those in the past?

The conversation was enriching for me, as all of my interviews with Leah are. In Leah you see, also a unique story of entrance into economics — through high school debate, not mathematics, where she grew to love studying the nuances of public policy from an objective yet passionate research-oriented point of view. The roads we take through our lives look like a straight line in hindsight but as we’ve seen with other guests are anything but at the time. Leah became an economist the way she became an economist, but I think it is a story nonetheless that many can identify with.

And an article in economics that she thinks about a lot? Goldin and Katz 2002 JPE, “The Power of the Pill: Oral Contraceptives and Women’s Career and Marriage Decisions”.

In this week’s episode of The Mixtape with Scott, I had the pleasure of interviewing Kyle Kretschman, Head of Economics at Spotify. It was a great opportunity for me because Kyle is one of the first economists I have spoken to who didn’t enter tech as a senior economist (e.g., John List, Susan Athey, Michael Schwarz, Steve Tadelis). Kyle entered tech straight out of graduate school. He spent much of his career at Amazon, a firm that has more PhD economists than can be easily counted. Under Pat Bajari’s leadership there, Kyle grew and his success was noticed such that he was then hired away by Spotify to lead up their economics team. At the end of the interview, I asked Kyle an economics article that has haunted his memories and he said “BLP”, which is affectionate shorthand that “Automobile Prices in Market Equilibrium” by Berry, Levinsohn and Pakes 1995 Econometrica goes by. I really enjoyed this interview, and despite the less than ideal sound quality at times, I hope you will too.

But before I conclude, I wanted to share some more of my thoughts. This series I’ve been doing on “economists in tech”, which has included interviews with John List, Susan Athey, Michael Schwarz and Steve Tadelis, comes from a complex place inside me. First there is the sheer curiosity I have about it as a part of the labor market for PhD economists. As I have said before on here, the tech sector has exploded in the last decade and the demand for PhD economists has grown steadily year over year. Tech demand selects on PhD economists with promising academic style research inclinations. There is substantial positive selection in this market as firms seek out strong candidates can be produce value for them. This is reflected in both junior market salaries, but also senior. Job market candidates are economists with technical skills in econometrics and economic theory, not to mention possess competent computer programming skills in at least one but often several popular coding languages. They are also candidates who were often entertaining careers within academia at the time they entered tech, and in those academic careers, they envisioned themselves writing academic articles about research they found personally and scientifically important and meaningful. Going into tech, therefore, would at least seem to involve choice that may go far beyond merely that of taking one job over another. It may involve a choice between a career in academia and a career outside it, which for many of us can feel permanent, as though we are leaving academia. And for many economists, it may be the first time they have ever contemplated such a thing. If they do internalize the story that way, if they do see taking a job in tech as “leaving academia”, then I can imagine that for at least some economists, that may be complicated, at least.

But there’s another reason I have been wanting to talk to economists in tech and that is I am very concerned about the welfare of our PhD students. In a recent article published in the Journal of Economic Literature, economists interviewed graduate students in top economics programs. They found there incredibly high rates of depression, anxiety, loneliness and even suicidality. This is a common feature of graduate studies, but it is interesting that PhD economists have incredibly good employment opportunities and yet the depression and anxiety plague there too. One of the things that struck me in that study was the disconnect between what graduate students felt about their work and what their advisors felt about their own work. Many students, for instance, do not feel they are properly supported by advisers, do not believe their advisers care about their research success and do not even care about them as a person. Whereas most Americans (and faculty) feel that their work has a positive impact on society, only 20% of PhD students in economics feel that way. (I discussed the article as well as my own research on the mental health of PhD students here.)

I suppose part of me feels a great sigh of relief to see the labor market for PhD economists expanding in light of those troubling statistics. If students know that life is full of infinite possibilities, then perhaps they can begin to process earlier what they want to do in the short years they have on this small spinning ball of rock we call Earth. If students do not in the end want to become professors, if they do not have the opportunities to become one, they should know that there is no “failure” involved there. Careers are just that — careers. They do not tell us who we are. The sooner a student can detach from the unhelpful story that our value is linked to a vita listing our accomplishments, the sooner they can begin their own life work of choosing their meaning. Can having more labor market opportunities with more employers competing for them help do that? Well no, not really. At least, not exactly. It can disrupt certain equilibrium, but then the new equilibrium can just as easily cover that up too. Still, I do like the idea that to keep students in academia, universities and departments must fight harder for them, pay attention to them, and invest in them as people. I like the idea that students have more options and that the options are diverse. Will it help their depression? Well, that’s another matter, as that’s complex. And presumably the economists in the survey I mentioned were themselves well aware of the career options they had since they were coming from the nation’s top 10 PhD programs in economics.

I suppose my point is that ultimately, the burden of life really cannot be resolved with money or career. We are trained to look there because we have boundless appetites. But ultimately the hard work of navigating life can only be helped so much by a job. We must still decide for ourselves what meaning we will choose for ourselves. But one thing I know, and one thing which I think our profession is profoundly bad at saying out loud, is that if we make our identity connected to vitas, we will not just be miserable, we will be hopeless, and probably poisoned. Such a mindset leads to endless laps on a brutalizing treadmill of meaningless performance in which a person chases for first place in a race they don’t remember signing up for and which they cannot win. They compare themselves with others running, not knowing that they too are brutalized by their own treadmill, not realizing that it is impossible to catch up with someone else as there is always someone else ahead of us. The sooner we learn that the joy we long for will not come when we get a top 5, the sooner we can look elsewhere. It has taken me many years to relearn a lesson I learned decades ago — I am whole now. I am complete now. I still run, and I still chase, but I am not chasing completeness. I am not chasing my own wholeness. Being whole and complete has nothing to do with a career. Careers are ultimately orthogonal to hope, which does not mean they do not matter — they absolutely matter. But if asked to deliver meaning, we will find that our jobs are as weak as wet spaghetti at such a task as that.

So, I suppose in some ways I simply want to announce — there are incredible opportunities for economists inside government, commerce and academia. But the weight of this life is not likely to be lighter in any one of them, for the weight we feel in life is largely self imposed, inside us, in the stories we tell about who we are and for many of us who we are not. Those stories are real, because we feel them and because we believe them, but they are not true. All stories are wrong, but some are useful, and the story that our lives can only matter if we have certain types of jobs or certain types of success, while it may be useful to getting a paper out or accomplishing something important, in a much bigger sense it is hollow at best and pure poison at worst.

TRANSCRIPT

This transcript will be updated once the more complete transcript is finished; for now it was transcribed using voice-to-text machine learning.

Kyle Kretschman:Might not have prepared myself well enough to be attractive for some of the most pop most top tier schools. Scott Cunningham:In this week's episode of the mix tape with Scott, I had the pleasure of interviewing Kyle kretchma the head of economics at the streaming platform. Spotify. Before I dive into the interview, though, I wanted to give you a bit of a heads up about the sound quality. Unfortunately, the sound quality in the interview on Kaza side is a bit muffled. We discussed refilming. It tried to find a way to tweak it, but there were certain constraints on the actual sound itself that kept us from being able to do it. And we didn't feel that refilming, it would be good because we thought that the interview had a lot of serendipitous kind of spontaneous tangents and things spoken about that. We thought students and people in academia would want to know, would need maybe even need to know. And I doubted that I could recreate it, cuz I don't even know why it happened. Scott Cunningham:So I'm gonna post a video version of this at my subs, for those who feel that a video version would help them kind of follow it in so far as the audio might be at times challenging. So check out the subst for those of you that wanna watch, watch it instead of just listen to it, hopefully that'll help. I won't say much here by way of introduction, except to say a few things about Kyle, because I wanted to let Kyle tell you his story in his own words, cuz it's his story to tell. And it's an interesting story. Kyle's a PhD economist though from the university of Texas Austin, which is down the road from where I live and work at Baylor, where he wrote on topics in graduate school and applied econometrics, empirical industrial organization or empirical IO and public choice after graduating, Kyle went to Amazon, not academia. Scott Cunningham:In fact, given we might start the boom of tech hiring PhD economists in the early to mid 20 2010s. You could say Kyle maybe was sort of one of the earlier hires among that second wave of PhD economists that went there. He worked for several years at Amazon before being hired away by Spotify to head up and lead a new economics team there, perhaps this is part of a broader trend of tech firms building up more internal teams, not just of data scientists, but like Amazon departments of economists who knows recall though from an earlier interview with Susan athe where, when I asked Susan why she said pat Maja had done something amazing at Amazon, she said he made economists productive. And in time he made many of them productive and very in productive from what I've been able to follow. And Kyle is from what I can gather someone whose skills matured and deepened under the leadership of Papa jar at Amazon and other leaders at and other economists at Amazon. Scott Cunningham:And he was ultimately hunted down by a major tech term to create an economics team there I'm by no means an expert on the labor market for PhD economists. I just have been very intrigued and curious by the, the, the Mar the labor market for PhD economists in tech, because well, partly because of realizing first that cause of inference was really valued in tech, but then to sort of realize that there was just this very large community of economists there, but I don't think it's controversial to say over the last 10 to 15 years, the tech industry really has been disruptive in the labor market for PhD economists. They continue to hire at the junior and senior market in larger and larger volume selecting more and more on people who likely would've gone into academia into tenure track or tenured positions. They pay very high wages, some of the very, some of the highest wages in the country, both at the junior level and especially at the, at the higher end at the, at the more advanced levels, people can earn compensation packages by the, in the, by the time they're in their thirties, that many of us didn't know were possible. Scott Cunningham:It's in my mind, historically novel, and I might be wrong about this, but it, it seems historically novel that the PhD economists who likely would've produced academic research papers in tenured and tenure track jobs have begun to branch out of academia, but maintain those skills and maintain that research output. It's partly driven best. I can tell, buy Amazon, I might be wrong, but by Amazon and paja, as well as Jeff Bezos own view, that economists are what I guess we would just say value added for many firms. Therefore I'm continuing to wanna speak with economists in tech to help better trace out the story. This interview with Kyle follows on the back of earlier interviews with people in tech like John list, you know, a, a distinguished professor of economics at the university of Chicago, but also the former chief economist that Lyft and Uber now Walmart Michael Schwartz, former professor of economics at Harvard. Now, chief economist at Microsoft and Susan athe former chief economist at Microsoft professor at Stanford and now chief economist at the DOJ. I hope you find this to be an interesting dive into the industry. Learn a little bit more about economists there, but by, by learning the about one particular important economist, there a, a young man named Kyle crutch, head of economics at Spotify, my name's Scott Cunningham. And this is the mix tape with Scott. Scott Cunningham:Well, it's my pleasure today to have, as my guest on the mix tape with Scott, Kyle crutch, Kyle, thanks so much for being on the call. Kyle Kretschman:Hey Scott, thanks for having me really appreciate the time to talk Scott Cunningham:Well before we get started with your career and, and everything. I was wondering if you could just tell us your name and your title and where you work. Kyle Kretschman:Sure. Yeah. As you said, I'm Kyle kretchma, I'm the head of economics at Spotify, Scott Cunningham:Head of economics at Spotify. Awesome. Okay. I can't wait to talk. So let me, let me, let's get started. I was wondering if you could just tell me where you grew up. Kyle Kretschman:Sure. So most of the time I grew up in outside of Pittsburgh, Pennsylvania, about an hour north of the city, real real small town probably had one stop light. And maybe the, the funny story that I can share is what I took my wife there. She asked where's the Starbucks. And I said, no Starbucks here. There's no Scott Cunningham:Starbucks. Kyle Kretschman:Yeah. So pretty small town called Chippewa township in Pennsylvania. Scott Cunningham:Oh, okay. Is that near like Amish stuff or anything like that? Kyle Kretschman:No, that's the other side of the state. So this would be Western Pennsylvania about near the end of the turnpike, about five minutes from the Ohio border. Scott Cunningham:Oh, okay. Okay. You said, but you, did you mention, you kind of grew up in different places? Kyle Kretschman:Yeah. So before that, my father worked in civil engineering and so would do build roads and bridges basically across every, across the nation. So I was actually born in Louisiana, lived there with, I think for a whole two, three weeks. I don't quite remember. Cause I was pretty young obviously, but then Michigan and then spent some time in Philadelphia before moving out to Pittsburgh around second grade. Scott Cunningham:Oh, that's kinda like, that's like when people described their parents being in the military, just kind of moving around a lot. Kyle Kretschman:Yeah. A little bit. So, but Scott Cunningham:Then you settled in the second grade Kyle Kretschman:That's right. Yeah. So outside of Pittsburgh and then stayed in Pittsburgh through high school and even through undergrad. Scott Cunningham:Oh, okay. Oh, you went to undergrad in Pennsylvania. Kyle Kretschman:Yeah, I did. So I went to undergrad at the university of Pittsburgh. Oh, okay. It was, yeah. If, I guess maybe continuing the story growing up in a town with no Starbucks. I was, I was pretty intrigued by going to a city. Yeah. And find out that lifestyle and yeah, we might have lived pretty close, like an hour away, but we didn't go down to the city very much. So Pittsburgh was just really, really enticing for a city to, for, to go to undergrad in. And so I basically looked at all schools that were in cities and so the proximity plus then the, the ability to just spread my wings and explore what it's like to be in a city was really, really enticing. Scott Cunningham:Did any of your friends go to pit with you? Kyle Kretschman:Yeah, so there's probably, I grew, I graduated from a class of about a little over 200 people in high school and I think there was like five or six people from high school that went to pit for my class. So definitely had some really good friends who went and kept in touch with, through undergrad. Scott Cunningham:Mm. Yeah. So it wasn't, were you sort of an early generation or you weren't, were you a first generation college student in your family or did your parents go to college Kyle Kretschman:Combination? So my dad went to Penn state civil engineer, as I mentioned, me and my mom actually graduated from undergrad the same week. So my mom went back to school later in life after me, after we went to school. And so yeah, we, we were able to celebrate graduation cuz she went to a small private school right outside of the city also. Scott Cunningham:Oh, okay. Okay. Yeah. Well, so what did you like to do in high school? Kyle Kretschman:So I played a lot of sports before high school and then I kind of switched into, and this was a traditional sports of football, basketball, baseball, but then I switched into tennis in high school. And so that kept me busy, but along with a lot of academics and really, really liked computer science. So played a lot of video games growing up, really enjoyed like that aspect in combination. Scott Cunningham:What games were your, were you, did you play on a, on a video game, plat platform? Like an Nintendo or did you play? Kyle Kretschman:Yeah, no, we played a lot of plays very much into like role playing games. Some of the arcade games like Marvel versus Capcom. So yeah. Yeah. Very, very interested in gaming. Yeah. Maybe I was a little too early for that. Cause you know, every, everybody in the 1990s was like, oh, I could make pu money playing video games, which wasn't true back, which wasn't true back then, but that's right. You know, nowadays Scott Cunningham:You can that's right. Yeah. You know, that's right. You can do it. There's all kinds of ways you can make money doing things today that nobody knew was possible 10, 10 or 15 years ago. Even Kyle Kretschman:My Scott Cunningham:That's cool. Yeah. I, I, it's funny, you know, computer games can keep a, keep a kid in high school going, you know, like especially I think they're kind of misunderstood. I, I had a lot of friends that, well, I mean, I, I, I had, when I didn't have a lot of, we moved from a small town in Mississippi to Memphis and I, those, those that first year when I didn't have a friends, I did bulletin boards and played Sierra online games like Kings quest. And it's like, it's like, you know, not intertemporal smoothing, but like inner temporal socializing, smoothing, you know, so that you just kind of get through some periods that would otherwise be a little lonelier. Kyle Kretschman:Yeah, for sure. And I mean, I mean for this audience, like most video games are some sort of form of constrained optimization. So there was, there was the inkling that I, I liked understanding how economies worked in high school through this and yeah. Going back to my mom, my mom always said like she encouraged it and she encouraged education. And there was actually kind of like that nexus, whenever I took economics in high school, it was like, oh, you know, some of these games really are full economies that are constrained and constrained in a way that you can understand and complete in, you know, under a hundred hours. Right. But there was that combination that was kind of showing itself of computer science, computer gains and economics of putting itself together. Scott Cunningham:So you were kind of thinking even in high school about economics in that kind of like, you know, optimizing something and like, like almost that modern theory that we get in graduate school. Kyle Kretschman:I think more, I had the intuition when I didn't have know how to say what it was in high school because my high school was pretty forward and that it offered both advanced computer science courses that could get you through definitely through first year of undergrad, maybe even through second year with advanced placement. And then they also offered advanced placement economics. And so I, I ended up taking advanced place in economics my junior year when most people took senior year. And so whenever I was going small Scott Cunningham:Town, even in that small town, they had, you had good your high school. Good econ. Kyle Kretschman:Yeah. It was a real, it was a really good high school that would put together good curriculum that did a lot of college preparatory work though. They, wow. They really leaned into the advanced placement, the AP courses to get students ready to go to school. Scott Cunningham:Wow. Wow. So even at, as a junior, you're taking AP econ, you know, you don't have to take AP econ. That kind of is say that, that sounds like somebody that was kind of interested in it. Kyle Kretschman:Yeah, very much. Yeah. And again, as soon as I, I definitely didn't get to the graduate level of understanding, like, you know, LaGrange multipliers, but the, the micro and macro sequence just made intuitive sense to me. It was like, it was kind of where I was like, yeah, this fit. And this is how I think. And some people might criticize me now that I think too much like an economist. Right. Like, but at the same time, it just like, it started to put together that language and even more so some of the frameworks that really kind of drew me into it. Scott Cunningham:Well, did you, did you, did you notice that you had this interest in computer science and this interest in economics and that they might be one, did you get a feeling that they could be in conversation with each other? Kyle Kretschman:Not Scott Cunningham:At first, our ancestors a hundred years ago. Didn't, you know, those economists didn't think that way, but now it's just so natural for this generation of economists to be almost one half, you know, one third mathematician, one third economist, one third computer scientist. Kyle Kretschman:Yeah. So not at first, but I, I feel like I made have like lucked into it, honestly, because whenever I chose to go to Pitt, I chose to start as computer science because I knew what that pass was. I was inspired by my older brother, the great teacher in high school. And like, I was definitely like, okay, a software software development engineer career is great. It's cutting edge. It's there. But after probably like the first year, it just didn't feel that end state didn't feel right. And so I made kind of the hard decision to choose, honestly, to switch into economics as a major, because I wasn't sure what the end state would be, where I was going with it. Cuz it was definitely felt more amorphous, you know, it's a social science, so yeah. It didn't feel like it was gonna be as clear cut and as, and have as much certainty. But pretty quickly, like after a year was like, oh, well we're doing, we're using E views at the time. All right, this is coding. I know how to do this. This is great. Right. And starting, starting to see some of that in undergrad was like the, kind of the aha moment that like, yeah, this is, this is a place where I can apply this love of coding and problem solving, but problems and solutions that I find really, really hard and interesting. Scott Cunningham:It was because of econometrics though. It was in that. Kyle Kretschman:Exactly. Yeah, yeah. Scott Cunningham:Yeah. Wow. That's, that's really interesting because you know, I think it's still the case that, you know, you can easily end up with an econometrics class that remains purely theoretical and doesn't end up, you know, exposing the student with a lot of actual coding, but it sounds like your professors were, were getting you into working with data. Kyle Kretschman:That's correct. Yeah. Both. Both within the class. So like I said, we used E views at the time. Yeah. And again, kind of like learning as a go, I, I don't think I really knew what I was doing whenever we were typing commands and E views, but the computer scientist in me was like, okay, well this is a function. I know functions. Didn't put outputs, but definitely didn't understand necessarily things that were going under the hoods or you know, all of the theory that goes with it. Oh, right, right, right. So it was, you Scott Cunningham:Knew the coding part, you knew you were coding, but you did, but like the, the actual statistical modeling was kind of the new part, but that was a way for you to kind of engage it a little bit. Kyle Kretschman:Yep, exactly. Scott Cunningham:Oh, that's interesting. That's interesting. Well, so what were you gonna have to choose between a computer science and an econ major did or did you end up doing both? Kyle Kretschman:So I chose an econ major, but then I had what I would call basically minors or concentrations in computer science, but then also in statistics and also in math, because once, once I had an internship at a bank and was doing data entry and I was like, eh, I don't think this is what I wanna use my economics degree for. Yeah. I had a couple professors at pit named Steve Houston and Frank Giani who brought me on as a research assistant, an undergrad to start being part of some of like their survey projects and data collection. And even, even one of 'em I don't, Steve was crazy, but he even let me TA classes on undergrad, so oh, wow. But he kinda, I mean, I, I say that jokingly because it was formative for me, it was like, okay, this is great. How do I do more of this? And he was like, well, you go get your econ PhD. And I was like, so I can be a teacher with computer science and doing economics altogether. He goes, yeah, let's do that. And so it was with the help and support of some of these really good professors and education to kind push me on this path consider to get Ancon PhD. Scott Cunningham:Mm. And that's when you were like, so how, how, what, what year would you have been in your program? Kyle Kretschman:Probably. I think I was in my junior year where I was starting to explore this. And then in my senior year is where I was like, okay, I'm actually gonna be doing more more of this and applying to grad school because going back, as I said, I entered with some credits. So my senior year was very, I didn't need a full course load. So I was looking for other things to keep me busy, which maybe, maybe that's one of the themes of this conversation is I kinda kind of like the variety and really have variety seeking behavior too. Yeah, Scott Cunningham:Yeah, yeah. Yeah. So you graduate, was there like a field that you were mostly interested in? Kyle Kretschman:I thought I would be going into macro economics. Macro. Yep. Yeah, because Steve worked on the council of economic advisors and I was really inspired by that and the application of economics within, within policy and just again, always applied economics, not necessarily theoretical. So yeah. Then again was, that would be sort of like labor and macro was like the initial idea, but finally Scott, I didn't do all my homework and like, think about like what grad school looked like or all it looked like. I kind of went a little bit more naive than I think other people with, again, ideas of how I could become like a teacher, an educator with some of these tools versus like how disciplined and single thread you need to be on research to be within an econ PhD program and to see that. Scott Cunningham:So you, so you kind of were like, so when you were thinking about graduate schools, what, how, what, what did you sort of, can you walk me through like what you were thinking and how you went about trying to apply to graduate school and where you ultimately chose? Kyle Kretschman:Yeah, sure. So applied probably the, the top 10 and the top 10 probably said no thanks. But also then was targeting specific schools that we had relationships with that I knew would provide computer science and macros. So university at the Iowa at the time, this was 2000 and had a really strong macro program. And then also at the university of Texas with Dean Corbe there, they also had one in Russ Cooper. And so those were like the two that I was like targeting at outside of what the top schools were. But yeah, as I, I kind of mentioned, I, I might not have prepared myself well enough to be attractive for some of the most pop with top tier schools because kind of, you know, as I said, bounced around and would be yeah, a little bit working on it a little bit different things and have computer science versus being solely focused on like economics and math and things that might be more of what the top tier schools were looking for. Scott Cunningham:Yeah. Yeah. You know, you know, it's like the, I mean, I'm the same way. I didn't ha have any econ classes in college. I was a English major, but the, the, the diff there's so many students that sort of seem to almost for whatever reason, know a lot sooner what they want to do and then like make those choices. And then there's just many of us that are, you know, in a process of search yeah. That when you're in a process of search, well, you, you know, by definition, that's like you're using that time to search. Kyle Kretschman:That's exactly right. As Scott Cunningham:Opposed to saying, I've gotta take, I've gotta become a triple major computer science, math, econ, and have to do like, you know, these set of these set of steps that, you know, there's no way I could even have known to do it unless somebody had told me it's weird. I mean, it's just funny how the little things can have such big repercussions for your whole life, but it's, but it, it worked out great. So you end up, where do you end up going? Kyle Kretschman:I went to the university of Texas at Austin. Scott Cunningham:Yeah. Yeah. What year was that? And Kyle Kretschman:So, so this would've been 2002. Scott Cunningham:Oh, okay. So you go to oh 6 0 7. Kyle Kretschman:Okay. And so ended up working. So I ended up working a lot with Jason, Ava. Yeah. And who came in and became the, the head of the department. Yeah. Applied econometrician who just did an amazing job going back to whenever I said, I didn't know how things worked under the hood, in those formulas. He didn't even let us use those formulas. So anytime we were doing applied econometric econometrics with them, not only we learning to teach, we're learning the theory, but he said, you have to code it yourself. You have to do the matrix algebra, you have to calculate standard errors. You can't really call those functions. So that was probably again, that wasn't until the third year, but yeah, in the first year to go back a little bit, Scott Cunningham:I, that played to your strengths though. I bet that played to your strengths. Yeah. Just at the end of the day, wanting to be someone that, that wrote down the raw code. Kyle Kretschman:That's exactly right. And, but the first year I didn't play my strength. Yeah. Yeah. So the first year I felt, I felt a little bit outta water and I was like, this is, I remember when we were proving what local non association. And I was like, this is, this is one hard, but also like, again, going back to like, that is this actually how I wanna be spending my time and right. I, I was like, yes, I do. But I was like, I, I knew that I needed to get to those applied applications. Yeah. And so that's, again, why I was thankful to be able to work with Jason and Steve Trayo and a few other, they applied econometricians at Texas that really encouraged me to explore starting in the second year. They didn't us like pin it down. And so I, I thought I, at the second year I worked like wrote the first, a paper on school choice and trying to see if I could find some sort of instrument on school selection on public versus private. And again, so that led to like that idea of like applied econometrics was really, really the thing that like, I was like, okay, now this fits again. Once we got into second and third year Scott Cunningham:Was, was picking up that intuition, that kind of like labor style identification, causal inference kind of approach. Was that something you picked up from Jason or was that just like from your labor people? Oh, okay. Kyle Kretschman:Yeah. That's yeah. From Jason and Steve a lot. They did a great job of doing that. And yeah. So then, yeah. Then I, then I threw in, I knew threw a little bit of a switch in there also, and my co-author Nick master and Arti and closest friend and classmate in Texas was very theoretical and very interested in applied empirical IO. And so we started working in that field also together. And so then I got to work with the Han me vet and Ken Hendrix on using empirical IO. So, oh, wow. Yeah. And so again, Scott Cunningham:This is the more structural, more structural econometric. So you've got this like reduced, you've kind of got this like traditional labor reduced form type of, part of your brain. And then you've got this empirical IO structural part of your brain kind of emerging at the same time. Kyle Kretschman:That's right. That's exactly right. Yeah. And then we threw, we threw everybody for a loop. I also saying we wanted to study study politics and how money turns into vote using both using all these tools. So yeah, I can see here kind of saying in hindsight, like it all makes sense in this story that I'm telling you, but at the time it was more of what you were talking about. It was searching. It was, I wanna be working on really interesting applied problems. I love the toolkit that economics provides in framing. And yeah. I have to be coding to be able to utilize these tools that I've had built up in the past. Scott Cunningham:Yeah, yeah. Yeah. So, so matching with Nick was really important Kyle Kretschman:Very much. Scott Cunningham:And why, if you hadn't to match with Nick, I mean, just kind of outta curiosity, if you could articulate the value added of that whole partnership, what was it? Kyle Kretschman:Yes. Sure. So, so we matched basically from math camp going into, going into the first year because Nick came both from the pure math and physics background and also had some experience in the air force. So the air force was sending him to Texas and he, we were, we were definitely, we definitely didn't have a lot of vend overlap on the fact. He's like, well, I would have the intuition and some of the computer skills, Nick would have the theoretical math skills, Scott Cunningham:The theoretical math skills. Yep. Kyle Kretschman:And then we just had, we had the common factor that we wanted to work hard together and learn together and we're willing to, we're willing to intellectually hash out really tough things together. Yeah. So yeah, he huge credit to him through being able to put up with me. And he says, he says the same thing once in a while. But again, matching with somebody that had the, the more real analysis proof based understanding of math was so valuable for me. And especially, Scott Cunningham:I think some empirical IO, especially empirical IO, just being able to, you know, think like an economist in the area of IO is thinking real deep about, you know, a rich set of models and modeling approaches. Kyle Kretschman:That's Scott Cunningham:Exactly right. That's definitely not what you're learning in your econometrics classes, even though they might go together. Kyle Kretschman:Yep. So, so yeah, it was just a, it was a really good match from the beginning. And so we complimented each other and we're, we're able to build a strong enough relationship to be able to be able to hash out, have really long nights yelling at each other, we say in the office, but it never, it was always for educational purposes and lifting each other up. Scott Cunningham:Was that different than what you thought grad school was gonna be like? Kyle Kretschman:Yeah. So I knew the research component a little bit. I just didn't under understand the unstructured research on how that was gonna go and like the cadence and where it was gonna and how that was gonna be so required to develop your own viewpoint. Yeah. I thought it would be more directed cuz as a 22 year old, that was the experience I had generally. So that was the big one was the undirected and I liked it, but it was also very difficult. Scott Cunningham:How would you describe what you're talking about to your college self? Who kind of like, you know, he, he doesn't really, he doesn't even have the vocabulary for what you're describing. What would you say? It was like, Kyle Kretschman:I think you use a good term. You have to be not only wanting to search, you have to be willing to search, but you also, then you have to put in the guardrails yourself to keep it focused because you're not necessarily gonna have those external guardrails that you will have from an alternative path of going to either like a master's program that's gonna be more structured or going in an industry or going to get a job. Right. Like I mentioned at a bank for like a 22 year old where entry level jobs are gonna be more structured. Yeah. So yeah, I just, I, I probably knew it, but I didn't know what it meant to be and what, what it meant to experience it. Scott Cunningham:So how did Jason and, and Steve kind of, and any other faculty, how, how did they, how did they, I, so I did this interview with Susan athe and she was saying that, you know, the amazing thing that pat Maja did at Amazon was he managed to make economists productive, which kind it was kind of a weird, weird way of saying it. And so in a way it could, in a way you could imagine a department that sort of has like a, you know, this idea of like research has got to come. There's like a, there's like a, a journey that a graduate student has to come on to just to basically make a decision to be a researcher. Yeah. You know, and you could imagine that creating the conditions for that is, is involves faculty member, doing stuff that's not necessarily obvious. What, how did they, how do you think they contributed to that for you personally? Kyle Kretschman:For me personally, at the time, again, it goes back to encourage the exploration versus mandating or saying that I need to be on one path. So like even Nick and I at the time explore the idea of a private company and how, what, what that would be into like pinching, pitching a venture capitalist on, on that. So all those things, again, in grad school, they, they were encouraged, but they weren't structured at the time. Yeah. So yeah, I can, I can, I understand Susan's comment because I was, I was one of those economists who started pretty early with pat and we, we have a lot of good mechanisms that we've learned and built at Amazon when I was there at the time through pat, through lay other people who were willing to make the jump into this entrepreneurial space that hit the election and the, of coalesce of economists doing open book, empirical research, along with data science. Right. Just becoming more and more valuable and applicable, but is kind of what Susan piloting that we can, we can talk more about if you Scott Cunningham:Want. Yeah. I do wanna talk about that. I wanna talk about the, the decision though, you know, to, to be, because you, you sort of started off in college, you know, you said things like, oh, you can become an educator and then you've gone in this non-academic direction and you know, it, it, and that's like a, that's a more common story now, you know, right. Of, of top talent, very talented PhDs that you could have easily seen 20 years ago, would've been an academia. Their counterfactuals are, are following you. And so, you know, it's, it's a, it's a big part of our, you know, collective story as economists that this, this new labor market that didn't, that didn't exist historically now exists and draws in so much talent. And I was just curious in a way you're kind of like a, a first generation person like that, you know, when you think about it, right. Cause text's not very old, right. Facebook, Facebook, what it's like 2007. And so, you know, so you've got this, you, you, you've got this, this chance to kind of say like, it must have been, so I don't wanna put words in your mouth, but I guess I was just wondering, what were the feelings like as you considered not taking an academic track and when did it start to be something in your mind that you thought that's gonna be something I'm explore Kyle Kretschman:Probably pretty early, because if you wanna really trace the roots of like tech economists back, it starts obviously with Hal varying at Google and me and Nick, actually, we, we sent an email to Hal, probably 2008 saying, do you have any, have any use for some summer interns who can do some empirical IO? And he said, no, not, not at this time, but so, but he Scott Cunningham:Answered the email. Kyle Kretschman:He did answer the email. Yeah. It was nice, nice of him to answer. Cause we knew he was probably pretty busy, but so it, honestly, when Amazon started hiring economists, I was probably searching for about a year to move into tech. If you wanna move back to the decision point coming outta grad school, honestly it was a challenging labor or a challenging job market for me, somebody who is a lover variety, who is working on empirical IO problems with campaign, policy, campaign, finance reform, policy recognition. That's, that's not fitting a lot of the standard application process. Yeah. Once again, that's so that's probably a theme for me. And again, at the time it was hard. I was, I was in the running for jobs at VA wakes force that I thought would be really good fit because they're the EDU the emphasis would be on education with the research ability to do research and work on problems that were more widely probably policy oriented. Yeah. But neither neither of them came through. So I just always knew that I industry was gonna be an option. And so Scott Cunningham:What year is this? What, Kyle Kretschman:What, what this would've been in this would've been in Scott Cunningham:20 11, 20 11. Okay. Oh, so you moved through the, you moved through the program or kind of relatively quickly. Oh 7, 4, 4, 5 years. Okay. Kyle Kretschman:Five years. Yeah. Five years. Yeah. Oh six to 11. Okay. But so for about a year, about six. Yeah. Yeah. And so starting in 2013 is whenever I started applying to the first tech job as a data scientist and got it went great until I talked to the VP who was a business part, like pure business person. When I was talking to the hiring manager at the time, it was a company who was providing college counseling as a software service. And so they would do this at their, their clients were both for profit and not for profit companies. And we were talking like, we'd get into details about treatment effects models and how we could measure the impact of their intervention. It went great. But then I had the flyout scheduled, but then the interview with the VP, he said, well, how am I gonna monetize your algorithm? Right. And I was like, I'm not sure I know what algorithm means, but right. I, I wasn't prepared for that language and that application and how you turn econometric modeling and measurement into, into business impact at the time. Yes. Right. So spent another year looking around with different opportunities like that and honestly learning again. So, so whenever Amazon, so this would've been in 2014 and then Amazon was hiring its first big cohort with pat. So this was a cohort that was about, I think there was about 13 of us. It was a no brainer. Kyle Kretschman:Whenever, whenever we did the interview, it just was like, all right, this is exactly right for me. I was hop. I was hoping it was right on the other side. And I could probably tell you some funny stories about the interview process, but I was like, this is, this is what's meant to be. Yeah. So it, it, it was like a 10 year journey from 2004 when I switched outta computer science into 2014 being like this, just this fit. Scott Cunningham:Right. Right. Right. So outta curiosity, you know, is, is there, is there something that you think is supposed to be learned by the fact that when you were on the job market and you had that interview with that, that gig and the, and you get to the VP and he articulates questions that are not traditional econ questions, or even econometrics questions like business profitability to act, it's kind of ironic, isn't it like to everybody? That's not an economist. That's actually what we, they think we do, you know, is like, they think we do all that stuff. And then they don't know that we're like, like you said, you know, trying to set up a Lara and solve, solve it, like what's a Lara, but do you think your competition at that time did know how to answer questions like that? Like non-economists in those positions Kyle Kretschman:Probably at an inflection point. Yeah. Because this is the same time. Wherever machine learning is becoming more common toolkit with an industry. So there would be like machine learning algorithms that are designed for, you know, prediction, problem sequencing, anything like that that are specifically designed to be used in a business setting to monitor. Scott Cunningham:So they, they not only know machine learning, it's like, they also can kind of immediately articulate why this would be profitable. Kyle Kretschman:I think so. Yeah, because again, the computer, so it's like in learning the language and this is the language that would probably be more understood within a machine learning computer science version is okay, well, I'm gonna use this to change the recommendation engine right. Is very common one. Yeah. That's obviously gonna be, so how are you gonna monetize it? I'm gonna improve the match and the recommendation engine it's gonna have this. So I think at the time there was a little bit of it, but, you know, hopefully I think, I think I learned pretty quick that you can, you can use econometrics in a similar vein. As I said, it's a flavor of data science, Scott Cunningham:Have you had to become a blue collar machine learner? Kyle Kretschman:I've had to understand it, but not, I think you mean by blue collar, you mean like implementing it Scott Cunningham:And yeah, I just, when I, I usually say blue collar in the sense of like, you know, you, don't like, you know, you basically are picking up these skills, but you weren't like, you know, you didn't get a PhD in computer science. You know, Kyle Kretschman:The answer was then that answer is definitely yes. So like as we, as our cohort and as we grew, the economics discipline at Amazon, that was a big part of it is how one could we bring in some machine learning scientist help educate and teach us. Mm. And yeah. So, and even in, sometimes in lecture style, we would do that because it was so important, but then even more so learning to so that you can interact with different stakeholders specifically, like machine learning scientists. Mm. Then understanding when you can actually implement it and marry it within the econometric models was definitely a huge part of the education process. Scott Cunningham:So you go to Amazon, is that right? That's like your first entry into tech Kyle Kretschman:That's Scott Cunningham:Right. Is Amazon, what's your title? Kyle Kretschman:So Scott Scott Cunningham:A scientist or economist. Kyle Kretschman:I, it was something like business intelligence engineer. There wasn't an economist job family. There was, as you said, it was kinda the forefront. I think it was this. Yeah. I think that's what it was, but Scott Cunningham:Cause it is now right. Baja has a that's Kyle Kretschman:Right. Scott Cunningham:He created a job title called economist. Kyle Kretschman:That's right. Yeah. And that got set up about a year in, so like, and I was part of the group. So we would set these, we would set up like these people and process mechanisms that allow economists to be so influential and productive within Amazon. Scott Cunningham:Mm, okay. So how is he doing it? Why, why is Susan saying he performed a miracle by making economist productive? Can you kind of describe, like, if you had to just guess at like the counterfactual, if it hadn't been, you know, pat, it hadn't even been an economist that was hired into Pat's position. Like, what is it that he, what, what is it that he, or Amazon or whatever is making you go transform and become this new version of yourself? Kyle Kretschman:There's, there's a lot of factors and I could probably spend an hour on this, but I'll, I'll try to, I'll try to reduce it down to like some key mechanisms and ideas. The first is that Amazon is probably the most data driven company. I know. Mm. They are so focused on measurement, both of things you can directly measure. And, but they are. So they were very early interested in economic measurements that are UN observables either coming from like coming from econometric models. That, that was whenever pat demonstrated some of those that was like the light bulb went off the, so, because again, it, Amazon was run by and still generally is people with operation science background. And so this over index on measuring as, as coly and as precisely as possible, well that's that's economics. So that, that was part of it. Another part of it is culturally Amazon operates that makes decisions based on six page white papers, you wanna make some economists really productive, have them write a six page white paper instead of giving them a presentation, especially to people like who may be in the background with MBAs or other people who have a comparative advantage, we economists have a care advantage in writing. Kyle Kretschman:So it was little bit of like a surprise, but you might hear these anecdotes where it's true. Like whenever you go into a, a decision making meeting, you come in with your six page white paper that says here's the business decision to be made here is my recommendation. And here's why, and people sit there and it can be a room for five people can be a room of 25 executives. They sit and read the paper and they read the whole thing. Is there an append that can go on forever depending on how big the meeting is. Sure. But that structure of, of data driven decision making, combined with how you're presenting your argument is written seems like, seems like economists should be pretty good at that. Right? Scott Cunningham:Is that a pat thing? He came up with work, the work he made, Kyle Kretschman:What was the six page idea was from Jeff Bezos. And so that was, would Scott Cunningham:Those be circulated throughout the, throughout the, the, the firm, Kyle Kretschman:The stakeholders who needed to be part of the decision making they be circulated. But again, this is every, like everybody's writing six pages. PowerPoint is basically outlawed at, at Amazon. And again, that happened mid 2000. Sometimes people can Google it to find out, but that six page culture and decision making culture, just again, fit economists. Scott Cunningham:So how is a six page paper similar to the kinds of writing that, you know, you sort of associate with economists and how is it different? Kyle Kretschman:So its I'll start with the differences. So one with the six page versus like a 30 page academic, you are not going to be able to share the research process. You are not supposed to share the research process. You're supposed to share the clear recommendation and how you got to that recommendation. Right? So if you think about like a 30 page academic paper XT, be condensed down into those six pages. In my view, they're just, that's just not how the industry operates, but you probably would know better than me on that where, but so again, where it's the same is again, it's a data driven argument. The purpose of this paper, the abstract here is the hypothesis that I have that and here's how I tested it. And here's how I'm making my conclusion. So what I always found really honestly easy was I felt like I was doing the scientific process. Like I felt I, I was with business decision making it generally work within what is the hypothesis? How are we doing this? How are we testing it? What are we think some alternative conclusions could be, but what are we making towards it? So yeah, yeah. Again, it was closer to what I felt like would be a scientific paper in and that hold of day driven mindset is again, that's more, it's very common. Amazon have a common Spotify now Scott Cunningham:Has that been influential throughout, throughout industry? Has that, how have you noticed Amazon influencing Kyle Kretschman:Some Scott Cunningham:Yeah. Like most people don't understand. Kyle Kretschman:Yeah. There there's some companies who definitely have completely adopted it. There's some companies who haven't, but the, the six pager again, that's, this is not a, this isn't a concept just to economist and tech. This is the concept is, is held up as one of the key mechanisms for all of Amazon. Scott Cunningham:Mm mm Hmm. Kyle Kretschman:One other. Scott Cunningham:How often were you writing those? Kyle Kretschman:Depends on what level you were farther in my career. That's the only thing I did was write six page papers and it would be part of like, my team would help, but again, anytime you have a key business decision to be made or an update, like you're gonna be writing the six page. So yeah, it's again, the farther, the more seniority you have though, the more that becomes your job is to communicate side and guide through these business decisions. Scott Cunningham:Do they, to you, Kyle Kretschman:They belong to the team because it's always Scott Cunningham:Put 'em on a, you can't they're like proprietary though to Amazon. Kyle Kretschman:Oh, correct. Yeah. No, they, they're not publicly available. They're Scott Cunningham:Proprietary. Like it must is it what's that feel like to do something? What's it, what's it feel like to, to do something that creative in that kind of like scientific that's siloed within the firm? Does that feel strange? Kyle Kretschman:No, it didn't. Because what it enables is to be able to work on some of the hardest questions without having to worry about without having to worry about com communication strategies or right. For press release. So no, it felt like we were able, and this is going back to like some of the things that pat and we did at Amazon make successful. We worked on some of the hardest problems at Amazon from a very early stage because we said that it wouldn't be publicly available. Right. So that's gonna do that. And Scott Cunningham:That's been a key part. Yeah. Because okay. I get it. Okay. That, that makes a lot of sense. Yeah. So who did you discover? You were, go ahead. Sorry, Kyle. Kyle Kretschman:No, I was gonna say maybe the last me to highlight. Cause again, I, I, we could probably spend this whole interview on this, but the, the other key mechanism that pat pioneered was the proliferation of economists as a job family was not pat saying and us saying, go do this. And I can give through my own personal example. It was the other business executives, seeing the measurement, seeing the results on product, just saying, okay, I want that. So it really was a demand, AKA demand, internal demand for more economists, that was gonna say, I want this with my business decision making process and want these people who can do this and collaborate across the difference. It was not a, oh, we're gonna put economist in the siloed function that everybody's gonna come here. And that was, that was my story. But the very first year I worked on projects directly for the consumer CFO, basically the whole year. It wasn't necessarily by design, but it was what happened. And at the end of the year, year and a half, the, the VP of finance said, come over here and do this with me and come build, come build an economics team and an economics function here within my organization. And that's really is again, that's the real key was it was business decision makers, demanding the ability to understand this and demanding the skill set, just like they would data science, machine learning because of demonstrated value. Scott Cunningham:What were they witnessing with their own eyes that was so compelling that they would Inc that it would increase demand. Kyle Kretschman:So both I'll call it like ad hoc economic analysis on maybe big strategy projects, but also then the introduction of econometric systems into product. Scott Cunningham:Mm. What does that mean? Introduction of econometric systems into products. Kyle Kretschman:So say you have a product that is gonna, let's go back to the recommended system. And I use that again as an abstract, but within there you might make a change to it and you might make a change with the recommender system. That's gonna cause a treatment effect. Right. So, okay. So we can do that one off to estimate that, but you could also then build an economic system. That's gonna measure those treatment effects and changes like an AB platform or things like that. So maybe people might be more common and familiar with like experimental platforms. This would also be then econom. This would be sub out the AB part of it and sub in an economic model, that's going to be doing always on measurement sometimes at a, you know, service level. So sometimes within like individual pages, sometimes it's gonna be at a monthly level, but the integration of econometric models into the product. Scott Cunningham:Right, right. Wow. So how are you a different economist because of that experience at Amazon, if you had to guess, what was it the treatment effect? Kyle Kretschman:Oh, it mean it was, it was incredibly formative because it to tie like it put the fit together with the application to where I could understand and really to where it is, my job is to take a business question, turn it into a scientific process that can be solved with econometrics. And then also be thinking about, is this a problem that needs a scalable solution? Right. So, so Amazon taught me business integration taught me so many different languages, taught me leadership and management taught me how to work with stakeholders in collaborative ways, but then even more so how to deliver the value through econometric measurement, both again, as I said, not only, not only just in ad hoc research papers or one off analysis, but also then where does this fit directly within the products that we build in tech? Scott Cunningham:Yeah. So where'd you go, seems like people don't stay very long in tech. That's like normal. Whereas like, is, is that right? People kind of like, it, it's less normal to stay your whole career at Amazon unless is that wrong or, Kyle Kretschman:I mean, it's got it still do. So it's probably tough to say that because really the, the field started, like you said, really proliferated in 2012. So I stayed at Amazon for six years and I thought I'd be staying even longer. But Spotify came with the opportunity to one work on something I care very deeply about, which is the music industry. I'm a huge music fan. They also came with the idea to build again. So, you know, that was the part that really enticed me was Spotify did not have any PhD economists who were in an and, and economist roles. They had like one in a data science role, but they didn't have the structured economic discipline that they were seeing that Amazon was proliferating. And also then going into like Uber, Airbnb and the other tech companies. And so they said, can you build again? Kyle Kretschman:And I said, yeah, I'm, I'm excited to build. And then last one, all these there's definitely personal considerations here too. And Spotify just really did a great job showing how the company as a whole has Swedish cultures and values. And at the time I had a nine month old and they said, this is a great place to come be a father with the balance and that, and I said, all right, let's make the jump and come to Spotify. And so now I've been here about two years. So cuz I, I actually went to Spotify in may of 2020. Scott Cunningham:So remind me again, your job title at Spotify. Kyle Kretschman:So I'm head of economics. Scott Cunningham:Is, is that the, is that, is that like chief economist? I, I feel like I see different, different job titles and I don't know exactly what, what everything, Kyle Kretschman:Yeah. It, it it's on the path to it. So I'm, I'm the highest ranking PhD economist at Spotify. Scott Cunningham:I see. Okay. I've been there for two years. Okay, go ahead. Sorry. Kyle Kretschman:Yeah. Cause again, that's what I was brought into build was to build, like we did at Amazon was overall integration of PhD economists within the different business units. Scott Cunningham:So this is the part I'm, I'm having some hard time, like, you know, putting, visualizing or putting in my own words. What exactly will it look like if you have been successful in five years at that goal and what would it look like if you had been a complete, complete bust? What are the two things that are like empirical that I would be able to, to observe? Kyle Kretschman:Yeah. A complete bust is probably that an economics discipline is not, is not part of Spotify and there's not, there's not a job family. So a complete bus would've been, I, I moved to Spotify, an economics discipline. I either in, or I'm working data science job, what success looks like is actually what we put first from a, so I'll talk about the people in process, discipline success. We, I came into was Scott Cunningham:Real quick. So Kyle Kretschman:Foundation on basically. Yeah. Scott Cunningham:So, so failure actually would mean that the economist community within Spotify just never materialized, is that what you're saying? And that, and that means like this, having groups of economists that, that think and use the kinds of training we had in graduate school, but in a way that is actually productive in the firm is, is that, is that right? Kyle Kretschman:So, so yeah, and again, that's, Scott Cunningham:The job is successful if you're able to actually create internal demand for economists. Kyle Kretschman:Yep. That's right. And that's, that's what I would say against from the process side. And then from the product side, that's using econometric research in the ways that I've been talking about it's using it both not only for individual analysis, but also then building econometric measurement systems that improve the product to get towards Spotify's mission of, of billion listeners and fans who can connect with over a million creative artists who are making a living. So that's, so it's a combination, it's the combination people process. Do we have the people set up? Do we have this integrated system of economists working alongside all these different types of stakeholders along with the product side of, do we have these measurement techniques that we're applying in a way that is important to Spotify's not only Spotify's business, but all the stakeholders that have an interest in Bon life. Scott Cunningham:So I feel like, you know, I think to academics that, that, and, and maybe even to some degree students, maybe I'm, maybe I'm completely an outlier here and I'm wrong, but you know, I think there's this like really shallow is a negative word. It, I mean, shallow, literally more and just like, it's just the thinnest knowledge possible of what exactly, you know, the, the, the core skillset of a successful economist is in tech. You know, and for many people they think, I think they, they think it's such a primitive level. They're like, it needs to be somebody that can code, you know, it's a data scientist, but, but it, but it, but that's not what I associate with economics. Right. So what would you, what would you articulate? It is, Kyle Kretschman:So it's the ability to do econom applied econometric research. That's applied to business problems. Mm. So within that is coding. Yes. Scott Cunningham:Right, right. Within that is coding. Kyle Kretschman:I, the vast majority, I won't say everyone, but the vast majority of tech economists are gonna have some level of coding and maybe they're not coding anymore. Like I'm not doing any coding anymore, but like they, they have that ability. So that's just again, that's, that's a skillset, but the real ability is doing long-term economic research. Because the questions that we get asked are very hard and difficult, and they are maybe in the academic setting, maybe they are publication worthy, takes that take three years, four years to actually solve with the right model. Yeah. But it's the ability to take that three year research roadmap and make it progress. So when you're doing that, you need to have your summary statistics that the business can see, understand, give feedback on because that accelerates the research process and also accelerate the business impact. So, one, I guess one comparative skillset that I've learned is what I call research with an open book. You shouldn't really go a month without talking to your stakeholders. You should be showing where the research is. You need to be the person who owns that three year measurement roadmap, but you're the person who's gonna be having to take the feedback at a consistent basis. And that's, that's really the different part. So, but again, that goes back to the applied econometrics part Scott Cunningham:Of it. So what is the, so walk me through your, walk me through typical days in your job now. Kyle Kretschman:So my typical day is much more generally now on the management people side. So it's definitely going to be, as I said, building this discipline because we have the creator economics team who's focused on the supply side. We spun up the ads economics team that has a completely separate unit again. So that's like part of the people process, but then it's creator economics team has a hybrid data science and economics team. That's doing all these things that I'm talking about. So with that right now, you know, we're working on long term roadmaps for both product and research and being there. And then also giving I say, honestly, translating a lot of the work that the team is doing into the actual business decision making. So, so I kind of work if you think of the research process. And I think this is kind of generally true, most managers or at least tech economist managers, they're working at that hypothesis generation and then communication of results where then the team itself is working with the data analysis, the statistical models along that research loop to integrate. So it's more of an, my, my role really is an integration of being a translator of economic ability and measurement into the business. Scott Cunningham:So, you know, so you've clearly have a comparative advantage evidence by, you know, your, your successful Mo creating of a career. Right. And, but I was curious, what do you think your, what do you think that is? You know, how would you explain that to like, you know, like on a subreddit, explain to me like I'm five, how, how would you like explain, explain to me what you see as your compared to advantage now in life, you know, that you, that you, that you have the, that your presence as this value added for the firm. And, and was that now that you look back, when you think about who you were as a young person back at UT, or even at pet, do you see signs of that and what, what were the signs Kyle Kretschman:The, yes I do, because really my comparative advantage now is as I mentioned, I seek variety and specifically I seek variety on really hard, interesting problems. Mm. So that's, that's gonna be it, you can abstract away from the economist role. You can abstract away anything else, but I like to work on things that are hard to solve, which is that's gonna be valuable in so many different contexts. And then specifically in the economist frame, again, we're, we're probably comparing to other economists, I'm much more of a variety speaker. So I wanna work on a variety of problems. And I mean, you're Scott Cunningham:People that are not like that. Kyle Kretschman:I mean, right. So there would be people who are gonna be much more focused on changing the methodology for, I mean, Scott, you work in, cause inference, you tell me, what are you, what are you working on? That's gonna be changing some sort of estimator. Right, right. That's that just doesn't hold that much appeal to me, I'll say it. What does appeal to me is this entangling a problem, making it less, making a super ambiguous problem solvable. Right. And then having people dive in and solve that. So, yeah. Yeah. I guess the Reddit theme variety speaking on ambiguous problems, Scott Cunningham:It's funny, you know, like maybe you heard at some point in your career, I'm just guessing someone, or I could imagine someone saying you're not focused. Kyle Kretschman:Yeah, I Scott Cunningham:Did. And, and in fact that might have been true and really not relevant. Right. Well, Kyle Kretschman:It was probably, it was probably relevant feedback to certain end goals. But again, once I, once I realized what the end goals were, it's all of this to tied them all together, all of this led up to where and economi and tech is exactly where yeah, I was, I was meant to be helping quote. Scott Cunningham:Right. Right. Well, you know, I think like one thing that I, I feel like is really positive about the, the growth of the, of tech. And in many ways I see now at universities are now in direct competition with tech and I, and I wouldn't, I don't think that was true 50 years ago that maybe, maybe universities were in competition with government. I don't even know if that was exactly like it is now, you know? And it, it seems like what I see with you is, is a person who, who, who took ownership of their life. Right. And just sort of said, I don't wanna put words in your mouth, but it, but it seems like one of the things that it's like, you know, I think young people can be encouraged by or should hear is that, you know, here's a person that took ownership of their life. This was their career. It's the only life they had. This was their career. And they, they, they had the fortitude and the resilience and the, you know, to, to be themselves and make a career. Is that true? You think, Kyle Kretschman:I appreciate you saying that. And it's probably true again, I kind of like hesitate because again, during, in my twenties and the searching phase, it definitely didn't feel that Scott Cunningham:Way. Didn't feel that way. Kyle Kretschman:Right. But now in the decade of my thirties, yeah, it did. Because, but I was, I was more than willing to explore different avenues. Yeah. Because that's what, because I wasn't, I wasn't willing to settle, I guess I'll say that, like, I wasn't willing to settle for something that I didn't feel was the right fit for me. Yep. And so that's, that's what a big driver that's been able to help me get here for Scott Cunningham:Sure. Yep. Yep. I think that, I think the, the truth is economics is a really valuable PhD and it supports a, it, it supports many personalities with it and it, but you know, like I do think sometimes one of the skills of that, of people that are in that stage of searching is just to be able to be, to wait. Yeah. You know, you're like, so you don't always know that you're planting seeds and, and you know, you can't harvest seeds when they're seeds, you know, you can't harvest plants when they're seeds, you have to wait. Yeah. But you have to be disciplined and continue to water it too, though. Kyle Kretschman:Yeah. And also be, be willing to take some risks once in a while Scott Cunningham:And willing to take some calculated Kyle Kretschman:Risks. But yeah. Specifically like for tech economists, now there is a lot more information. And I will say probably five years ago, I probably cautioned some people from going to get their PhD in economics if they wanted to be a tech economist, because it was still, you know, the majority of people who jumped into it were risk takers, willing to do that. There wasn't this. But now I believe the head of NABE has said that there's over 1500 economists in tech. Now we have a great conference that we spun up while we're, while, while I was at Amazon, that I was a part of that now NABE tech in November. So if people wanna learn, now, the ability is there. So in grad school, like come to NA tech, come see, come look at the program, come see exactly. Cuz it was, it's a conference that's designed by tech economists for tech Scott Cunningham:Economist. Yeah. Susan mentioned it too. A B E national association, business economics. I actually, it's a huge conference and I actually don't think in academia, a lot of people know about it. Kyle Kretschman:Yeah. That's cause Scott Cunningham:We just know about the as SSA. Kyle Kretschman:Right. And that's one of the reasons I kind of wanted to plug it here, honestly. Cause again, it, it, it started out probably 50 people. I forget what year, but now it's up to, I think they're expecting over 500. And so that is, so some of this discovery is searching that if people, if people who are in their PhD wanna learn more about, that's gonna be the best place. Scott Cunningham:I only have two more questions and then I'm gonna let you go. One of 'em is kind of touchy, feely, but okay. So, so, and I, and I I'll just admit, as I ask this question, you're gonna immediately know, I must think this way. And so that does not mean that you think this way, but you know, I love being an economist. I love economics. I love being an economist. And if I was to go get a job and the job title said, didn't say economist on it. I would be, I would feel, I, I can already tell that would be something I would have to work through. Cuz I feel so connected to the tribe. And I was just wondering, you know, do you think that economists and tech feel connected to the tribe? Kyle Kretschman:It, it gets to which tribe Scott Cunningham:Do you think they feel like a part of the broader American economics association? Kyle Kretschman:My I'll say my personal view is probably not because that's not the colleagues that we've grown up with over the past 10 years. Do we feel part of the tech economist community? Yes. There is one there very much is one. And do we feel part of that tribe very much. Do I feel part of the academic economist tribe? Probably not. Yeah. But to tie it all together, like do I feel like I'm an economist? Yes. Yeah. A hundred percent. Yes. And again, we're making differentiations probably for your audience, but in the grander scheme of things, if you look at, if you look at anybody in tech, who's an economist they're gonna to, to non they're, they're economists Scott Cunningham:They're economist. Right? They, they, they, they, they think different. Kyle Kretschman:Yeah. I mean, I'll give a quick anecdote, like one of my, one of my good friends and colleagues at Amazon Neil go, she's not there anymore. His wife and my wife just would always be like, they just respond to the, the question in the exact same way. Scott Cunningham:Yeah. Right. Kyle Kretschman:So that's like, they very much are just so rational with their responses in general, you know, there's exceptions, but they, they would compare notes and we'd have the same general view and outlook on personal finances. Let's say so. Yeah. But I think what you're heading at is, is their different tribes that are growing up. Yeah. Because I think there's probably always been different tribes. And so tech economists is a tribe that is, I mean is more than growing up as again, we think there's probably hook going on 2000 of them and there's very strong connections within that. Scott Cunningham:Yeah. I guess I wonder sometimes I'm like, what's the point? The point is community. The point is not for the, the goal is not for the AA to have everybody. The goal is for, you know, I think the goal is for people to thrive and feel connected to their broader, to a broader community. And if that's grown and ly in tech, that's good. I just always kind of think to myself, like, but at the end of the day, the universities are staffed by faculty in the AEA and it, it just seems like, and so it's like, it, it seems like at the graduate training, you know, many people can feel job dissatisfaction because they know they're not gonna become an academic, you know? And that, that can be a, that might be a source of some of the struggles. There's a new paper in the general economic literature about mental health struggles amongst PhD students and economics. Scott Cunningham:And there would, you could, if you read between the lines, if you read between the lines, there was a real disconnect between them and their advisor. They, the, the, you know, you don't know causality, cuz it was just a survey, but like, you know, the, the students were like, they just seemed to have job dissatisfaction. They didn't think their work mattered. You know? And, and I just kept thinking in my mind, you know, do they know about all their options? Are any of those options stigmatized? You know, because like, if you are on, if you've held it up to students that the only meaningful career that you could possibly have is at a, is tenured at a, at a university writing papers that get into top fives and you, and you sense inside your heart, you know, that just, I don't think that that's, that's not doing it. Kyle Kretschman:Yeah. I think it ties together what you said earlier that economics welcomes a whole bunch of different viewpoints of world views, but maybe doesn't reward the diversity of the way it should in academic. And so like from a diversity viewpoint, these are different career paths, but even more shows should welcome more ideas. So like that's probably where the economics field needs to continue to lean in is with the diversity to enable people, to not to know different paths, to raise up people on different paths and have those opportunities. And I think tech is providing a huge pathway for Scott Cunningham:That. Yep. Yep. I do too. I am very grateful for it. It seems really exciting. It seems like a world of just like working on I'm sure that the way we think about, you know, gossip at Guinness coming up with, you know, students tea of this story of a, of a person in industry making these major contributions, we're just at the beginning of, of that, you know, with just the sheer volume of, of social scientists in tech, working on fascinating important topics. My last question, what is your favorite paper in economics or no, let me say it this way. What is a paper in that has, that has, for some reason stuck in that has stuck around in your head Kyle Kretschman:BLP so yeah, it, it has, because that was probably the first time that I felt like the wow moment of we can, we can estimate substitution effects with observational data. Don't need experiments. We have this method that obviously has proliferated into so many different ways and that stuck with me because throughout all my stuff, I've generally been super in substitution effects. So yeah. Yeah. I would put that in that research thread. Scott Cunningham:Yeah. I, I, I don't think that that's that you having that empirical IO structural, you know, having written in your head. Kyle Kretschman:Yeah. Right, Scott Cunningham:Right. That's great. Well, it is a real pleasure to, to get to walk through. I appreciate you walking me through your, your life and tell me about your, your career. It's really nice pleasure to meet. Kyle Kretschman:Yes. Thanks Scott. I really appreciate the.

I first met Peter Arcidiacano, professor of economics at Duke University, while I was a PhD student at the University of Georgia and I have followed his work since from a distance. I originally followed Peters work because he’d written several articles about sex from a two-sided matching perspective. I was struck by the fact that we both saw thinking about sexual relationships in terms of a matching problem. Two sided matching perspectives focus on the assignment mechanisms that bring people together, and when it comes to sexual relationships, the relative supply of possible partners and competition for those partners will in equilibrium result in pairings, some of which may become the most life sustaining and defining partnerships of those peoples lives. Peter’s work was gratifying to read, and I have often looked up to him for his successful merging of theory and econometrics to study topics I cared about.

The economic way of thinking is not about topics, nor is it is not about data, even though economists tend to have particular topics they study intensely and use data usually to do so. The economic way of thinking does though typically involve careful study of allocation mechanisms, such as prices and markets, that bring the productive capacity of communities into existence. These things are important as they animate humans to work together, produce output that manages the production itself, and increasingly towards the end of history, left surplus for humans to enjoy. Who ends up in what activities doing what types of specialized work ultimately shapes that which is made, how much and how it is distributed. The allocations end up not only shaping our lives, but our children’s lives. Starting conditions can cast a long shadow lasting centuries even causing certain groups to creep ahead as more and more of the surplus mounts and accrues to them, while others watch as a shrinking part of the growing pie flows to them.

In the United States, in the 21st century, one of the key institutions in all of this assignment of love and commerce has been the university. And within the university system, there are gradations of institutional pedigree and at the top of the pack sits elite institutions whose students seem practically destined to shape and receive the surplus. Given the path dependence in wealth, and how it has interacted with race, it is therefore no wonder that policymakers and economists have for decades sought to refine the rules by which schools can select high school applicants for admission. In many ways, our country’s fight over the use of race in selecting students into college is the old debates about capitalism and the self adjusting market system writ large.

So it’s in this broader context about work, schools, matching and allocation mechanisms that I think of Peter and his scholarship. When I review the range of topics on Peters vita, I see the signature marks of the modern 21st century labor economist. Someone interested in markets and how they work to connect people into productive cooperation. Someone interested in institutions, someone concerned about inequality and discrimination, someone versed both in economic theory and econometrics, someone at home with a bewildering array of numbers in a spreadsheet. To me, it is natural that Peter has pivoted so fluidly between topics like sex, work, discrimination and higher education because in my mind these are all interconnected topics concerning the assignment mechanisms we use in America to organize society and maintain our collective standard of living.

I invited Peter on the Mixtape with Scott as part of an ongoing series I call “economists and public policy”. The series focuses on how economics and economists think about and attempt to shape public policy. It includes people with a variety of perspectives, and even some who are critics of economics itself. Previous guests on the podcast in the “economists and public policy” series have been Sandy Darity, Elizabeth Popp Berman, Anna Stansbury, Mark Anderson, Alan Manning, Larry Katz, Jeremy West and Jonathan Meer.

Peter has not only produced academic articles in some of economics’ most impactful outlets — he has recently served as expert witness in two major discrimination cases, one of which put him on the opposite side of the stand as David Card, winner of 2021 Nobel Prize in economics. You can read about the cases here. They involve the broader topic of race and affirmative action at universities. The cases more specifically involve whether Harvard and UNC Chapel Hill admissions criteria show signs of discrimination.

One of the things about Peter’s involvement as expert witness that I want to highlight, though, is that his expert testimony was, at its core, an example of the role that econometrics can play in the shaping of public policy. It is more and more the case that economics’ role in the shaping of public policy in the 21st century will involve not merely economic theory, but also statistical analysis of complex datasets too, and I think it is worth pausing and noting that the economist shapes public policy oftentimes these days as much through interpreting data as her counterpart did using pure economic theory. I hope you find this discussion with Peter thought provoking and informative about both his work on these cases, but also about the role of economics and econometrics in forming public policy. But I also hope that the interview will give you a deeper insight into Peter and who he is.

Scott’s Substack as well as The Mixtape with Scott are supported by user subscriptions. Please share this episode to people within and outside economics that you think might be interested. I love doing these interviews and using the substack to do deeper dives into econometrics and the lives of economists and if you find this work valuable, please consider subscribing and supporting it.

Scott's Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Transcript

Scott Cunningham:

In this week's episode of Mixtape: the Podcast, I had the pleasure of talking with a professor at Duke University, named Peter Arcidiacono. I can never pronounce it correctly, no matter how many times I try. I first met Peter in graduate school. He was, probably then, an assistant professor at Duke, where he has spent his entire career. I was a PhD student at the University of Georgia. And he had a research paper on a topic that I was also working on, involving marriage markets. He's been an incredibly prolific producer in the area of labor economics and education, as well as affirmative action. And he uses tools in econometrics, that I largely never invested in, structural econometrics and discrete choice modeling. So when I read his work, I usually do it, both, because I'm interested in the paper and the paper topic, but also because I'm hoping that this will be a chance for me to open my mind a little bit more, and pick up on some of that econometric modeling, that I lack.

Peter is also an expert witness in a high profile case, right now, involving affirmative action and racial discrimination at Harvard University, and the University of North Carolina Chapel Hill, both of which have been combined into a single case. As I understand it, it's going for the Supreme court soon. In this interview, we walk through a lot of big and small issues around society's preferences around poverty, inequality, as well as the role that higher education is playing in both. My name is Scott Cunningham, and this is Mixtape: the Podcast.

Scott Cunningham:

Okay. This is great. I don't know if you remember. So this is an interview with professor of economics at Duke University, Peter Arcidiacono. And we're going to be talking about a range of topics. But just to give the reader and the listener a little bit of background, Peter, could you tell me a little bit about yourself, and what your involvement is with a current case, going before the Supreme court, involving University of North Carolina and Harvard University?

Peter Arcidiacono:

Certainly. And thanks for having me on. I've been at Duke now, for over 20 years. This was my first job out of grad school, and stayed here ever since. And a lot of my work has been on higher education, both with regard to choice of college major, as well as affirmative action.

And one of the really dissatisfying things about working on affirmative action, is that universities hide their data. So you can't really get a good sense of how the programs are working, because you typically don't have the data. And I think that that really matters, because to me, so much of the discussion about affirmative action, is in the binary. Either we have it, or we don't have it. But what it means to have it, is something, as economists, we would think about, that's something we would be optimizing over. And so, there's really a large space between race as a tiebreaker in admissions, and what somebody like Abraham Kennedy would advocate for, which would be more of a quota system.

And so, thinking about where you stand on that, to me, I had this opportunity to work on these two cases, two lawsuits. One brought against Harvard, and one against UNC, on the role of race in these admissions processes. And for me, it was an opportunity to look behind the veil, and see how these programs actually operated.

My intent was always to, a feeling as though, if I'm going to be an expert on affirmative action, I should know how these processes actually work. So my intent was always to use this for the purposes of research, as well. And we've written a number of papers out of the Harvard case. Four have been accepted now, and we just released a fifth one on racial preferences of both schools. And we'll see what happens with that. So those lawsuits, I testified in trial, at both those cases. My counterpart in the Harvard case was David Card, who recently won the Nobel Prize. I was wondering how I would respond to that. And actually, my response, I got to go up against a Nobel Prize winner.

Scott Cunningham:

Yeah. Right. Right.

Peter Arcidiacono:

So those experiences were somewhat traumatizing. But both experts, David Card and Kevin Hoxby, are pillars in the field, and people who have been very helpful to me, and who I have a great deal of respect for.

Scott Cunningham:

Yeah. Yeah.

Peter Arcidiacono:

So these cases have now, in both those cases, the side I was on lost at the first round. In the Harvard case, they also lost at the appellate round.

Peter Arcidiacono:

In UNC, it didn't actually go through the appellate round, because-

Scott Cunningham:

Oh, so-

Peter Arcidiacono:

... supreme court merged the cases.

Scott Cunningham:

... Both the Harvard University case and the Chapel Hill case, were already decided, but not at the Supreme Court level.

Peter Arcidiacono:

That's right.

Scott Cunningham:

Okay.

Peter Arcidiacono:

So the decision was appealed. It's now before the Supreme Court.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

I think the Supreme Court scheduled here, arguments in October, and then, we'll see when they release a decision.

Scott Cunningham:

Okay. So, and these are both cases involving affirmative action and racial discrimination amongst particular groups of people? Is that groups of students, is that right?

Peter Arcidiacono:

That's right. Though, in the UNC case, there's actually no claim of Asian American discrimination. So that actually, you only see that at Harvard. You don't see that at UNC. That doesn't mean, I think that Asian discrimination is unique to Harvard.

Scott Cunningham:

Right.

Peter Arcidiacono:

I think it has to do with the fact of there not being that many Asian Americans in North Carolina.

Scott Cunningham:

North Carolina, right.

Peter Arcidiacono:

It's always been a bigger issue at the very top schools.

Scott Cunningham:

And you were called in, as an expert witness, for the plaintiff in both of those cases.

Peter Arcidiacono:

That's right.

Scott Cunningham:

Right. So David Card is the expert witness for Harvard, representing Harvard, against an accusation of, well, what exactly is the accusation against both of these institutions, and who brought these accusations against them?

Peter Arcidiacono:

So the group is called Students for Fair Admissions. And they basically got groups of students to, as their plaintiffs. Though, it's not about those particular students, in terms of remedies. And in Harvard, there's three claims. One, whether or not they're discriminating against Asian American applicants, relative to white applicants.

Scott Cunningham:

Mm-hmm.

Peter Arcidiacono:

Two, whether the size of the preferences given for underrepresented groups, is constitutional.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

And three, whether there were race-neutral alternatives that they could have used. So the Supreme Court has said, "If there is a race-neutral alternative, you should use that."

Scott Cunningham:

Okay.

Peter Arcidiacono:

I'm not really involved at the race-neutral part. We had a different expert for that aspect.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

Though, in both cases, Card and Hoxby actually did the race-neutral part, as well.

Scott Cunningham:

What exactly does the constitution say a admissions committee can use, when drawing up a student cohort?

Peter Arcidiacono:

Well, so I'm not sure what the constitution has to say on it, but I can say what the history of this of the court challenges have been.

Scott Cunningham:

Okay.

Peter Arcidiacono:

So I think, it's Title VI of Civil Rights Act said, "You're not supposed to use race-"

Scott Cunningham:

Race.

Peter Arcidiacono:

"... in these types of things." And there are other categories too.

Scott Cunningham:

Okay.

Peter Arcidiacono:

But race is the focus of this one. Now, the reason they had that, was because of the history of ill treatment of African Americans.

Scott Cunningham:

Right.

Peter Arcidiacono:

And this is obviously going in the other direction-

Scott Cunningham:

Mm.

Peter Arcidiacono:

... with regard to African Americans receiving preferences in the admissions process.

Scott Cunningham:

Mm. Mm.

Peter Arcidiacono:

So, but then, the history was that the original decision, the Bakke case, said, "Look, you can't use race in admissions, because of reparations. You can only use it because of the benefits of diversity." So the state can have an interest in diversity. And that was a compromised position to get that swing justice, to sign onto it.

Scott Cunningham:

Mm.

Peter Arcidiacono:

Since then, there have been a number of cases. I think the ones that are most relevant right now, are the ones that came out of the Michigan cases.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

And there was one at the undergraduate level, which they found that you could not use race as part of an explicit point system.

Scott Cunningham:

Mm.

Peter Arcidiacono:

So you can get points for having a good SAT score, points for being a particular race, you add them up together, then you could rank the-

Scott Cunningham:

I see.

Peter Arcidiacono:

... applicant.

Scott Cunningham:

So there were schools that were doing a point system based on individual characteristics, like race. And that was, at that moment, it was unclear whether that would be legal. It was, I guess, or was it something that schools were, potentially, in a legal, bad situation, when they were using it? Or was it just not known?

Peter Arcidiacono:

I don't think it was clear. And that's where the court ruled. You cannot use it in that way.

Scott Cunningham:

Got it. Okay.

Peter Arcidiacono:

At the same time, there was a case against Harvard's Law School. And on that one, they said that you could use race, holistically. As an economist, I can express anything as a formula. And then, the question is, whether you see all parts of the formula or not.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

So it gets a little tricky. And I think that, from my perspective, I would've rather had the ruling go in the exact opposite way.

[inaudible 00:11:59] on if we're going to find in favor of one or the other.

I would prefer a point system to a holistic one, because then, everything's clear.

Scott Cunningham:

Clear. Yeah. It seemed really precise-

Peter Arcidiacono:

[inaudible 00:12:09], to hide their data.

Scott Cunningham:

... Yeah. It seems like lots of times with the law, the imprecision of this language, as though it's a solution to the problem, is really challenging for designing policy.

Peter Arcidiacono:

I totally agree. Yeah.

Scott Cunningham:

So, okay. I want to set up the reader a little bit, oh, the listener, to know who you are before we dive into this, because I'm loving this thread, but I want people to know who you are. So before we get more into the case, can you tell me where you grew up, and why you got into economics? Your first, what was the touchstone that brought you into this field?

Peter Arcidiacono:

So I grew up in the Pacific Northwest. My first set of years were actually in Ellensburg, Washington, which is a town of 13,000. My dad was a math education professor.

Scott Cunningham:

Oh, okay. What university was he a professor at?

Peter Arcidiacono:

Central Washington University.

Scott Cunningham:

Okay. Okay.

Peter Arcidiacono:

And then-

Scott Cunningham:

Hey, but what'd you say it was? What was it again?

Peter Arcidiacono:

... It was math education.

Scott Cunningham:

Math education.

Peter Arcidiacono:

Yeah. So he was teaching teachers how to teach math.

Scott Cunningham:

Oh. So you've always been, it's in the family to be interested in education?

Peter Arcidiacono:

Yes. And-

Scott Cunningham:

And even this math education part. That's another way of describing an economist that studies education.

Peter Arcidiacono:

... Right.

Scott Cunningham:

Math education.

Peter Arcidiacono:

Well, my parents actually met in linear algebra class, so.

Scott Cunningham:

Oh, that's romantic.

Peter Arcidiacono:

And I've got two brothers, and they were both math majors.

Scott Cunningham:

Oh, wow. Okay.

Peter Arcidiacono:

I was the only non-math major.

Scott Cunningham:

Okay. Okay.

Peter Arcidiacono:

But I came into college, and started out in chemistry. I think, Econ PhD programs are filled with former, hard science majors.

Scott Cunningham:

No joke. Yeah, yeah. They hit organic chemistry, and then, they changed their major.

Peter Arcidiacono:

Right. And I just couldn't stand the lab. It was too social. And one of my good friends, a guy who ended up being the best man at my wedding, was a couple years ahead of me, told me I should take an economics class.

Scott Cunningham:

Mm.

Peter Arcidiacono:

And it was amazing. I think that just the way of thinking, just worked naturally for me.

Scott Cunningham:

Well, so when you say way of thinking, the way of thinking that was, can you tell me what your 19 year old self would've been jarred by? What are the specific things, that economic way of thinking, that he was noticing?

Peter Arcidiacono:

Well, it just fit with a lot of how I operated. So I view economics as a great model of fallen man.

Scott Cunningham:

Uh-huh.

Peter Arcidiacono:

Fundamentally, I was the guy who always looked for the loopholes. So responding really well to incentives. I had a keen eye for how I could game the system.

Scott Cunningham:

Right. Right.

Peter Arcidiacono:

And so, I think a lot about what economics is doing, is the dismal science, right? The reigns on the parade of well-intentioned policies.

Scott Cunningham:

Right. Right.

Peter Arcidiacono:

How are people going to get around the policies? Well, that's where I lived, was figuring out how I could game the system.

Scott Cunningham:

Right. Right. Right. So you were, this idea of that rational choice paradigm, is that what you mean?

Peter Arcidiacono:

Yeah.

Scott Cunningham:

And that-

Peter Arcidiacono:

Yeah.

Scott Cunningham:

... that people would just simply, if they have goals, those goals don't just go away with a policy. They might just continue to try to achieve those goals at lowest cost, even then.

Peter Arcidiacono:

Exactly.

Scott Cunningham:

Right. Right.

Peter Arcidiacono:

And the other studying thing, which I think, really affected why I ended up doing the research that I did, was, for me, the chemistry classes were just way harder-

Scott Cunningham:

Uh-huh.

Peter Arcidiacono:

... than economics classes.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

And I'm not trying to say that any classes are easy.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

But there is definitely large differences-

Scott Cunningham:

Mm-hmm.

Peter Arcidiacono:

... in every university, and what the expectations are-

Scott Cunningham:

Mm-hmm.

Peter Arcidiacono:

... across fields.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

And that distorts people's behavior.

Scott Cunningham:

Mm.

Peter Arcidiacono:

So I view it, that most colleges are subsidizing students, to go into low paying fields. And how do they subsidize them to do that? They offer higher grades-

Scott Cunningham:

Mm.

Peter Arcidiacono:

... and lower workloads, smaller class sizes. All those things work, so that lots of people come in wanting to major in well-paying fields, and switch in, and switch out.

Scott Cunningham:

Right. Right. Right.

Peter Arcidiacono:

And they do so because of the incentives the universities provide.

Scott Cunningham:

Yeah. So you got interested in economics, and that's like, you're describing some sort of price theory, microeconomics. But you've also have made a career out of being such a strong econometrician in this area of structural econometrics and discrete choice modeling. How did you get interested in those topics? What was your first reaction to econometrics?

Peter Arcidiacono:

I had a very strange econometrics background. So my first year econometrics, was taught by Chuck Manski.

Scott Cunningham:

Oh.

Peter Arcidiacono:

The whole year. And so, it was lots of bounds.

Scott Cunningham:

Uh-huh.

Peter Arcidiacono:

And then, my second year, it was all John Rust.

Scott Cunningham:

Mm.

Peter Arcidiacono:

So a complete swing, right? So you go from the non-parametrics, what can you identify under the smallest number of assumptions?

Scott Cunningham:

Mm.

Peter Arcidiacono:

To what can you identify, if you want an answer something really big.

Scott Cunningham:

Right.

Peter Arcidiacono:

You got to make a lot of assumptions to make that.

Scott Cunningham:

Oh, boy. That's an interesting journey, right there.

Peter Arcidiacono:

So I actually never had the mostly harmless econometric-

Scott Cunningham:

Right. Right.

Peter Arcidiacono:

... at all.

Scott Cunningham:

Yeah. Yeah. Yeah. Yeah.

Peter Arcidiacono:

And the econometrics has always been-

Scott Cunningham:

This was Wisconsin?

Peter Arcidiacono:

... That's right.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

That's right.

Scott Cunningham:

What year was this?

Peter Arcidiacono:

In the econometrics, the advances were always more, because I needed to do something to estimate my models.

Scott Cunningham:

Right. This was mid nineties? This would've been the mid nineties, or late nineties?

Peter Arcidiacono:

I'd like to say late nineties. Yeah-

Scott Cunningham:

Late nineties? Okay. Yeah-

Peter Arcidiacono:

... [inaudible 00:19:10].

Scott Cunningham:

... Yeah. Yeah. Yeah. Yeah. Okay, keep going. Sorry.

Peter Arcidiacono:

So I was thinking about my own experience, in terms of choosing a college major, and thinking about, Well, people are learning over time. They start out those STEM classes, and figured out, wow, this is a little bit harder than I expected.

Scott Cunningham:

Yeah. Yeah.

Peter Arcidiacono:

And then, moved through.

Scott Cunningham:

Right.

Peter Arcidiacono:

So I had a mind, I actually had the idea for my job market paper, my first year. And had this idea of a forward looking model, of how people choose their college major.

Scott Cunningham:

Mm-hmm.

Peter Arcidiacono:

And so, then, I go into John Rust's office, because he's my second year econometrics professor, and was describing this problem to him, that people are making decisions today, giving expectations about the future.

Scott Cunningham:

Mm-hmm.

Peter Arcidiacono:

And he says, "Yeah, I think I can help you with that." And I was like, "No, you don't understand. This is a really hard problem." And of course, John Rust had written the [inaudible 00:20:13] paper about how to estimate these types of models-

Scott Cunningham:

Right.

Peter Arcidiacono:

... And he was fantastic with me. [inaudible 00:20:20]. He didn't say idiot. You could at least look at what I do, before you come to my office. He was fantastic with me.

Scott Cunningham:

Yeah. Yeah.

Peter Arcidiacono:

And actually, the funny story about that too, is he's actually the only reason I'm an economics professor, because-

Scott Cunningham:

Oh, yeah?

Peter Arcidiacono:

... I only got into one grad school. Got rejected from much worse places in Wisconsin. It was the only place that accepted me.

Scott Cunningham:

Mm.

Peter Arcidiacono:

And the joke was that that was the year John rusted everybody in. So there were 53 of us to [inaudible 00:20:57].

Scott Cunningham:

That's awesome.

Peter Arcidiacono:

17 got PhDs.

Scott Cunningham:

Wow.

Peter Arcidiacono:

And if you look at another guy, one of my friends, I just actually found out we were actually at a conference in honor of John Rust, this past weekend.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

And it turns out, that was the only place that admitted him, as well. And he's been incredibly successful too.

Scott Cunningham:

The John Rust fixed effect is filled with stories.

Peter Arcidiacono:

That's right.

Scott Cunningham:

That's really cool. That's really cool. I'm curious, thinking about what your, I want get to the Harvard and the Chapel Hill. But before we move on, you could imagine, had you gone to Princeton, or MIT, and worked with, or Berkeley, and worked with these, the treatment effects guys, like Imbens, and Angrist, and Card, and Kruger, and O'Reilly, and all these people. It's not just that your knowledge of econometrics would've been slightly different. Even the kinds of questions, that you would be asking, might be different. So I'm curious, what do you think your training and structural, under Manski and Rust, how has that shaped, not just the way you do your work, but even the types of questions that you ask, that you imagine, you might not have asked? For instance, just even thinking, modeling choice-

Peter Arcidiacono:

[Inaudible 00:22:40].

Scott Cunningham:

I'm sorry. I don't know. Did I lose you?

Peter Arcidiacono:

You froze on me.

Scott Cunningham:

Ah, I froze? Okay-

Peter Arcidiacono:

You're still frozen.

Scott Cunningham:

... I'm still frozen? Okay. There. Okay.

Peter Arcidiacono:

Now, you're back. So you're asking about what types of questions.

Scott Cunningham:

Yeah. What kinds of questions do you think you ended up being really interested in, and working on? Not just the model that you wrote down, but even the actual topics. Because I'm curious, I'm wondering if listeners could really frame their understanding of this structural, versus this causal inference, tradition. Not just in terms of the technical pieces, but like this is practically how, the work a person ends up, that you think you ended up doing, versus if you had got Angrist as an advisor.

Peter Arcidiacono:

Oh, I think it has shaped me quite a bit. I am certain that if I'd gone to a place like Chicago, I would've probably ended up working with Steve Levitt. I am naturally attracted to some of those topics, that are more of a freakaconomics-type nature. And if you look at it, we actually had competing papers-

Scott Cunningham:

Yeah.

Peter Arcidiacono:

... on discrimination in the Weakest Link game show.

Scott Cunningham:

Uh-huh. Yeah.

Peter Arcidiacono:

And I've written a couple of sports papers. So I have that in me, to think about those types of things. If I'm-

Scott Cunningham:

Topics, right? Right.

Peter Arcidiacono:

... Yeah.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

I think that the Manski Rust combination did have a big effect on me, and, in the types of questions that I asked. Which is what structural brings to the table, is thinking about mechanisms.

Scott Cunningham:

Mm.

Peter Arcidiacono:

So when you think about the effect of affirmative action on outcomes, understanding why the effect is what it is, matters.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

How it affects application behaviors. How is affects what majors issues. What would be those counterfactuals? And for that, I think you need some of these structural approaches.

Scott Cunningham:

Right.

Peter Arcidiacono:

Now, one of the things about those structural approaches, to say, typically involve making some pretty big assumptions.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

And I think that that's where the Manski influences had on me, because I also have papers that use subjective expectations data. And I think that that is actually an incredibly promising area of work.

Scott Cunningham:

Mm.

Peter Arcidiacono:

It's quite clear that people don't know as much as they should know, when they make important decisions. Certainly, higher education being a prime example of that.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

COVID really makes that clear, you know? How can it be that the people who are unvaccinated, are least likely to wear a mask? Clearly, they're operating under very different beliefs about-

Scott Cunningham:

Right.

Peter Arcidiacono:

... what's going to happen.

Scott Cunningham:

Right. Right. Right. Right. Okay. So let's move into this Harvard Chapel Hill project. So setting it up, tell me, what is the first event that happens, that makes this a case against Harvard? Not counting alleged discrimination, but the actual historical event, that leads to a need for an expert witness.

Peter Arcidiacono:

Well, I think the need for the expert witness came about, because Harvard had to release their data, in the context of the trial.

Scott Cunningham:

Mm.

Peter Arcidiacono:

So in the context of the lawsuit, the claim was there were some smoking guns that suggested the possibility, for example, of Asian American discrimination

Scott Cunningham:

That would not fit this holistic criteria, that you mentioned earlier?

Peter Arcidiacono:

Well, so, it's an interesting question, right? So you can't have with the holistic criteria, you can take race into account, but the question is whether you could take race into account, in a way that penalizes a group, relative to white applicants?

Scott Cunningham:

Yeah.

Peter Arcidiacono:

So it might be one thing to say, "We're going to give a bump for African Americans, relative to whites."

Scott Cunningham:

Yeah.

Peter Arcidiacono:

Maybe another thing to say, "We're going to give a bump for whites, relative to Asian Americans."

Scott Cunningham:

Yeah. Yeah. Right. Right. Okay. So they've had a lawsuit brought against Harvard. Harvard's had a lawsuit filed against them. What year is-

Peter Arcidiacono:

[inaudible 00:27:32]. Sorry, say it again.

Scott Cunningham:

What year would that have been?

Peter Arcidiacono:

Oh, man. I think it was back in 2015, or something like that.

Scott Cunningham:

2015. Did anybody see that coming? Or was this odd, this is just inevitable?

Peter Arcidiacono:

I think that, they were advertising for plaintiffs, students who had been rejected. So certainly, there was an intent to file such a lawsuit, for sure. And then, they had to weigh what universities to file it against. And they chose Harvard, because of the patterns on what were going on with Asian Americans. And I think UNC had more to do with the, there was some evidence in the record, from past cases, that race-neutral alternatives would work there.

Scott Cunningham:

Mm-hmm. Okay. So you get involved. How do you get selected as the expert witness? And what's your job, exactly, in all this?

Peter Arcidiacono:

So I think I get selected, I've written a couple of survey articles on affirmative action. And I view it that there are lots of nuances. So the fact that I would actually say there are nuances, as opposed to it being always good-

Scott Cunningham:

Right.

Peter Arcidiacono:

... made it attractive for them, I think.

Scott Cunningham:

Mm-hmm.

Peter Arcidiacono:

And back in 2011, there was actually a protest here, at Duke, over one of my studies.

Scott Cunningham:

Oh, really?

Peter Arcidiacono:

Yes. So that one, we were actually using Duke data, and confronting a tough fact, which is lots of black students at Duke came in, wanted to major in STEM and economics, but switched out. In exploring why they were switching out at such a higher rate, relative to white applicants.

Scott Cunningham:

Mm-hmm.

Peter Arcidiacono:

So for men, it was very extreme.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

8% of white men switched out of STEM and economics-

Scott Cunningham:

Mm-hmm.

Peter Arcidiacono:

... to a non-STEM, non-economics major. Over 50% of black males switched out. And you look at that, you think, that's a problem.

Scott Cunningham:

Mm-hmm.

Peter Arcidiacono:

And once you account for the differences in academic background, prior to Duke, all those racial gaps go away. And I think what, the path to the protest to serve in the long run. So I won't get into all details of that, but I think that they didn't believe the fact at first.

Scott Cunningham:

And what was the fact exactly, that the racial discrimination, the racial bias, the racial differences vanished, once you conditioned on what, exactly?

Peter Arcidiacono:

I conditioned on academic background.

Scott Cunningham:

Oh, I see. Okay.

Peter Arcidiacono:

Course and such like that. But I think even the original effect, they were surprised by, which was that the switch out rates were so different.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

And at that time-

Scott Cunningham:

But why is that a protest against you? What does that have to do with you, if you're just documenting facts?

Peter Arcidiacono:

Well, I think that the negative press headline said, potentially racist study says black students are taking the easy way out. And so-

Scott Cunningham:

Potentially racist study.

Peter Arcidiacono:

Potentially racist study. Yes.

Scott Cunningham:

This study was racist.

Peter Arcidiacono:

That's right. And I think that the issue, it actually makes a lot more sense now, than it did to me at the time. And economists thought this was crazy at the time. It's actually interesting, because I got attacked from people all over the country. It didn't make a major news flash, but within certain circles, it did. And actually, one of the people who wrote about it at the time, was Abraham Kendi. This was before he changed his name. He's not the, he wasn't famous in the same way that he is now. But the fact that I wasn't pointing the finger at the departments, I was pointing the finger, I think it was interpreted as victim blaming. It's their fault that they're switching out because they're not prepared. That's never how I would want to frame it. I would want, to me, this is, the issue is that you're not prepared-

Scott Cunningham:

You think you framed it?

Peter Arcidiacono:

No, I don't think so. But the way economists talk about things is different.

Scott Cunningham:

I know. I think that something, I think we're, a generous view is that we can't, we don't know what we sound like or something. I get into this a lot with my work on sex work, and I've, I work really hard to try to be very factual. And it, the use of words can be so triggering to a group of people. And I can never, I still can't quite articulate what exactly it is, in hindsight, that I, what word I used that was so wrong. But you feel like you would write that paper differently now?

Peter Arcidiacono:

Knowing that non economists would read it? Yes.

Scott Cunningham:

What would you do differently?

Peter Arcidiacono:

Well, I think, you have to be much more, when I say, it counts for the differences in switching behavior.

Scott Cunningham:

Okay.

Peter Arcidiacono:

The way other people hear that, is I'm able to explain why every single person switches their major, and has nothing to do with other factors. That's the reductionist claim against economists, as opposed to, on average, this is occurring.

Scott Cunningham:

Mm.

Peter Arcidiacono:

So I did a radio interview at the time, and one of the people on the show was a blogger from Racialicious, who was a regular on the show. And-

Scott Cunningham:

Yeah.

Peter Arcidiacono:

... I didn't really know anything about the show, going in.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

And she spent, so she got to go first, and she talked about how problematic my study was. And the way she described it, were ways that I did not think was consistent.

Scott Cunningham:

With what the the study was.

Peter Arcidiacono:

Right. And so, my response to that, really, by grace-

Scott Cunningham:

Yeah.

Peter Arcidiacono:

... was to say, if I thought that was what the paper was saying, I'd be upset too. And then, was able to pivot into, look, we're actually on the same side on this. We want black students at Duke to succeed in the majors that they're interested in. And to that point, we need to identify the barriers that are affecting that, and what resources we can provide, to make it so that that would not be the case.

Scott Cunningham:

So what are you going to say to your old, let's say you could go back in time, 10 years to that young economist, writing this paper. Without telling him exactly what specifically to say, you can only say a general principle. As you think about writing this, I want you to think about writing it in a different way. What exactly should you be? I guess, what I'm getting at, is how would you pause, what is, what pedagogically should we be communicating to young economists, about language and audience, that we haven't been doing historically, so that we are not unnecessarily tripping people up and creating confusion?

Peter Arcidiacono:

Yeah. Yeah. I think it's really tricky, because on a lot of things, it's just very hard to have a discussion where the emotions are not involved.

Scott Cunningham:

Yeah. Yeah.

Peter Arcidiacono:

So when you speak about things related to race, and you talk about things in a very matter of factual way-

Scott Cunningham:

Yeah.

Peter Arcidiacono:

... that can be heard as you don't care.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

You are not interested in fixing the problem at all. You're just explaining away why we don't need to do anything.

Scott Cunningham:

Right. Right.

Peter Arcidiacono:

And that's how, there's actually this marriage book, I really like, which is, again, I'm going to say this, it's going to come across as stereotyping. This is obviously distributions overlap, but it's called Men Are Like Waffles, Women Are Like Spaghetti. And the ideas is that men compartmentalize everything. So we're talking about this specific issue, not seeing how it relates to the broader picture.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

The advice, the marriage advice I always give now, is don't try to solve your wife's problems. That's always a mistake.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

And, but that's effectively, as economists, exactly what we do. We are working in the little waffle box.

Scott Cunningham:

Right.

Peter Arcidiacono:

Focused on this particular problem.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

And I don't know how to change that with regard to economics papers. I really try to be very nuanced in my language and such.

Scott Cunningham:

Yeah. Yeah.

Peter Arcidiacono:

Maybe in how you motivate the paper, recognizing the racial inequities and the historical discrimination.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

But there is a sense in which it will not be enough.

Scott Cunningham:

Yeah. Yeah. There's this, I can't, I just now drew a blank on the, I teach it all. I can see the slide in my deck, but there's a famous computer scientist. And he says, this principal about writing code, and he says, "Be conservative in what you do, and be liberal in what you accept from others." And it's this principle of code writing, which I guess is like, he's basically saying, "When you write code, it needs to be, the noise to signal ratio needs to be very, very low. You need to be very clear in what it's doing, in a very efficient choice to minimize this, these unnecessary errors." But when you're receiving the code, either from your earlier part of the code, or for some other foreign source, you have to change your viewpoint in that sense, because really, the goal, when you're on the receiving end of the code, it seems like your goal is to be this antenna.

Scott Cunningham:

And this antenna is trying to extract information from any meaningful information from the noise. And so, you have to have, as a listener, a certain amount of grace that tolerates that this other person may make mistakes, doesn't say it all right, goes really, really to great lengths to try to, you go to great lengths, to try to figure out exactly what the message is, and what it isn't. And it does seem like, successful communication is a, about a sender who is being clear, and a receiver that is being charitable in what they're going to allow the sender to say, unless the goal is conflict.

Peter Arcidiacono:

That's right.

Scott Cunningham:

If the goal is conflict, then obviously, you don't do that. What you do with conflict, is you find the most bad, then, it's just bad faith. It's just like, trap a person, win the debate. And sometimes, many of us don't realize who we're talking to. We don't know if we're talking to a good faith or a bad faith person. But there's limits, I think, to what an economist or anyone can do, if the person they're talking to really is not interested in connecting.

Peter Arcidiacono:

That's right. And it's interesting, because I think when I either speak publicly, or even giving seminars to economics audiences, the first part is building trust.

Scott Cunningham:

Totally.

Peter Arcidiacono:

We have the same goals.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

We may have different views about how to get there. And I've got some information that may change your mind on this.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

And the issue is whether they can hear the information I say, or if it's going to be ruled out because I'm a bad person.

Scott Cunningham:

Right. Well, let me ask you something. So these tests for, okay, so you correct me where my thinking is wrong. Testing for racial discrimination in admissions. I could imagine econometrics one, I get the data set from Harvard, and I run a regression of admit onto a race dummy.

Peter Arcidiacono:

Right.

Scott Cunningham:

And then, I interpret the statistical significance on the race dummy. And then, I add in more observables. In what sense is this, philosophically, what we are trying to do in the United States, legally, to detect for whatever it is that's violating the constitution. And in what sense is it a big fat failure, that's not what we're trying to do? Can you elaborate that as a multivariate regression-

Peter Arcidiacono:

Yeah. So I think, how to interpret that beginning coefficient, I don't think that coefficient has much of an interpretation, particularly in admissions, because of who applies. And that was, one of the papers that we published on this, is about Harvard's recruiting practices.

Scott Cunningham:

... Mm.

Peter Arcidiacono:

And Harvard, they recruit a lot of people. And particularly, African Americans, who simply have no chance of admission. And so, you could make it. And that could be part of the reason, right, would be, we want to appear as though when you do just that one regression with that one variable-

Scott Cunningham:

Mm.

Peter Arcidiacono:

... through affecting my applicant pool, I can always make it so that coefficient-

Scott Cunningham:

So what happening? So if I've got a university, just in real simple sense, let's say a university, if they're white, they span their, they basically task to the university, to whoever, and they say, "Get a pool of white applicants, use this rule. Get a pool of black applicants, use this rule." And it's just very, very different rules.

Peter Arcidiacono:

That's right.

Scott Cunningham:

Okay. If I then run a regression, how in the world am I going to detect racial preference in admission, when racial preference was used in the drawing up of the application in the first place?

Peter Arcidiacono:

So I think that's where, I think one of the principles that, it's not randomization for sure.

Scott Cunningham:

Right.

Peter Arcidiacono:

But one of the key principles, is how do you think about selection on observables versus unobservables?

Scott Cunningham:

Yeah. Right, right.

Peter Arcidiacono:

And so, if you can account, in the case we just described, if it was differences in test scores alone, once you account for test scores, then you could see how they were treated differently. Conditional on those test scores.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

And typically, the way that works, is that when you add controls, the coefficient on the discriminated group typically goes down, because there was, because of history discrimination, that there was going to be differences in those things. That was why you had the program in the first place.

Scott Cunningham:

Right. Right.

Peter Arcidiacono:

But what's interesting in the case of Asian Americans, is it tends to go in the opposite direction. Right? So they're stronger on a lot of the observables.

Scott Cunningham:

Right.

Peter Arcidiacono:

You add controls, it looks like the coefficients becomes more negative. For African Americans-

Scott Cunningham:

The coefficient, as in, the, so if I did a regression of admit onto an Asian dummy, nothing else, it'll be positive?

Peter Arcidiacono:

Well, it depends. So it would be positive if you had nothing else, and you excluded legacies-

Scott Cunningham:

Legacies.

Peter Arcidiacono:

... and athletes.

Scott Cunningham:

Okay. So I dropped the legacy and the athletes. I regress admit onto an Asian dummy. Asians are more likely to... So when does the, so what-

Peter Arcidiacono:

When it's slightly positive and insignificant.

Scott Cunningham:

... Okay.

Peter Arcidiacono:

As soon as you add anything related to academic background-

Scott Cunningham:

So then, I put in high school GPA and zip code, and I start trying to get at these measures of underlying academic performance, observable. And that's when it flips?

Peter Arcidiacono:

Oh yeah. Yeah. This is something I just did not appreciate before the Harvard case, is how incredibly well Asian Americans are doing academically.

Scott Cunningham:

Mm.

Peter Arcidiacono:

If you did admissions based solely on academics, over half would be Asian American. That is a stunning number. All groups would go down, and Asian Americans would be the only group that went up.

Scott Cunningham:

Okay. Say that again. I didn't quite follow. So what will astound me? What would it?

Peter Arcidiacono:

So Asian Americans, they're in the low twenties, in terms of their share of admits, or something like that.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

When you look at typical applicants.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

If you had admissions based solely on academics-

Scott Cunningham:

Mm-hmm.

Peter Arcidiacono:

... with some combination of test scores and grades-

Scott Cunningham:

Mm-hmm.

Peter Arcidiacono:

... they would be over half of Harvard's.

Scott Cunningham:

I see. Got it. They're just, it's just such an incredibly selective group. Selective, in terms of the measures of probable performance and success, and all these things. They are, as a group, high... What's the right word? How do you, this is one of these things, we're using the languages, is really careful. I was going to say, I know economists, we have models that say high type, low type. And obviously, it's like, what's the right way to start talking about these young people? These are young people at the beginning of their, everybody comes at a difference. So what's the right, what's the loving, charitable, honest way of talking about people with these underlying differences?

Peter Arcidiacono:

Well, I think that, what happened to them before college, was such, that on average, you see tremendous differences-

Scott Cunningham:

Yeah.

Peter Arcidiacono:

... in the skills that have been accumulated-

Scott Cunningham:

Right.

Peter Arcidiacono:

... prior to college.

Scott Cunningham:

Right. Right. So there appears to be, one way you could describe it, is to say, there appears to be differences in human capital.

Peter Arcidiacono:

That's right. But I think human capital, I guess-

Scott Cunningham:

Unobservable human capital appears to be different, but it's like showing up on these observable dimensions.

Peter Arcidiacono:

... That's right.

Scott Cunningham:

Got it.

Peter Arcidiacono:

And for me, that doesn't, in any way, point the finger, and say there's something wrong-

Scott Cunningham:

Right.

Peter Arcidiacono:

... with the groups that aren't doing well on that.

Scott Cunningham:

Yeah. No.

Peter Arcidiacono:

And in fact, there's some people who argue, look, the differences in test scores, the reason African American score worse on the tests, is because of stereotype threat.

Scott Cunningham:

Mm-hmm.

Peter Arcidiacono:

And that idea is that everybody expects them to do poorly. And so, they do poorly.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

To me, that's giving the K through 12 education system a pass. There are real differences-

Scott Cunningham:

Right.

Peter Arcidiacono:

... in the K through 12 education experience-

Scott Cunningham:

Mm-hmm.

Peter Arcidiacono:

... for African Americans.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

That's what we need to fix.

Scott Cunningham:

Right.

Peter Arcidiacono:

We can't shy away from the real issue. And that's actually one of my big concerns with places like the UC system, saying, "We don't want standardized tests anymore." We're just going to ignore that there's a serious deficiency.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

Not that the people are deficient, that the educational system was deficient-

Scott Cunningham:

Yeah. Yeah.

Peter Arcidiacono:

... for these students.

Scott Cunningham:

It's interesting. It's like, one of the papers I teach a lot, is, I know you're familiar with, is Mark Hoekstra's review of economics and statistics article, on the returns to attending the state flagship school. I've always thought-

Peter Arcidiacono:

Yeah.

Scott Cunningham:

... that this really interesting study, it feels relevant to what you're working on with Harvard and UNC, because it's about, I feel like when I was in graduate school, I came away from my labor courses, just realizing attending college is crucial. College is an anti-poverty program, as far as I can tell. You could see it in my work on crime, with the, you and I actually have some similar backgrounds. We're both interested in sex ratios and marriage markets.

Peter Arcidiacono:

Yeah.

Scott Cunningham:

But you could see the incarceration rate of African American men just plummeting, with college attainment, levels of college enrollment. But so, it's like, I graduated thinking, "Oh, well, the returns to college are important." But then, it's like, Mark's paper highlighted that there was this heterogeneity, even there. Even in these, in terms of the flagship school and Harvard.

Peter Arcidiacono:

Right.

Scott Cunningham:

And the reason why this stuff is important, I feel like it gets into these complicated things with regards to how we've decided to organize America, because the United States, we purchase goods and services using, goods and services go into the utility function. In many ways, that's the, trying to get utility functions that are virtuous and correlated with a life that's worth living, is the big goal. But we buy those goods and services at market prices, using labor income. And so, then, it always wraps back into this issue about something like Harvard or Chapel Hill, which is, some of these schools have imbalanced returns that affect labor income and quality of life, or might arguably, subjective wellbeing, as it's measured by utility. And I guess I'm just sitting here thinking to myself, if you have a group of people who are just for historical, it's not even historical accident, because they were historically discriminated against in the United States.

Scott Cunningham:

But at this point, it's a stock. African Americans have come to the table with this different kind of human capital, that's going to end up shaping all of their labor income. It's going to have massive impacts on labor income, where they go to college. It's like, I don't see how you can separate out the fact that there, we've got to decide, collectively, what exactly is the goal for these different groups of people that live here in the United States, and that one of the existing mechanisms for income, is college. And it all wraps back into this whole issue, about what exactly should the composition of the student body be, given these ridiculously imbalanced returns to each of these individual schools?

Peter Arcidiacono:

That's right. But I think that some of those things could be balanced more, if we were doing the things that were actually successful in changing the human capital-

Scott Cunningham:

Yeah. Yeah.

Peter Arcidiacono:

... upfront. And so, one of the most, it was really disappointing, in my mind, when, after Floyd, I think KIPP Charter schools decided that their motto was no longer appropriate. Be nice, work hard. And I say that, mainly because no excuse charter schools, which no excuse, that's something that you can't really say quite the same way now.

Scott Cunningham:

Right.

Peter Arcidiacono:

These schools were incredibly successful at closing the achievement gap.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

They were actually very successful.

Scott Cunningham:

Right.

Peter Arcidiacono:

We could be doing that. That's where the resources ought to go.

Scott Cunningham:

Right, right.

Peter Arcidiacono:

Instead, what you see in California now, is they're getting rid of advanced classes. There's two ways to deal with an achievement gap, right? You can bring the people who aren't doing as well, up.

Scott Cunningham:

Right.

Peter Arcidiacono:

Or you could bring the people who are doing well, down. The getting rid of the advanced classes, is not bringing, in my mind, those students up.

Scott Cunningham:

Mm-hmm.

Peter Arcidiacono:

And if anything, it's providing huge advantages to people of means, because you cripple the public education system, take the path out for them to develop that human capital.

Scott Cunningham:

Right. Right.

Peter Arcidiacono:

And then, the people with resources send their kids to private schools, so that stuff isn't going to go on.

Scott Cunningham:

Right. Right. Right.

Peter Arcidiacono:

And that's where I think a lot of the discussion, we can talk about affirmative action at Harvard. At the end of the day, that's really about appearances. The people are going to Harvard are all, most of them are coming from an incredibly rich backgrounds.

Scott Cunningham:

Right. Right.

Peter Arcidiacono:

Regardless of what race. There are differences across the races. But that's where the action is.

Scott Cunningham:

Right. Right.

Peter Arcidiacono:

And that's what we typically focus on in education. But where we really need to be doing more, is for the lower income kids.

Scott Cunningham:

Mm-hmm.

Peter Arcidiacono:

And COVIDs is going, we're starting to see that that's going to be a train wreck. Our education for this kids who went to-

Scott Cunningham:

Yeah. Yeah, yeah, yeah.

Peter Arcidiacono:

... public schools.

Scott Cunningham:

Yeah. Yeah. Yeah. There's certain elasticities, that I think COVID highlights, which is that there's a, there are groups of students who, probably, their ability to substitute to the best case scenario in a very difficult situation, was really, they had a very high, they were able to do it. It may not have been, it wasn't a perfect substitute. They were able to continue to do it. And I think for some groups of students, it was a train wreck.

Peter Arcidiacono:

Yeah.

Scott Cunningham:

Just their ability to make those substitutions to whatever was required, could be anything ranging from the access to physical resources, like computing, computers, and wifi that's stable, and all these things, to, just simply, the way your brain works. Just being able to be present. I definitely think that COVID cut a mark through the students, that, it did in our family, completely cut a mark through students in weird jagged way, for sure.

Peter Arcidiacono:

But within your family, you're able to substitute in ways that other families cannot.

Scott Cunningham:

Yeah. Yeah.

Peter Arcidiacono:

And that's the catch. And I think that, I don't work a lot in the K through 12 space, so this is a non-expert opinion on that. But if my read on the studies, is if you find positive effects of, say, charter schools, Catholic schools, smaller class size, if you're going to find positive effects for anyone, it's going to be inner city African Americans. And I think that the reason that you see that, is the way family substitutes, that they're not, their families are not in as good of a position to substitute-

Scott Cunningham:

Yeah. Yeah.

Peter Arcidiacono:

... the way my family is. My kid has a bad teacher, we're going to do the bad effects.

Scott Cunningham:

Right. Right. Right.

Peter Arcidiacono:

So you're going to think, "Oh, the teacher's fine." But no, we even did the effects of that teacher, in ways that other families cannot.

Scott Cunningham:

Right. Right. Right. So what do you think is the smoking gun evidence, that that Harvard University has to... What's the smoking gun fact, that's evidence for, that's the most damning evidence for racial discrimination in admissions, that-

Peter Arcidiacono:

So racial discrimination against Asian Americans, I think that there's a, there's so many damning facts. Well, I'll start with the first one, which is Harvard's own internal offices. They have their own internal research teams. They estimated models of admissions, and consistently found a penalty against Asian Americans.

Scott Cunningham:

... Mm.

Peter Arcidiacono:

You could look at that. You'll hear people say, "Well, those are simplistic models." The fit of those models was incredibly high.

Scott Cunningham:

Yeah. Yeah.

Peter Arcidiacono:

I think. So they were explaining-

Scott Cunningham:

I think people underestimate the shoe leather sophistication that goes on in these admissions office, with developing their own internal models.

Peter Arcidiacono:

... Well, and what was striking, is Harvard's defense of this was, "Well, we really didn't understand the model."

Scott Cunningham:

Mm.

Peter Arcidiacono:

Well, what was interesting, is that those models also had whether or not you were low income, in it.

Scott Cunningham:

Mm.

Peter Arcidiacono:

And they were confident that those models, the same model, showed that they were giving a bump to low income students.

Scott Cunningham:

Mm.

Peter Arcidiacono:

It's like, you're going to interpret the coefficient one way when it's the result you like, and another way, when it's the result you don't like.

Scott Cunningham:

Right. Right. So their own models showed, so what was the penalty? What was it? It was a dummy, a coefficient on a binary indicator for Asian American, or Asian?

Peter Arcidiacono:

That's right. That's right.

Scott Cunningham:

How big was it?

Peter Arcidiacono:

And then, also, it even had stuff on the personal rating. You can see, there was charts from their office that shows, what do you know, Asian Americans on all of Harvard's ratings, are scoring either much better than whites-

Scott Cunningham:

Mm.

Peter Arcidiacono:

... or the same as whites, even on the alumni personal rating. So Harvard has these alumni interview, the students, and even on that, Asian Americans are doing similarly to whites.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

And then, you see their own personal rating, based, not on meeting with the applicants. They do much, much worse.

Scott Cunningham:

Yeah. Yeah, yeah, yeah. Yeah. Well, so what does Harvard have to prove?

Peter Arcidiacono:

Well, I think typically, in something like discrimination cases, well, what they have to prove, probably depends on the judge, I suppose-

Scott Cunningham:

Yeah. Right.

Peter Arcidiacono:

... is the catch. What they were able to say at trial, were things like, "Well, the teachers must be giving them poor ratings. We don't think that Asian Americans are deficient on personal qualities, but maybe the teachers are scoring them poorly." How that is an excuse. I don't-

Scott Cunningham:

Yeah. I don't see what they're trying to... This is, I guess, where it's frustrating, because I'm struggling to know exactly what the objective function for Harvard is, in their own stated goals. What is their objective function? To create a particular kind of cohort? What is the cohort?

Peter Arcidiacono:

... Well, I think you'd get a lot of gobbledygook when it comes to that-

Scott Cunningham:

That's what I was wondering. Yeah. Okay.

Peter Arcidiacono:

... Yeah. So, but I think it is also interesting to think about the counterfactual of, if this case was not associated with affirmative action at all-

Scott Cunningham:

Yeah.

Peter Arcidiacono:

... would it have played out the same way? And to me, I think the answer is no. Honestly, I don't think Card even takes the case.

Scott Cunningham:

Mm.

Peter Arcidiacono:

I think it would've been a much better... Your worse look for Harvard than it was. I think that it was a bad look for Harvard as it was, but because of who brought the case, and because of its ties to affirmative action, that gets back to that waffle analogy, right? If you look at it in the context of the waffle, there's just simply no argument in my mind, for the way they're treating Asian Americans.

Scott Cunningham:

Mm-hmm.

Peter Arcidiacono:

It's a clear cut discrimination case.

Scott Cunningham:

Mm.

Peter Arcidiacono:

And if you just put it in a different context, it would just be completely unacceptable. Imagine Trump Towers having a discrimination suit brought against them by black applicants. And the defense being, "Look, it's not that we're discriminating against black applicants. They just happen to score poorly in our likability rating."

Scott Cunningham:

Mm-hmm.

Peter Arcidiacono:

That would be outrageous.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

There would be protests. This is because it's tied to that third rail of affirmative action.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

But to me, the judge could have ruled, "Look, you can have affirmative action, but you got to stop discriminating against Asian Americans relative to whites.

Scott Cunningham:

So then, if you could fill up half of Har... So is this what the thing is? Harvard, as a university, collectively, however this ends up being decided, collectively, they have a preference over their student composition.

Peter Arcidiacono:

Right.

Scott Cunningham:

And that preference is discriminatory.

Peter Arcidiacono:

Their preference, I think, lines up with Kendi's in some sense. They would like to have their class look like the population.

Scott Cunningham:

They would like to have it look like, that they would like 13% African American, whatever percent, what is it, Asian American is what, five, is single digit?

Peter Arcidiacono:

Yeah.

Scott Cunningham:

Yeah. And they would like to have a balanced portfolio of Americans.

Peter Arcidiacono:

And, but even that, I think, is giving Harvard too much credit, in the sense that, what we choose to balance on, we choose to balance on skin color.

Scott Cunningham:

Right.

Peter Arcidiacono:

You're not balancing on income.

Scott Cunningham:

Right.

Peter Arcidiacono:

You're not balancing on parental education.

Scott Cunningham:

Yeah. Yeah.

Peter Arcidiacono:

A whole bunch of other things you could've balanced on. Why-

Scott Cunningham:

Yeah. There's like an infinite number of character. Every person is a bundle of, just almost an infinite number of characteristics. And it's not practically... Yeah. Yeah.

Peter Arcidiacono:

... If you really want a representative class, then you do a lottery among high school graduates.

Scott Cunningham:

Yeah, yeah, yeah. Exactly.

Peter Arcidiacono:

That would be the only way.

Scott Cunningham:

That would be the only way, the only way it would be to have a randomized student body. Okay.

Peter Arcidiacono:

Do you feel like ask this about, was by somebody from a class at Duke, about how would you make the admissions process more equitable?

Scott Cunningham:

Uh-huh.

Peter Arcidiacono:

And I'm like, it's a selective admissions process. I don't even know what that-

Scott Cunningham:

Right.

Peter Arcidiacono:

... means. Even a process where you did the lottery, why is that equitable, because you've got the winners and the losers? The lottery. We're not equalizing outcomes for everybody. We're equalizing X anti.

Scott Cunningham:

Yeah. It's like, this is all this comp, this is this deep collective choice, social preferences questions about... And it's weird. I guess we're talking about this at Harvard, because we believe that Harvard University will literally change a kid's life, more than going to University of Tennessee, Knoxville, or something like that. Right? That's why we're having this conversation.

Peter Arcidiacono:

Yeah. I think that that's the perception, that it will literally change their kids' lives.

Scott Cunningham:

Yeah. Yeah.

Peter Arcidiacono:

I'm not totally convinced that of there being massive gains-

Scott Cunningham:

Right.

Peter Arcidiacono:

... relative to the counterfactual for-

Scott Cunningham:

Yeah.

Peter Arcidiacono:

... at that level.

Scott Cunningham:

Right.

Peter Arcidiacono:

I think, when you're at the margin of going to college or not-

Scott Cunningham:

Yeah.

Peter Arcidiacono:

... that's the big margin.

Scott Cunningham:

That's the big margin. Yeah. Yeah.

Peter Arcidiacono:

College quality effects, I think get undone a little bit by college major effects.

Scott Cunningham:

Right, right, right.

Peter Arcidiacono:

So yeah. I think that's a real valid question, about whether it's worth it to be paying the huge sums of money to go to just a slightly better school.

Scott Cunningham:

Okay. I want to conclude with this. So you've now spent many years working on this, going deeper into administrative data, about questions that, about phenomenon that we had only speculated about, possibly. What are the top two things that you learned, that, being a career economist that had worked on topics in education and affirmative action, what are the top two things that you learned, that you just literally, it was, you just, would've never learned, had you not been involved in these two cases, that are of real, that you think that matter for other people too?

Peter Arcidiacono:

Right. So I think one of the things, is that the Supreme Court rulings in the Michigan cases, were really a handout to elite private schools.

Scott Cunningham:

Oh, interesting.

Peter Arcidiacono:

Then, when you think about doing holistic admissions, a place like Harvard can do that in a way that UNC cannot.

Scott Cunningham:

Mm.

Peter Arcidiacono:

UNC is effectively, and you can see this in our models, they are more formulaic.

Scott Cunningham:

Mm.

Peter Arcidiacono:

I don't even know how Berkeley's doing admissions now, without test scores. Think about how many applications they get, with the resources that they have. I just don't know. The other part to-

Scott Cunningham:

Wait, [inaudible 01:08:39], Peter, are you basically saying that Harvard and elite universities get, first of all, a profound amount of information about each student, and they get this incredibly right tail applicant, that they don't even need test scores, to find the students that are going to be successful?

Peter Arcidiacono:

Well, I think they can get information in a way that the public schools can't, because of the resources. So you won't have to take those test scores, but they're going to see, you might have things like winning a science fair-

Scott Cunningham:

Right.

Peter Arcidiacono:

... in other proxies.

Scott Cunningham:

Right.

Peter Arcidiacono:

And that's information that the public schools aren't even going to really collect-

Scott Cunningham:

Yeah, yeah.

Peter Arcidiacono:

... because they don't have the resources to do that. You can see that, just in the number of letters Harvard requires versus UNC.

Scott Cunningham:

The fact that any university would ever voluntarily say, "We're not going to collect this informa, or make decisions based on this anymore," you should already just automatically think, the only people that would ever voluntarily do anything, is because they don't need that thing.

Peter Arcidiacono:

Right.

Scott Cunningham:

Right. They're endogenously sorting into something that, probably, they incur almost no cost of doing it. So an elite university that drops some sort of admission criteria, probably, they've got something just as good sitting right there.

Peter Arcidiacono:

Well, I think, I'm not sure if that's something just as good. I think it's also a protection against an Asian American discrimination suit, because Asian Americans are just doing so well there.

Scott Cunningham:

Right.

Peter Arcidiacono:

You need to take it out of the criteria. And that probably brings me up to the second point, which I think is a huge one. I think holistic admissions favors people from privileged backgrounds. And I'd say that even more so, after going through these cases.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

We think about test scores as being unfair, because of coaching, and they're correlated with income. And that, of course, is true.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

But the other things are even more unfair. Harvard has an athletic rating for non recruit athletes, and the people who score best on that are white legacies.

Scott Cunningham:

Mm.

Peter Arcidiacono:

And why is that? Because part of that, is are you likely to walk on to one of Harvard's sports teams? Well, Harvard sports, normally we think about sports as being more of an equal opportunity thing. That's not true at Harvard. Harvard offers more varsity sports than any school in the country.

Scott Cunningham:

Mm.

Peter Arcidiacono:

And what's a marginless sport? It's sailing. So sailing, who does sailing? People who are coming from really rich families.

Scott Cunningham:

Yeah. Right.

Peter Arcidiacono:

That was another stunning thing, is that I think we really need to rethink college athletics. It's one thing to think about Duke's basketball team-

Scott Cunningham:

Right.

Peter Arcidiacono:

... as a pathway where, sure, of course, we're going to admit you. You're... But it's another thing, in my mind, to think about a sailing team as being, I don't have to be particularly strong academically, as long as I'm on the sailing team.

Scott Cunningham:

Right.

Peter Arcidiacono:

That seems like a pathway for the rich.

Scott Cunningham:

Mm. Mm. Well, how much longer do you think you'll be involved in this? This is, you've got now a treasure trove of a data too, right? You have a lot of questions that you're going to be probably mining for a long time, just even your scholarly career, outside-

Peter Arcidiacono:

I wish. If I had access to the raw data.

Scott Cunningham:

Oh, you don't?

Peter Arcidiacono:

I don't.

Scott Cunningham:

What do you, so what do you use? How can you be an expert witness without the raw data?

Peter Arcidiacono:

Well, I had the raw data at the time I wrote my reports.

Scott Cunningham:

Oh.

Peter Arcidiacono:

So all my reports have tables from that.

Scott Cunningham:

Got it.

Peter Arcidiacono:

And so, we've published four papers, effectively, out of my reports and out of other things revealed in the trial.

Scott Cunningham:

Yeah.

Peter Arcidiacono:

They actually tried to get the database admitted into evidence, and not surprisingly, Harvard objected.

Scott Cunningham:

Yeah, yeah, yeah. That would've been, once it's in evidence, anybody can have it.

Peter Arcidiacono:

That's right. My fear with a lot of this stuff, is universities will selectively release their data to people who will get them the answers that they want.

Scott Cunningham:

Mm.

Peter Arcidiacono:

And no one else will be able to look at it.

Scott Cunningham:

Yeah. Yeah. Yeah.

Peter Arcidiacono:

It's a serious problem. It's like pharmaceutical companies being the only ones allowed to evaluate the trials of their drugs.

Scott Cunningham:

I bet you, the Asian American community was, has been, I bet this has been very troubling.

Peter Arcidiacono:

Oh. And confirming for them. So that, I didn't really know about this. But then, I talked to some of my former Asian grad students, and they're like, "Yeah, this is, we've known this has been going on for a long time." And I think what, to me, what's probably more troubling to them, is not that it's going on, but the fact that now that it's been exposed, how comfortable people are with it. That's disappointing.

Scott Cunningham:

Yeah. Yeah. Yeah. It's so nice to talk. We met a long time ago. You came and presented your terms of in, I think it was, no, I don't think you presented the terms of endearment paper. I think that you presented something else, but you were going to. But Chris Cornwell had invited you to Georgia while I was a graduate student. And we, I was in Chris's office with you and, because we had this marriage market. I had a marriage market dissertation, and-

Peter Arcidiacono:

Yeah.

Scott Cunningham:

... and you had this other thing. And it's nice to meet again after probably 16 years.

Peter Arcidiacono:

Yeah. It's been a long time.

Scott Cunningham:

Been a long time. Yeah. Well, good luck with everything. Thanks for sharing everything about this. This has been really interesting for me. I hope for other people too.

Peter Arcidiacono:

Oh, I love doing it. I really appreciate you having me on. When I look at the other people you have on, it's like, wow. This guys, which one of these doesn't belong?

Scott Cunningham:

All right. You have a great day.

Peter Arcidiacono:

You too. Talk to you later.

This week I have the pleasure of introducing Dr. Anna Aizer, professor of economics at Brown University and editor-in-chief at the Journal of Human Resources. I am a long time admirer of Dr. Aizer’s work and have followed her career with curiosity for a long time. Some of her papers imprinted pretty strongly on me. I’ll just briefly mention one.

Her 2015 article in the prestigious Quarterly Journal of Economics with Joe Doyle on juvenile incarceration, for instance, has haunted me for many many years. It was the first or second paper I had seen at the time that had used the now popular “leniency design” to examine the causal effect of being incarcerated as a youth on high school completion and other outcomes as well as adult incarceration. Simply comparing those outcomes for those incarcerated and those not incarcerated as a kid will not reveal the causal effect of juvenile incarceration if juvenile incarceration suffers from selection bias on unobservable confounders. So Dr. Aizer with Joe Doyle used a clever approach to overcome that problem in which they found quasi-random variation, disconnected from the unobserved confounder, in juvenile incarceration caused by the random assignment of juvenile judges. As these judges varied in the propensity to sentence kids, they effectively utilized the judges’ own decisions as life changing lotteries which they then used to study the effect of juvenile incarceration on high school and adult incarceration. And the findings were bleak, depressing, enraging, upsetting, sad, all the emotions. They found that indeed being assigned to a more strict judge substantially raised one’s chances of being sentenced as a kid. Using linked administrative data connecting each of those kids to their Chicago Public School data as well as Cook County incarceration data, they then found that being incarcerated significantly increased the effect of committing a criminal offense as an adult, and it decreased the probability of finishing high school. The kids, best they could tell, mostly didn’t return after their juvenile incarceration, but if they did return, they were more likely to be given a emotional and behavioral disorder label in the data. My interpretation was always severe — incarceration had scarred the kids, traumatizing them, and they weren’t the same.

The paper would haunt me for various personal reasons as I saw a loved one arrested and spent time in jail on numerous occasions. I would see kids in my local community who had grown up with our kids arrested and think of Dr. Aizer' and Joe Doyle’s study, concluding the most important thing I could do was bail them out. The paper was one of many events in my own life that led me to transition my research to mental illness within corrections and self harm attempts by inmates even.

But there’s other personal reasons I wanted to interview Dr. Aizer. Dr. Aizer went to UCLA where she studied with Janet Currie, Adriana Lleras-Muney and Guido Imbens. Recall that when Imbens was denied tenure at Harvard, he went to UCLA. Currie, who had attended Princeton at the same time as Angrist, Imbens’ coauthor on many papers on instrumental variables in the 1990s, was an original economist focused on the family, but unlike Becker and others, brought with her that focused attention to finding variation in data that could plausibly recover causal effects. The story, in other words, of Princeton’s Industrial Relations Section and design based causal inference, going back to Orley Ashenfelter, was spreading through the profession through the placements of scholars at places like UCLA, which is where Dr. Aizer was a student. In this storyline in my head, Dr. Aizer was a type of first generation member of the credibility revolution, and I wanted to talk to her not only for her scholarly work’s influence on me, but also because I wanted to continue tracing Imbens and Angrist’s influence on the profession through UCLA.

The interview, though, was warm and interesting throughout. Dr. Aizer is a bright light in the profession working on important questions in the family, poverty and public policy. For anyone interested in the hardships of our communities and neighborhoods, I highly recommend to you her work.

Now let me beg for your support. Scott’s Substack and the podcast, Mixtape with Scott, are user supported. If your willingness to pay for the episodes and the explainers (I’m going to write some more I promise!), please consider becoming a subscriber!

Scott's Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Transcript

Scott Cunningham:

In this week's episode of the Mix Tape podcast, I had the pleasure of interviewing Dr. Anna Aizer, professor of economics at Brown University in Rhode Island and editor in chief of the Journal of Human Resources. I have had a keen interest in Anna Aizer and her career and her work for a couple of reasons. Actually a lot, but here's two. First, she did her PhD at UCLA when Janet Currie was there, as well as when Guido Imbens was there. Imbens taught there after he left Harvard, for those of you that remember that interview I did with him. Recall my overarching conviction that Princeton's industrial relations section, which was where Orley Ashelfeltner, David Card, Alan Kruger, Bob Lalonde, Josh Angrist originated from, as well as Janet Currie.

My conviction that this was the ground zero of design based causal inference. And that design based causal inference spread through economics, not really through econometrics, and econometrics textbooks, but really through applied people. She also worked with Adriana Lleras-Muney, who's also at UCLA now, who was a student of Rajeev Dehejia, who wrote a seminal work in economics using propensity score, who was also Josh Angrist’s student at MIT. So you can see, Anna fits my obsession with a sociological mapping out of the spread of causal inference through the applied community.

But putting aside Anna as being instrumentally interesting, I am directly interested in her and her work on domestic violence and youth incarceration among other things. I've followed it super closely, teach a lot of these papers all the time, think about them even more. In this episode, we basically walked through her early life in Manhattan to her time at Amherst College, to her first jobs working in nonprofits, in areas of reform and poverty, to graduate school. We talked about her thoughts about domestic violence and poverty and crime along the way, too. And it was just a real honor and a pleasure to get to talk to her. I hope you like it as much as me. My name is Scott Cunningham and this is Mix Tape podcast. Okay. It's really great to introduce my guest this week on the podcast, Anna Aizer. Anna, thank you so much for being on the podcast.

Anna Aizer:

Pleasure to be here. Thanks so much for inviting me.

Scott Cunningham:

Before we get started, could you tell us obviously your name and your training and where you work?

Anna Aizer:

Sure. I'm a professor of economics at Brown University. I did my PhD at UCLA oh many years ago. Before that actually I got a masters in public health. Sorry. I have a strong public health interest and focus in a lot of my work. I'm also currently the co-director of the NBR program on children. That is a program at the NBR that is focused entirely on the economics of children and families. I'm the editor in chief of the Journal of Human Resources.

Scott Cunningham:

Great. It's so nice to meet in person. I've been a long time reader of your papers because you write about these topics on violence against women. There's not a lot of people in economics that do. And the way that you approach it shares a lot of my own thoughts. I'm going to talk about it later, but it's really nice to meet in person.

Anna Aizer:

Sure. Nice to meet you, too.

Scott Cunningham:

Okay. I want to break up the conversation a little bit into your life. First part, just talk about your life growing up. And then the second part, I want to talk about research stuff. So where did you grow up?

Anna Aizer:

I grew up in New York City.

Scott Cunningham:

Oh, okay.

Anna Aizer:

Yeah, I did.

Scott Cunningham:

Which, borough was it?

Anna Aizer:

Manhattan.

Scott Cunningham:

Oh, okay.

Anna Aizer:

Yeah. Yeah. Upper side. But when I went off to college, I went to rural Massachusetts.

Scott Cunningham:

Yeah.

Anna Aizer:

Yeah. I went to Amherst, which is a very small liberal arts college in the Berkshires. That was a very different experience for me. And believe it or not, I was not an econ major.

Scott Cunningham:

Oh, you weren't?

Anna Aizer:

In fact I was not. I only took one econ course my entire four years in college.

Scott Cunningham:

Oh, wow. Wait, so what'd you major in?

Anna Aizer:

I majored in American studies with a focus on colonial American history and literature.

Scott Cunningham:

Mm. On literature. Oh, that's what I majored in, too.

Anna Aizer:

Yeah.

Scott Cunningham:

Yeah, yeah, yeah. Oh, wow. So early American history. So what, was this was the 1700s or even-

Anna Aizer:

Yeah. So I did a lot of 17, 1800s, a lot of the New Republic period. My undergraduate thesis was actually on girls schooling in the Early Republic.

Scott Cunningham:

Oh wow. What was the deal with girls schooling in the Early Republic?

Anna Aizer:

What was the deal with the girls schooling? Well, it depends. For most of the Northeast, the focused on girls schooling was really this idea that it was a new country, they were going to have to have leaders in this new country, and someone had to educate those leaders. Someone had to educate those little boys to grow up, to go ahead and lead this country. And so the idea was, well, we had to start educating moms so that they could rear boys who could then go on to this great nation.

Scott Cunningham:

I see. Women's education was an input in male leadership?

Anna Aizer:

That's correct.

Scott Cunningham:

Got it. Got it. Wow. Okay. Well, that's interesting. I get that. You start educating women though, I suspect that you get more than just male leaders.

Anna Aizer:

I think that's right. It was an unintended consequence.

Scott Cunningham:

Unintended consequence. They didn't think that far ahead. Okay.

Anna Aizer:

Yeah. That's a very good point to make, because two women who were educated in one of the first schools dedicated to educating women so that they could go on and rear their boys to be strong leaders were Katherine Beecher, who went on to create one of the most important girls schools in Troy, New York. And Harriet Beecher Stowe of course, who wrote Uncle Tom's Cabin.

Scott Cunningham:

They're related?

Anna Aizer:

Yeah. They are sisters. They are sisters.

Scott Cunningham:

Oh, they're sisters.

Anna Aizer:

They were one of the first sets of girls who were educated in this mindset of we need leaders so let's have some educated moms. And they of course had other ideas and they went and formed schools and wrote incredibly important works of fiction that ended up playing a pretty significant role in the Civil War.

Scott Cunningham:

Wow. Was this the thing over in England too? Or was this just an American deal?

Anna Aizer:

I don't know the answer to that.

Scott Cunningham:

Huh. I guess they have a different production function for leaders in England where as we it's very decentralized here or something. Right?

Anna Aizer:

Right. So you're saying in England they already had their system of you go to Eaten, and then you go to Cambridge or Oxford. Right. I think that's probably right. So we didn't have that here.

Scott Cunningham:

Yeah. That's right. I mean, you're creating everything from scratch. And with such a reactionary response to England who knows what kinds of revolutionary approaches you're taking to... That's probably pretty revolutionary, right? Say we're going to teach women even though it's in order to produce male leaders, it's still thinking outside the box a little bit.

Anna Aizer:

Yeah. I suppose that's true. Yeah.

Scott Cunningham:

That's cool. How come you didn't end up in... So you end up at Amherst. As a kid in Manhattan, what were you doing? You were reading books and stuff? You were a big reader?

Anna Aizer:

I suppose. Yeah. I suppose so.

Scott Cunningham:

Is that what drew you to Amherst, a liberal arts college?

Anna Aizer:

I don't really know. I don't think I actually knew what I wanted until much later in life. I was an American studies major, which at the time I learned a lot. It took me a while to gravitate to economics. Once I did, it was clear that that was really the right path for me.

Scott Cunningham:

Yeah. One question I want to leave your kid. So your parents let you ride the subway when you were a little kid?

Anna Aizer:

Oh yes.

Scott Cunningham:

Oh gosh. I bet that was so cool.

Anna Aizer:

Oh yes. I grew up in New York City during the '70s and '80s, which was far more dangerous than it was today. But at that time parents had a much more hands off approach to parenting. I think I was eight years old when I started taking public transportation by myself.

Scott Cunningham:

Oh my gosh. There was latch key parents back then?

Anna Aizer:

Sure.

Scott Cunningham:

So you jump on the subway. Where are you going at eight years old in Manhattan?

Anna Aizer:

You go to school.

Scott Cunningham:

You're just catching the subway to go to school?

Anna Aizer:

Yeah.

Scott Cunningham:

Oh, that's so cool. I bet you had a great childhood.

Anna Aizer:

I have to say it was pretty good.

Scott Cunningham:

Oh man.

Anna Aizer:

I can't complain.

Scott Cunningham:

Yeah. I grew up in a small town in Mississippi, but it was the same kind of thing. Well, it was very different than Manhattan, but just being able to have that level of... It's all survivor bias. The other kids that are getting really neglected and abused. But those of us that made it out a lot it's like, all you have is great memories of being able to do whatever.

Anna Aizer:

Right. Agreed.

Scott Cunningham:

So you wrote this thesis. At Amherst, did everybody write a thesis? Is that real common?

Anna Aizer:

Most people did. I think a third of the students wrote a thesis. It was very common.

Scott Cunningham:

But you're gravitating towards research, though?

Anna Aizer:

Yeah. So it was clear that I really, really enjoyed that a lot. In fact, more recently in my economic research I have done a lot more historical work than I had done initially. So I think that training has really come in handy.

Scott Cunningham:

Yeah. What did you like about that project that you wrote your thesis on? What did it make-

Anna Aizer:

Well, it was really a lot of fun. I focused on two schools in particular. I focused on this school in Lichfield, Connecticut, and another school in Pennsylvania, a Quaker school in Westtown. I focused on those two schools because those two schools, for whatever reason, kept a lot of their records. They have really wonderful-

Scott Cunningham:

Oh my God. You had their records?

Anna Aizer:

Yeah. So you have really wonderful archives where you could just go through and read all about what they were thinking about, when they founded the schools, what the curriculum should be like. And even some of the writings of some of the students and teachers.

Scott Cunningham:

Oh my gosh.

Anna Aizer:

So it was really just a tremendous amount of fun to read all of that stuff, all that primary materials.

Scott Cunningham:

Oh my gosh. Wait. Did you actually have the names of the kids? Did you see their-

Anna Aizer:

Sure. They had all of that.

Scott Cunningham:

Did you have the census records and stuff?

Anna Aizer:

Oh, I guess you could. I mean, this was so long ago before people were doing all that cool linking, but yeah, you absolutely could.

Scott Cunningham:

Oh, that's so neat. I wonder where those kids ended up. What did it make you feel doing that research, that was so original and just being out there in these archives?

Anna Aizer:

Well, it was just amazing how much you could learn by just peeking into people's lives. It was really exciting. It was really fun. And you just felt like you were discovering something new.

Scott Cunningham:

Yeah, yeah, yeah, yeah. So you liked that. But that's interesting because some people would be like, oh, discovering something new. I don't even care about that. When you were discovering something new, you were like, I like this feeling.

Anna Aizer:

Yeah. Yeah. I really did.

Scott Cunningham:

Yeah.

Anna Aizer:

I really did.

Scott Cunningham:

Yeah. So what happened? So you graduate?

Anna Aizer:

I graduated. My first job was actually working for an Alternative To Incarceration program in New York City. So I moved back home. You have to remember, this was early mid '90s, and this was the peak in terms of crime rates in the country, and in New York City in particular. And the jails-

Scott Cunningham:

Before you say this, when you were growing up, did your parents... Was it like people were cognizant... I mean, now you know, oh, it was the peak because it's fallen so much, but what was the conversation like as a kid about crime?

Anna Aizer:

In the '90s in New York City at this time, that was really the crack cocaine epidemic, so there was a lot of talk about that. That really did dominate a lot of the media at the time. It really was a big concern.

Scott Cunningham:

Yeah. Yeah.

Anna Aizer:

As we know, the city and the state, not just in New York, but nationally, really responded with very tough on crime approach, started incarcerating a lot of people. So much so that they were really out of space in the New York City jail. So Rikers Island was at capacity, even upstate prisons were pretty full. The city, not because they were concerned that we were putting too many people in jail, which has... After the fact we know that we did put too many people in jail, that there was a cost to these incredibly high incarceration rates.

Anna Aizer:

At the time, the concern was that we don't have enough space, so what are we going to do? The city funded an Alternative To Incarceration program for youth. It was called the Court Employment Project. It was really focused on kids between the ages of 16 and 21 who were charged with a felony in New York state Supreme Court. And these were kids who were being charged as adults, treated as adults in the system. New York City has since raised the age of majority, but at that time it was 16. So we were focused on really younger 16 to 21. Well then, most of the kids we were working with were 16 to 18.

Scott Cunningham:

What kind of felonies are we talking about? Is this the drug felonies? Or is it [inaudible 00:15:51]?

Anna Aizer:

Yeah. So a lot of it was possession with intent to sell, selling. But also robbery, that was pretty common as well. We were only working with kids that were facing at least six months in adult prison, essentially. That was the rule for our program. Because again, our program was really focused on trying to reduce the number of people who were being detained and incarcerated for long periods of time. So we were only dealing with people who had-

Scott Cunningham:

Wait, real quick. So you're in your early 20s?

Anna Aizer:

Yeah. So I would've been about 23.

Scott Cunningham:

How'd you find this gig? You were just going back to New York City? Or what was the deal?

Anna Aizer:

Yeah. I knew I wanted to go back home. At that time, jobs were advertised in the paper, so you looked through the help wanted ads and you just sent cover letters and resumes by mail to whatever jobs appealed to you. I was interested in those jobs. I was also interested in working with public defenders, so the Legal Aid Society in New York, I applied for a number of jobs there.

Scott Cunningham:

Where's this coming from? What's your values exactly at this time? You're concerned about poverty or concerned about something? What's the deal?

Anna Aizer:

Yeah. I think I guess I already was really worried. I was really concerned about low income kids who were really... I felt already were getting derailed at very young ages in a way that I thought would be very hard for them to recover. I think that in that sense was really confirmed when I started working that these were kids who in a split minute their lives were just totally changed. So certainly in the case of things like robberies, these were often group of kids with not much to do, just getting into trouble, and it just getting too far too quick. And before they knew it, they were facing two to six years. I mean, it was just really tragic.

Scott Cunningham:

Yeah, yeah, yeah. Yeah. I know. Six months. You think about it, too. You're looking at these six months in the program. You start looking at six months and you think, oh, that's six months. The thing is, those things cascade, because six months with a felony record serving prison becomes de facto a cycle of repeated six months, one year, two years.

Anna Aizer:

Sure.

Scott Cunningham:

You just end up... Well, that's going to be a paper that you end up writing, so I'll hold off on that. Okay. So you end up applying, you spray the city with all these resumes. And then this thing. So what is this company? This is a nonprofit?

Anna Aizer:

Yep. So it's a nonprofit that had a contract with the city. They had a contract with the city. Again, they were funded really because the city could not afford to put any more people on Rikers Island.

Scott Cunningham:

So it's like a mass incarceration response almost?

Anna Aizer:

Yeah.

Scott Cunningham:

Capacity constraints.

Anna Aizer:

They were at capacity, so they needed to do something. So what this program was, it was an intensive supervision program. The kids had to come in at least twice a week and meet with a counselor. The counselor would provide counseling services and also check in on them, make sure they were going to school or working or getting their GED. And then they would write up these long reports.

Anna Aizer:

I only worked in the courts, so I wasn't doing any of the counseling myself. I had no qualifications to do that. I worked in the courts, so my job was to screen kids for eligibility for the program, interview them, see if they were good candidates. Then talk to their families, talk to their lawyers. And then talk to the judge eventually about the program and about what we would be doing and why we thought this person was a good candidate. And then once they were in the program, I would then provide updates or reports back to the judge and the defense attorney to let them know how the individual was doing.

Scott Cunningham:

And wait. What is the treatment going to be that things are doing?

Anna Aizer:

Again, so it was really-

Scott Cunningham:

It's a deferment of you're going to go to jail?

Anna Aizer:

Yeah. That's exactly right. It was a six month program. If they made it through after six months, they would be sentenced to probation instead of jail time.

Scott Cunningham:

Yeah. They would refer adjudication type concept.

Anna Aizer:

Exactly.

Scott Cunningham:

Right. Yeah.

Anna Aizer:

Exactly. So that was the idea.

Scott Cunningham:

But it's non random. And I know you're not-

Anna Aizer:

It was, yeah.

Scott Cunningham:

You're not thinking about the future Anna Aizer [inaudible 00:21:17], but it's not random.

Anna Aizer:

No.

Scott Cunningham:

What is it conditioned on? Because you're doing all of it, right?

Anna Aizer:

Right. Right. So you look at a kid's record. You would look at whether or not the kid seem to have support. The downside was if a kid didn't make it through the program they might be sentenced to more time-

Scott Cunningham:

Really?

Anna Aizer:

than they would have... Maybe. I mean, the judge would-

Scott Cunningham:

Why? Because you're getting a new judge or something?

Anna Aizer:

No, it's the same judge. But the judges say, "Look, I'm going to give you a chance. Instead of sending you away now for six to 18, I'm going to give you an opportunity to prove yourself. Six months, stay out of trouble, complete this program. And then I'm going to send you to probation. But if you don't complete the program, I'm going to sentence you more." In the end, they might not have actually done that. They certainly didn't tie their hands in any way.

Scott Cunningham:

What do they doing? Why are they doing that? Why is a judge doing that? They're trying to deal with some sort of adverse selection or something? They don't want people to-

Anna Aizer:

They want to create an incentive for the kid to-

Scott Cunningham:

They're trying to create an incentive for the kid. Got it. Okay.

Anna Aizer:

Yeah. They-

Scott Cunningham:

Like a little scared straight thing?

Anna Aizer:

A little. I mean, the judges always think that. It's not clear that that works. I don't think that really matters so much in the decision making of young people. I think it's-

Scott Cunningham:

Yeah. Totally. Totally.

Anna Aizer:

But that certainly was on the mind I think of many of the judges.

Scott Cunningham:

It's funny though. When I think about this paper that we're going to talk about a little bit, it's like you're already aware of, oh, these judges have a little bit of discretion. They're saying a bunch of stuff that's not in the law. "If you don't do this, I'm going to give you penalize, I'm going to give you really bad grade at the end with another year in prison." Did that cross your mind that you were noticing that judges were... This judge does that and this other judge does not tend to do that, is that something you could have noticed?

Anna Aizer:

Absolutely.

Scott Cunningham:

Oh, wow.

Anna Aizer:

Yeah. So there were many, many judges. So this is Manhattan. This is the main criminal courts in Manhattan, so I had many, many judges, a lot of people. The way it works is once you've been indicted on a felony you come before one of these three judges. They're called conference judges. They try to dispose of the case. Either the case gets dismissed or they take the plea deal. But if that doesn't happen, they reach into a bin, literally a lottery-

Scott Cunningham:

It's like a bingo ball machine?

Anna Aizer:

It's a lottery with all these different judges' courtrooms. They pull out a number, and that's the number of the courtroom you get assigned to. You know right then if you get assigned to certain judges, for sure that kid is going to do jail time. And if you get assigned to other judges, for sure that kid is going to get probation.

Scott Cunningham:

Who knows this? The kids don't.

Anna Aizer:

The kids don't, but they don't know it.

Scott Cunningham:

They can't comprehend.

Anna Aizer:

But their attorney will know it.

Scott Cunningham:

And then maybe their parents.

Anna Aizer:

No, I don't think their parents would know.

Scott Cunningham:

Although, who in a group of kids that maybe their parents aren't as-

Anna Aizer:

I don't think their parents would know it, either. You would know it because you have to remember that all of the judges for the most part were either defense attorneys or prosecutors before they were judges, and you can tell. The judges who would-

Scott Cunningham:

Is that the main source of the discretion that you notice?

Anna Aizer:

I think so. I think so. I think the judges who previously prosecute-

Scott Cunningham:

I mean, they're such different. It does seem like the prosecutors and the defense attorneys are almost cut from a completely different worldview and set of values.

Anna Aizer:

I think that's right.

Scott Cunningham:

I had this friend that was a public defender in Athens and he was like... I think this is what he said. I'm not going to say his name because he probably didn't say this, but I thought he basically said, "I don't like prosecutors because they think they are always guilty."

Anna Aizer:

Yeah.

Scott Cunningham:

And you could tell. The public defender, they were like, "My whole job is to not do that." I could just imagine that shaping... Either there's a lot of selection into that or that just really... You hear that all the time. There's got to be human capital with that.

Anna Aizer:

Yeah. I agree. I think they have a different perspective, which is what draws them to either defense work or prosecutorial work. But then you have to remember their jobs are really very different. So the prosecutor he or she is just dealing with the victims, so that's who they're talking to all day. The defense attorney is talking to the defendant and getting to know them and their families. They really just have very different sympathies. And the judges come from one or the other.

Scott Cunningham:

One or the other.

Anna Aizer:

So you can see it.

Scott Cunningham:

So you're a kid, you're young person. What are you feeling over the course of working with this? Tell me a little bit about your growth and the thoughts that you're thinking about.

Anna Aizer:

Yeah. I really felt like these were kids that just got derailed, that these were kids, they were in a very tough situation. They made a decision and they had no idea what the consequences of that were going to be. Nor should they have. They were 16. It's very hard to know where these things end up. I did feel as though the criminal justice system was way too harsh.

Scott Cunningham:

You could tell. Because the whole point of this nonprofit you're working on is a response to such an excessive amount of penalization. They literally don't have any room.

Anna Aizer:

Yeah.

Scott Cunningham:

Yeah. They don't have any room for anybody.

Anna Aizer:

Yeah. They had no room. That's exactly right.

Scott Cunningham:

We're doing so much punishment we can't even do it right.

Anna Aizer:

That's exactly right. In the juvenile and criminal justice system, more generally, there's a disproportionate involvement of Black and Hispanic youth.

Scott Cunningham:

Yeah.

Anna Aizer:

But they are 100% poor.

Scott Cunningham:

Yeah. Right.

Anna Aizer:

So that's the other thing. And that just seemed incredibly unfair to me.

Scott Cunningham:

Yeah, yeah, yeah. Right.

Anna Aizer:

And it's not the case that not poor kids don't also mess up. They do.

Scott Cunningham:

They just can avoid the 10,000... There's 10,000 events from the mess up to the things that these kids are facing in this program that they have many ways of mitigating it.

Anna Aizer:

Yeah. That's right.

Scott Cunningham:

There's even in terms of parents spending a ton of money, or just saying you can't hang out with these people. There's a bunch of stuff that poor families just are like... So you're feeling heavyhearted.

Anna Aizer:

Yeah.

Scott Cunningham:

You could have gone in a different direction. You could have not gone to graduate school or gone to get this master's. What's the decision criteria where you're thinking I've got to go in a new direction?

Anna Aizer:

Yeah. At a certain point I just felt as though I needed more training. I wanted more of a professional degree, so I got a degree in public health where you learned a lot about the health system and financing and the social determinants of health. I felt like I needed, again, more training. I should say, I went from that job, not directly back to graduate school, but I went and I worked in not a homeless shelter, but a service center for homeless people also in New York City. I went from the criminal justice system to the homeless system. I was there for another year. And then I went back to school.

Scott Cunningham:

To what, two or three years total between Amherst and graduate school?

Anna Aizer:

That's correct. Yeah.

Scott Cunningham:

Yeah.

Anna Aizer:

That's correct.

Scott Cunningham:

It's interesting you go to public health because I think a lot of people that don't know anything about anything, they'll be like, well, she's doing criminal justice so I could have seen her going to law school. Now she's going to the homeless thing. Okay, well, maybe she could do social work. What were the things you were thinking of? And how did you end up choosing public health? Because a lot of people don't associate either of those things with public health. They heard the word health.

Anna Aizer:

Right. So a couple things. One, I thought about law school, but I felt as though lawyers deal with the problem after it's happened.

Scott Cunningham:

Right.

Anna Aizer:

And I felt like maybe we should focus more on preventing.

Scott Cunningham:

Right.

Anna Aizer:

And the other thing, when I worked with homeless people I really did start to feel like this was a homeless individuals... Homeless families are different. I worked with homeless single adults, and for the most part in New York City at that time, all of the homeless single adults had serious mental health problems.

Scott Cunningham:

Yeah. Right.

Anna Aizer:

I really came to see homelessness as a public health problem.

Scott Cunningham:

A mental health problem.

Anna Aizer:

Yeah.

Scott Cunningham:

They hit public health. Got it.

Anna Aizer:

Yeah.

Scott Cunningham:

Right. Right.

Anna Aizer:

So that's really how... I could have done social work, but that's not really what I wanted to do.

Scott Cunningham:

Yeah. But it's funny you say preventative. To me when I hear that I'm thinking, oh, Anna's already starting to think about public policy.

Anna Aizer:

Yeah. I think I was.

Scott Cunningham:

I wouldn't necessarily think that if you were to tell me you went and got a master's in social work.

Anna Aizer:

Yeah. No, I think that's [inaudible 00:31:54]-

Scott Cunningham:

Because that cold be clinical or much more working with the... You would've had that experience and you'd be like, I want to work with these families. But that's not what you thought, so something else is going on. So you're thinking I want to do what?

Anna Aizer:

Yeah. I think I really was interested in policy already then.

Scott Cunningham:

Yeah. And that makes the masters of public health make a lot of sense.

Anna Aizer:

Correct. Yeah.

Scott Cunningham:

I see. So where'd you end up going, Harvard?

Anna Aizer:

I went to Harvard. Yeah. I got a masters in health policy and administration. And then I moved to DC. I worked for Mathematica policy research for two years, and I learned a lot about policy research.

Scott Cunningham:

Are you getting a quantitative training at the master's of public health when you went?

Anna Aizer:

Yeah, so that's where I really took my first micro theory class and my first statistics class. So I took biostatistics and micro theory there. And when I worked at Mathematica, I worked with a lot of economists. So most of the senior researchers at Mathematica were economists by training. That's where I really got exposure to the way economists think about, research and policy evaluation. It was then that I decided I wanted to go back and get a PhD in economics.

Scott Cunningham:

Okay. So what was it? What's the deal? Why do you like economics at this point?

Anna Aizer:

The senior researchers at Mathematica were either economists or sociologists or political scientists. I just felt like the economists had a very clear way in which they set up problems. I think that goes back to economic models of decision making.

Scott Cunningham:

Yeah. Right.

Anna Aizer:

And it just struck me that that was just a very good way to conceptualize almost any problem. I also liked the way they thought about data. I think the people that I worked most closely with and came to admire were all economists. So that's how that-

Scott Cunningham:

And how long were you there? Were you doing public policy stuff at Mathematica?

Anna Aizer:

Yeah. I was doing a lot of evaluations of Medicaid programs. In particular, Medicaid managed care, moving from a different financing model for Medicaid and evaluating that, and various settings, and writing them policy briefs so that... God. It was either two or three years, I can't really remember, maybe three years. I think I was there three years and then I went back to graduate school.

Scott Cunningham:

And then you go to UCLA?

Anna Aizer:

And then I went to UCLA.

Scott Cunningham:

Am I right that you were working mainly with Janet Curry?

Anna Aizer:

Yes. So Janet Curry was my-

Scott Cunningham:

You worked pretty closely with her?

Anna Aizer:

Yeah. She was my main advisor. The other folks I worked with were Joe Huts and Jeff Grogger.

Scott Cunningham:

And who?

Anna Aizer:

Jeff Grogger.

Scott Cunningham:

Oh, Jeff Grogger?

Anna Aizer:

None of whom are there anymore.

Scott Cunningham:

Yeah, yeah, yeah, yeah. Right. Right. I'm just curious. I associate you a lot with... Because I wrote that book on causal inference I'm obsessed with the causal inference stuff in all these weird ways, with all the people. I see Princeton industrial relations section, Card, Angres, et cetera. And then I see Janet Curry. And then I see you at UCLA, and I associate you so much with that methodological approach, especially for some of the papers that I've known really well. Did you get a sense when you were at UCLA, oh, this is causal inference, this is different, this is the credibility revolution? Or was it just really subtle, or this is just how you do empirical work?

Anna Aizer:

That's a great question. So I should also say that my first year econometrics teacher was Hero Inmans.

Scott Cunningham:

Was it, really?

Anna Aizer:

Yeah. Hero [inaudible 00:36:18] UCLA.

Scott Cunningham:

Oh my gosh. I didn't know that.

Anna Aizer:

For a short period of time. I was lucky enough that he was there when I was there. So he taught me in my first and my second years. So of course he was very much big part of this. And actually Enrico Moretti was also at UCLA when I was there, so I took courses with him. I think between Janet, Hero, Enrico and Joe Huts, they were really in the thick of it. That was the way it was done.

Scott Cunningham:

That was the way it was done.

Anna Aizer:

That was the way it was done.

Scott Cunningham:

Yeah. What did you learn? What do you think the salient concepts were that had you... This is a make believe, right? But I'm just saying, had you gone to a different school where you didn't have any of those people, what do you think the salient econometric causal inference kind of things were to you that you were like, oh, this is what I notice I keep doing over and over again, or keep thinking about?

Anna Aizer:

Well, I would say that the method was in service to the question. I feel as though I'm seeing it more these days. People, they find an experiment, a natural experiment, and then they figure out the question. That's not how I remember it. You had the question and then the method was in service to that question. I worry that that's getting a little bit lost these days, that people have the experiment and then they're searching for the question. I think that ends up being less interesting and less important.

Scott Cunningham:

Yeah, yeah, yeah. Yeah. There were certain economists, I think, that were so successful as approaching it that way. It seems like it was cut both ways, because it seems like applied causal inference grew on the back of that kind of natural experiment first, but it almost becomes... To a kid with a hammer, everything's a nail, so it's just like, look through the newspaper, look for a natural experiment. What can I do? How can I do this? How can I [handle 00:38:49]?

Scott Cunningham:

And it is funny. I don't think it's as satisfying too, just even emotionally. I guess you can find discoveries that way, like you were, but it does feel like you don't end up building up all the human capital with the importance of that question. It's almost like, you're like, well, how can I make this question really important? As opposed to it is important.

Anna Aizer:

Right.

Scott Cunningham:

What were you studying? I know what you were studying. At UCLA, what was the question that you were really captivated by?

Anna Aizer:

So I was really focused on health. You have to remember, I'd done a master's in public health and I just worked at Mathematica, so I was really focused on health. So really all of my dissertation was on health. My main dissertation chapter was actually on Medicaid in California. It was on the importance of enrolling kids early in Medicaids. I don't know if you know much about the Medicaid program, but there are many kids, 60% of kids, who are uninsured are actually eligible for the Medicaid program, but not enrolled in the Medicaid program. And that's partly because-

Scott Cunningham:

60%?

Anna Aizer:

Yeah.

Scott Cunningham:

Wow.

Anna Aizer:

We could reduce the number of kids who are uninsured in this country by more than half if you just enrolled all those kids who were eligible for Medicaid in the program.

Scott Cunningham:

Yeah.

Anna Aizer:

And part of the-

Scott Cunningham:

We saw that in that Oregon Medicaid experiment.

Anna Aizer:

Yeah. Oregon was mostly adults. I don't know how these numbers differ for adults and kids. I'm really more focused on kids. It's partly by design because Medicaid is a program. If you show up at the hospital and you don't have insurance and you're eligible for Medicaid, the hospital will enroll you. And most people know that.

Scott Cunningham:

Oh, is that right?

Anna Aizer:

Yeah. I mean, because they have every interest. They want to get paid, so they'll enroll you in the Medicaid program, but there's a cost to that. Because what that means is that kids, if parents know that once they go to the hospital their kid will be enrolled in the Medicaid program should they need hospitalization, they don't end up getting them enrolled prior to that. So they miss out on the ambulatory preventative care that might prevent them from being hospitalized to begin with. And that's partly because of the structure of the program, but that's also because the states made it difficult for kids to enroll in the Medicaid program. In California, there was a big change. The application for Medicaid used to be 20 pages long. Imagine that, right? They cut it down to four.

Scott Cunningham:

What kind of stuff are they asking on those 20 pages?

Anna Aizer:

Who knows? Who knows what they're asking.

Scott Cunningham:

Good grief. I mean, they're wanting them on there. Are they screening them out or are they just-

Anna Aizer:

I think that's partly what they were trying to do, right?

Scott Cunningham:

Screen them out? Because it's expensive.

Anna Aizer:

It's expensive.

Scott Cunningham:

You've got some of these legislators, they're like, this is expensive and I don't even want to do this so add a dozen pages.

Anna Aizer:

Yeah. So just make it hard. Now, what happened in '97 was the child health insurance program, CHIP. And they said, "If you want CHIP money..." So that's federal money to ensure more kids. "If you want CHIP money, federal money, you are going to have to enroll more kids in the Medicaid program. You have to do outreach." So the states actually were forced, and that's actually what prompted California to go from a 20 page application to a four page application. They also spent about $20 million on advertisement and basically training community based organizations in how to complete a Medicaid application. So they train them. "Here, you can help your clients enroll in Medicaid. For every application that you help that ends up getting onto the Medicaid program we'll give you 50 bucks." And this really mattered. A lot of kids started enrolling in the Medicaid program who otherwise wouldn't, particularly Hispanic and Asian American kids.

Scott Cunningham:

Is this what your dissertation ends up being about?

Anna Aizer:

This is what my dissertation is about.

Scott Cunningham:

On both the shortening and the payment?

Anna Aizer:

So it was basically once they started doing this you started seeing big increases in the number of kids who were enrolled in the Medicaid program. And you saw declines in hospitalizations for things like asthma. Asthma is a condition for which if you're being seen and treated on an ambulatory basis, you shouldn't end up in the hospital.

Scott Cunningham:

Oh. Wait. So what's your control group and all this stuff?

Anna Aizer:

What the state did was they targeted different areas, and provided training to those community based organizations in how to complete a Medicaid application. So they gave me all that data.

Scott Cunningham:

Get out of here.

Anna Aizer:

So I had all the data.

Scott Cunningham:

So you're doing some IB thing? You're doing some-

Anna Aizer:

Yeah. It was, basically if you live in a neighborhood where a community based organization had already been trained then you were much more likely to be enrolled in the Medicaid program. So you can see that.

Scott Cunningham:

Oh my gosh. This is so cool. Were you excited when you found that?

Anna Aizer:

I was super excited.

Scott Cunningham:

I bet.

Anna Aizer:

I was super excited. This was so old. I was begging Medicaid to send me this data. Begging, begging, begging. And they weren't really answering. And then one day Janet came in to the office where all the graduate students sit, and she said, "I think I got this fax for you." She handed this 20 page fax that has all the data on what community organization got trained and when.

Scott Cunningham:

Okay. Anna, I want to ask a meta question real quick. You just said, these days people maybe start with natural experiment first, but originally it was question first. Okay. Not devil's advocate, but just a statement of facts. The one reason they may do that is because when you find these kinds of natural experiments or whatever, it almost just feels almost itself random. You're weren't even really looking for it. You read something in the newspaper, you're like, oh my gosh, they're doing this weird thing. And the risk of going question first is, you could have this incredibly important question, like the Medicaid project payment thing, and you're like, if everybody in my department, like Hero Inmans and Moretti and Curry, who are to answer a question either subtly or not so subtly, or to answer a question is going to require this credible design and we really need you to staple this dissertation together. You're going to have to have a-

Anna Aizer:

I think that's why you have lots-

Scott Cunningham:

It seems really risky. It seems really risky.

Anna Aizer:

Yeah. I think you have to have lots of ideas.

Scott Cunningham:

You have to have lots of ideas.

Anna Aizer:

I think you have lots of ideas. A good friend of mine in graduate school was Enrico Moretti's RA. He told me that Enrico had tons of ideas. Wes, this was my friend, his RA, would just do some really quick takes on all of these ideas. And if there was something there he'd pursue it. But if there was nothing there he'd drop it.

Scott Cunningham:

What does that mean, nothing there, something there? What does that mean?

Anna Aizer:

Either, if you can't find exaggerate variation or the exaggerate variation doesn't actually work, you don't have the first stage, he'd just drop it and move on to something else.

Scott Cunningham:

That's a skill. That's almost some therapeutic skill to be excited about something and willing to let it go.

Anna Aizer:

Yeah. I think that's right. I think that's actually-

Scott Cunningham:

You got a lot of ideas?

Anna Aizer:

I had a lot of ideas. It never worked out.

Scott Cunningham:

Never worked out. And that's normal.

Anna Aizer:

I think that's normal.

Scott Cunningham:

Yeah. That's not a bad thing.

Anna Aizer:

Yeah. I think that's how research should go. In fact, I'm not as good as Enrico, I probably hold on to things for longer than I should.

Scott Cunningham:

Yeah, yeah, yeah, yeah. Boy, where'd you end up publishing that work? I should know this, but I don't know.

Anna Aizer:

That published in Restat Review Economics Institute.

Scott Cunningham:

Oh, cool. So what'd you end up finding?

Anna Aizer:

So what I end up finding is if you pay these organizations to enroll... Well, a couple things. Advertisement, just blanketing the television and radio with information. Sign up for Medicaid, sign up for CHIP, that does not work at all.

Scott Cunningham:

Doesn't work?

Anna Aizer:

No.

Scott Cunningham:

Advertising doesn't work?

Anna Aizer:

It doesn't work. What works is having these communities organizations help families complete the application. That's incredibly important.

Scott Cunningham:

That's a supply demand kind of philosophy that you see in drugs, too. Mark Anderson has this paper on meth. They would post these advertisements of people that were addicted to meth. They look horrible. They lose their teeth and all this stuff. It didn't do anything.

Anna Aizer:

Yeah.

Scott Cunningham:

Maybe I'm wrong, but it seems like you're talking about a group of people. They're like, they need more assistance. They need somebody... You think about that thing you were saying earlier about these kids that are higher income versus lower income. When I said there were 10,000 steps that the higher income people had, it wasn't really like the kids, it was external forces that were investing, going after them.

Anna Aizer:

Yeah. Right.

Scott Cunningham:

It seems like incentives need to be targeted to people to go after. For whatever reason it is not enough to just simply have it. You need people going in and helping along the way.

Anna Aizer:

Right. Agreed. I agree. They need support.

Scott Cunningham:

They need support.

Anna Aizer:

Yeah.

Scott Cunningham:

Okay. So that is amazing. I bet your advisors were so proud of you for that project.

Anna Aizer:

I don't know.

Scott Cunningham:

I think so.

Anna Aizer:

You'd hope so, but that'll be icing on the cake.

Scott Cunningham:

Right. Exactly. Yeah. I guess that's not super important.

Anna Aizer:

Yeah, it is. You do always want your advisor... I mean, I had tremendous respect for all my advisors. So yeah, I'd be very pleased if they liked the work that I did. Basically, states did spend this money to enroll kids early, but it paid off because it meant that they were less likely to be hospitalized. In fact, some of these programs can be very much cost effective.

Scott Cunningham:

Yeah. Yeah. I had told myself, I was like, well, I'm asking Anna about the juvenile incarceration paper with Joe Doyle. And then I was going to ask her about domestic violence. And I feel like I've got to make a hard choice now, because I don't have a lot of time. So I was thinking, well, let's see how this goes. And then we can fit. So domestic violence. First thing I want to ask is, how did you get interested in that topic? And when did it start? In a way I could almost imagine, oh, you've been thinking about domestic violence forever.

Anna Aizer:

Yes. So I actually-

Scott Cunningham:

You've been thinking about women ever since college.

Anna Aizer:

Yeah. That's true. And made that connection. This was basically my first big project after I started at Brown. After my dissertation I was thinking, okay, what's my next big project going to be? And I think that's a very important decision for junior faculty to think about. After you finish publishing your dissertation you got to think about what's my next big project? Because it takes so long to publish anything in economics, that's really going to matter a lot. That might be the only thing you publish before you're coming up for tenure given how long.

Anna Aizer:

I was thinking about it, and I just felt like I didn't have a clear question in mind, but just been looking at the numbers it's incredibly prevalent, domestic violence. But it's also shown some pretty encouraging trends. Domestic violence against women has been declining pretty significantly. In the US, I think about... I haven't looked the number up recently, but it was about 1,000 women a year were being killed, and so many more actually are victims of domestic violence. And if you look at victimization surveys, between one and three and one in four women in the US report ever being the victim of domestic violence. It's really prevalent. And it just struck me, this is a big problem and I don't know how to answer it, but we should know more about it given just how prevalent it is. And so that's how I started.

Anna Aizer:

I have a good friend from high school, and she's a lawyer in New York City. She was working with victims of domestic violence. She's a lawyer by training. She used to say, "These women have nothing. They have no resources. They are so poor." That, to me, just made me think about, okay, I need to start thinking about income and resources and poverty and domestic violence, because clearly that's a big part of this.

Scott Cunningham:

Yeah. Yeah. It's so funny. I feel like you and I ended up responding to the bargaining theory papers in the exact same way. That's when I was studying a lot of my stuff on couples and things and bad behavior on the part of the men, I was always thinking about sex ratios in the marriage market. Why I was thinking about that was the ability to exit the partnership could be really, really important. And I was curious. You can talk about people not having resources and not necessarily be thinking in terms of one of these Nash bargaining, like Manser and Brown, and McElroy and Horn, and Shelly Lundberg kinds of ways of thinking. I was curious, were you thinking about those theory papers a lot? Or am I just projecting?

Anna Aizer:

I had this friend, again, who was working and telling me just how poor many of the women she was working with were. And then once you actually look at the statistics, the survey statistics, it's true that any woman can be a victim of domestic violence, but it is really a poor woman problem. So it's very clear to me that poverty has a lot to do with it. It's because many of these women have no other source of support. They have low levels was in schooling. They have few prospects in the labor market. And they're really stuck. That is ultimately-

Scott Cunningham:

Stuck as in cannot leave.

Anna Aizer:

Cannot leave. I mean, they have a very-

Scott Cunningham:

Because that's the solution. That's one of the most important solutions, which is probably you need to leave the relationship.

Anna Aizer:

Yeah. Or you need to be able to threaten to leave.

Scott Cunningham:

You need to be able to threaten to leave. How important do you think the credible threat is? Because my sense is, that's to an economist, because they're like, you should thinking about unions and stuff. They're like, oh, credible threats. That's all you got to, you have to do it. I feel like, I don't know if that really works. I actually think the truth is you're going to have to leave. And maybe there's some marginal guy. We're talking about the marginal guy, but whatever, that's the info marginal, whatever. The extensive marginal guy, he's got narcissism personality disorder, substance abuse problems.

Anna Aizer:

Yeah. You may be right.

Scott Cunningham:

He's got major, major problems. And that stuff is very inelastic to everything.

Anna Aizer:

Yeah. You may be right. I can't answer this because I don't know for sure. At the same time I remember talking to some folks about this, and their feeling was that it's all a continuum of a bad relationship. Violence may be one extreme, but relationships have ebbs and flows. They can be better at some points and worse at others. So they did feel as though a relationship didn't always have to be violent, that you could have relationships that were violent at one point but then were no longer. Of course, you also have relationships in which that's not the case, and the only solution is to leave. But there could very well be relationships where you can have better and worse periods.

Scott Cunningham:

Yeah. The reason why I bring it up is because I feel like these days you hear a lot about mental health. Well, you hear about mental health period, but in domestic violence there'll be also an emerging story of the narcissist personality disorder. I've been always lately thinking, I've been like, I wonder if this is true. Anecdotally, what you see a lot is how manipulative... And that's like a very judgemental way of putting it, but I don't know how else to say it. How manipulative one of the person can be towards the other where they're like, "Well, if you loved me..." They get all this trepped up stories about love. What love becoming almost this story.

Scott Cunningham:

I've wondered for those people that can't or won't... It's actually won't, right? They can leave. I mean, there are some people they will be literally harmed if they leave, so I'm not talking about those people. But I mean, the person that literally you're watching an equilibrium where they don't leave, I've wondered lately if it's like, the victim is all tangled up with loyalty and love.

Anna Aizer:

Yeah. Sure.

Scott Cunningham:

And it is taken advantage of by a person that no one can tell them not to love this person. That's nobody's business.

Anna Aizer:

Yeah. Yeah. I mean, it is a really complicated thing.

Scott Cunningham:

It is so complicated. It is so complicated. Finding the policies that provide resources to a person. Some of that might be a person that's at those earlier ebbs too, those earlier ebbs in the bad relationship. And you're like, well, some people may not be ready to leave yet.

Anna Aizer:

I mean, this a thing where I do think the right policy response is providing resources to women, but also probably interventions aimed at the assailant is probably going to be just as effective. Sorry. My phone is ringing.

Scott Cunningham:

That's okay.

Anna Aizer:

Hello. Sorry about that. I thought it might be my kids.

Scott Cunningham:

I wonder about these battery courts. Have you heard about these [inaudible 01:00:05] courts?

Anna Aizer:

Yeah. I mean, they're-

Scott Cunningham:

I wonder what you know about those?

Anna Aizer:

Yeah. Not a lot, I would say.

Scott Cunningham:

Yeah. These issues of poverty and mental health and all of these things interacting in order to get healing and healthy meaningful lives to all everyone is... I do think this is something that economists can offer, but it's not something that... I wouldn't say there's a ton of people. You're one of a small number of people working on domestic violence, it seems like.

Anna Aizer:

It's a very hard thing to study. Data's very difficult to come by for obvious reasons, for a good reason. I mean, this is data that needs to be protected. Glenn Ludwig and the crime lab in Chicago, they're doing work around violence reduction more generally. And probably many of those principles and findings probably relate to domestic violence as well, changing the behavior of young people so that they are less quick to react and less quick to react in a violent way. When they do, we would probably have some pretty important spillover to domestic violence as well, I think.

Scott Cunningham:

Yeah. Yeah, yeah.

Anna Aizer:

I think there are ways to reduce violence more generally that would probably apply to the setting of domestic violence.

Scott Cunningham:

Yeah. It's funny, circling back to that judge who threatens with higher penalties. I think economists, when they think about violence and things like that, you're an exception for thinking about outside options and stuff like that, but the shadow of Gary Becker's deterrence hypothesis, it can just be this straight jacket for a lot of people, because they just only think in terms of relative price changes on the punishment margins. When you talk to psychologists, or you read that psychology literature about narcissism or borderline personality disorder or substance abuse, you're talking about a group of people that are, for variety of reasons, have really low discount rates or just have beliefs that things don't apply to them. Or in no uncertain terms, the elasticities of violent behavior with respect to some unknown punishment that you don't even know if it's going to real, it just seems like, we don't really know, but [inaudible 01:03:06] really big.

Anna Aizer:

Yeah. So there was this criminologist named Mark Kleiman. Do you know that name?

Scott Cunningham:

Oh yeah. Mark Kleiman. Yeah. Yeah.

Anna Aizer:

I mean his big thing was, it should be swift, sure and short. That's how we should do punishment. He felt as though that would be far preferable to the system in which there's uncertainty. But if it doesn't work out, you're going to spend a lot of time in jail. He thought that was a fair model.

Scott Cunningham:

The thing is, though, swift certain and did you say short?

Anna Aizer:

Short.

Scott Cunningham:

Yeah. Well, with prison sentences lingering on your record it is by definition never short.

Anna Aizer:

Yeah.

Scott Cunningham:

You face these labor market scarrings and you can't get housing, you can't get jobs and that does not go away. So even if the prison sentence is short, the person... I just feel like this is the tension around violence in the country, which is punishment has so many margins where it is permanent. It's got so many margins. And just being in a cage is only one of them.

Anna Aizer:

Yeah. I mean, particularly for young people, jail is incredibly scarring.

Scott Cunningham:

Incredibly scarring. Incredibly scarring. We've been studying suicide attempts in the jail and we-

Anna Aizer:

Yes, that's right.

Scott Cunningham:

We walked the jail for this one particular jail. I have never in my life seen anything like that. I've been working on this project for four years. I hadn't walked to the jail. I don't know. It's not the first thing that came to my mind. The team finally walked the jail. I spent the whole day there. The jails have so much mental illness in it. They just are in... It's not even cages. A cage has... Air gets in. It's a sealed box. It's like Houdini's box. They stay there, and for a variety of regulatory reasons and so forth, they stay in there. Can't have a lot of materials if they are at risk. If they've come in with psychosis because of substance abuse or underlying mental illness stuff, they might get moved into certain types of physical quarters. I just can't even imagine, just in an hour, let alone... And that's just jail. That's not even prison. It's just absolutely a trauma box.

Scott Cunningham:

Unfortunately, we didn't get to talk about your paper with Joe Doyle on the juvenile incarceration. But every time I teach that juvenile incarceration paper, where kids were incarcerated as a young person, and then end up not going back. It's not even the future prison part, it's the not going back to high school.

Anna Aizer:

Oh, of course.

Scott Cunningham:

And then when they go back, they're labeled with a behavioral emotional disorder. It's really like anybody that's had any exposure to a kid involved in corrections, you're like, oh, I know exactly what that is. They were traumatized. You don't even have to come up with some exotic economic theory. They were traumatized. That's why they come back to school with a behavioral emotional disorder. It is [inaudible 01:06:59].

Anna Aizer:

Yep. That's good.

Scott Cunningham:

That paper is one of the most important papers I have personally ever read. I teach it nonstop. And I've even cried teaching it in class. I get so emotional when I get to that part, because, I don't know about you, but it seems like it's really hard not to come away with... A lot of papers you read, you're like, well, we're not really sure exactly all to make of it. But when I read that paper that you wrote, I just think, especially when you think about the leniency design, I just think these kids probably didn't need to go to prison.

Anna Aizer:

Oh yeah.

Scott Cunningham:

Honestly, what else are you going to say? They end up committing more crimes. And they are not going back to school. How was this the policy goal? What was it like writing that paper when you started to realize what was going on?

Anna Aizer:

Again, when I worked in this Alternative to Incarceration program we had kids come into the program who had spent some time in jail. And we had kids who had spent very little time, maybe just a night. The kids who had spent even just three weeks in jail, they always did worse in the program. Always. It was a known fact. The program knew it. And the question was, well, are these kids somehow different? There was a reason why they were in jail and these other kids weren't. Is that why they do worse in the program? Maybe they're in jail because their family didn't show up for them in court. They couldn't make bail.

Anna Aizer:

Or was it something about spending three weeks in jail that just made it impossible for them to complete the program? This was a big question that was on everybody's mind. We talked about this quite a bit at the program, and we didn't know the answer. When I finally figured out how to do it, working with Joe, I wanted to know the answer to a question that I had been thinking about for over a decade.

Scott Cunningham:

Gosh. Were you emotionally upset when you started to see coefficients get really big?

Anna Aizer:

It really was not surprising. It really wasn't, because these are kids who are only marginally attached to school. These are not the kids who were going to school, doing well in school. These are kids who were not really that attached to school for whatever reason. So you take them out even for a month, they're not going to go back. I mean, it's obvious. We saw that in the program. What they ended up doing was moving a lot of kids from school to GED because they had not been involved in school, they were not involved in school. It just was much more likely that they would be able to complete a GED than actually go back to high school and finish.

Scott Cunningham:

Your paper, it like hit home for personal reasons. We had an event happen. I wrote a professor. I was like, this thing had happened. Anna and Joe find this result. I feel hopeless. There's this kid in town and I raised money for him. Basically, I was like, you just got to do everything in your power to not let them spend an extra minute in jail. And all this scared straight stuff. Parents get into it, too. They're exhausted. They're like, "Well, he's got to learn his lesson." Nobody learns a damn thing in jail. They don't learn a lesson. Because you're just so hopeless. You start grasping at straws. And people will tell you that might happen.

Scott Cunningham:

I just keep thinking to myself, look, just get him out of jail. Whatever the folklore is, let the folklore about how to help a kid happen outside of jail. Just get him out of jail and then just let the next thing. But the thing is, they need so much help. These kids that end up getting tangled up with... They do these petty larceny things. They're high on Xanax and they do these petty larceny things. And it just starts adding up. You get them out and you just realize, you're like, oh, okay. Now where's the massive infrastructure to help them? And you're like, there's not one. So then they just keep getting arrested and they keep getting arrested. Then you're going, well, how many lottery tickets is the optimal number of lottery tickets to always be buying so that you're diversifying, so that one time maybe it clicks? And you're just like, I don't have the budget constraint for it.

Anna Aizer:

Yep.

Scott Cunningham:

I find it hard to be hopeful sometimes when you work on these projects. I think that's part of it. When you work on projects around violence and kids and jail, it can, it can make you feel hopeless. I, even with your paper, just feel like... I'm like, okay, the goal needs to be to avoid this. But then you just think, but nobody's listening to an economist.

Anna Aizer:

Well, a couple things. The rate of juvenile detention has been falling pretty steadily. The trends look very different from adult incarceration and detention, which have only started to fall more recently. Juvenile detention incarceration has been falling. Actually, I was speaking with some policy makers who said that a 2013 National Academy Of Sciences report on juvenile detention was really very impactful, and that it made very clear that detaining and incarcerating youth was a terrible idea. So lots of things have changed in response. The federal policy in this space is to do what everything can be done to reduce the number of kids that are detained and incarcerated.

Anna Aizer:

Now, most of these kids are being detained and incarcerated in local and state jails and prisons. Federal government isn't actually jailing people, so their policy levers are more limited. But policy makers, at least at the federal level and increasingly at the state and local level, do understand that incarcerating youth is a bad idea. And rates have been falling. So that's the good news.

Scott Cunningham:

Right. Right. One of the things in this podcast series I've been gradually beginning to focus on is how so much of changing policy from the perspective of the individual scientist is the long game. You are doing things, and just almost knowing that this is a part of a larger process that takes a long time, and the work matters. It matters. It's important to tell the truth and do good work and do your best. Even to get it published in the top journal so that people take it really seriously, too.

Scott Cunningham:

It is so nice to talk. I really mean it. I've felt just such a strong inspiration and things that I've learned from you and how you do ask questions and how you answer them. I watch you get administrative data. That was the thing with that Joe Doyle paper, I watched you get administrative data. I think, oh, I could get administrative. It's really been for me personally, just looking, I have watched you be an economist and just use it to navigate a little bit about, okay, Anna's doing this and it's possible to do this.

Anna Aizer:

I got to give a shout out to Joe Doyle, because he was the one that was able to access that Chicago data.

Scott Cunningham:

Golly. I mean, there's all that linking. It seems like it's an early linking paper even.

Anna Aizer:

Yeah. Shout out to Joe more generally. He was a great co-author on that project.

Scott Cunningham:

Yeah. Yeah. Well, thank you so much for being on the podcast.

Anna Aizer:

Yeah. Thank you, Scott. It was a pleasure talking to you.

Scott Cunningham:

Yeah. Okay. Bye, bye.

Anna Aizer:

Bye, bye.

Ronny Kohavi, PhD

“Economists in tech” is a podcast series of mine trying to tell the story of the movement predominantly Economics PhD talent into and throughout the emerging tech sector. Previously interviews have been with Michael Schwarz (Microsoft), Susan Athey (Stanford, now DOJ, formerly Microsoft), and John List (Chicago, Wal-mart). But this week I chose to share an interview I did a month ago with a prominent computer scientist named Ronny Kohavi.

Economists may not know about Ronny. Ronny did his PhD at Stanford in 1995, and was at ground zero to watch major advances happen in tech. His early work was in machine learning, and many of his most cited papers remain in that area too. But something that he has also been instrumentally involved in is from day one in tech being an aggressive evangelist, promoter and guide for the adoption and design of randomized controlled trials now used extensively within tech (called there A/B test not RCT). His recent book with Tang and Hu, "Trustworthy Online Controlled Experiments” discusses in detail his thoughts on this topic.

In a lot of ways, Ronny could just as easily fit in the “causal inference” series, but I chose to pin him in this because I think he is more broadly familiar to the tech sector for pushing for the randomized experimental design, and I thought that might be interesting for those of us who stand outside with curiosity tech. If you want to study with Ronny, he teaches a regular workshop at Sphere on RCTs.

Scott's Substack is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Anna Stansbury is a professor at MIT's Work and Organization Studies department. I interviewed her as part of my "economics and public policy" mixtape series. We discussed her growing up in England, and what drew her to economics, as well as her thoughts about labor market trends and other stylized facts and what she thinks they mean. I hope you enjoy this as much as I did!

Robert Michael, professor emeritus at University of Chicago, was a student of Nobel laureate Gary Becker from a productive period when Becker was at Columbia up through the late 1960s and in this interview shares a bit of why that time was so special. As you may recall, I have been doing my only little “mixtape” about Becker’s students and previously interviewed Bob’s old classmate and longtime friend, Mike Grossman. Bob describes a lot about the secret sauce that made Columbia such a special time for people like him, Mike, Bill and Elizabeth Landes, Isaac Ehrlich and many others. It wasn’t merely the chance to be mentored by Becker according to Bob; it was also Jacob Mincer and how complementary those two were — yin and yang, theory and empirical rigor. Bob would go on to helping shape the profession, not merely through his wonderful scholarship, but also through his overseeing of numerous important panel and cross-sectional datasets. The two with which I am most familiar are the National Longitudinal Survey of Youth 1997, which I wrote my dissertation on, and the National Health and Social Life Survey, a 1992 survey which was the first representative survey of adult American sexual behavior.

He co-authored two books about sex in fact — one entitled The Social Organization of Sexuality and Sex in America. Both of these books document the sexual practices of adult Americans from that early 1992 period, riding on the crest of the AIDS epidemic and helping us better understand the basic facts about sex in America. I think you will be deeply moved, though, listening to Bob describe the lengths to which they at NORC went to talk to respondents and learn about their sexual behavior was stunning and not surprising. He notes that some respondents wept during the survey because, as they said, they had literally never talked to anyone about some of these important parts of their lives, some not even their own spouses, therapists or doctors. And yet Bob had with his team at NORC created such a safe, compassionate and respectful environment that not only could he ask intimate questions to strangers, but in fact have a nearly 90% response rate of people willing to share. A true model of science — curious, careful and compassionate.

In this episode, I introduce listeners to Michael Grossman, an early pioneer in the field of health economics. His dissertation work under at Columbia University on "health capital and demand" became a cornerstone of the modern field of health economics. We discuss his time growing up in New York, his time with Gary Becker at Columbia as his student, how he got into health in the first place, and much more. Mike is a much beloved economist and I hope you will enjoy this interview as much I did.