https://astralcodexten.substack.com/p/against-that-poverty-and-infant-eegs

A recent paper claims to have found an Impact Of A Poverty Reduction Intervention On Infant Brain Activity. It’s doing the rounds of the usual media sites, like Vox and the New York Times:

The New York Times @nytimes Breaking News: Cash payments for low-income mothers increased brain function in babies, a study found, with potential implications for U.S. safety net policy. Cash Aid to Poor Mothers Increases Brain Activity in Babies, Study FindsThe research could have policy implications as President Biden pushes to revive his proposal to expand the child tax credit.nyti.ms

January 24th 2022

3,348 Retweets13,165 Likes

I was going to try to fact-check this, but a bunch of other people (see eg Philippe Lemoine, Stuart Ritchie) have beaten me to it. Still, right now all the fact-checking is scattered across a bunch of Twitter accounts, so I'll content myself with being the first person to summarize it all in a Substack post, and beg you to believe I would have come up with the same objections eventually.

Before we start: why be suspicious of this paper? Hundreds of studies come out daily, we don't have enough time to nitpick all of them. Why this one? For me, it's because it's a shared environmental effect being measured by EEG at the intersection of poverty and cognition.

Shared environmental effects on cognition are notoriously hard to find. Twin studies suggest they are rare. Some people have countered that perhaps the twin studies haven't measured poor enough people, and there's a lot of research being done to see what happens if you try to correct for that, but so far it’s still controversial.

All that research is being done by cognitive testing, which is a reasonable way to measure cognition. This study uses EEG instead. I'm skeptical of social science studies that use neuroimaging, and although EEG isn't exactly the same as neuroimaging like CT or MRI, it shares a similar issue: you have to figure out how to convert a multi-dimensional result (in this case, a squiggly line on a piece of paper) into a single number that you can do statistics to. This offers a lot of degrees of freedom, which researchers don't always use responsibly.