I don't have it handy, I can look for it later if you can't find it, but back when I was in grad school for my master's degree in education, I did a research paper about this topic. I don't know what the guardian study is, but I remember being startled by a study that looked at family library size in a multitude of different countries and found it pretty impressive. Didn't see it mentioned in your article so wanted to flag it.
Great question! Hill et al (2007) estimate that the average 3rd grader increases their reading achievement by ~0.4 SD during the course of the next year. Given that the book distribution programs I looked at primarily measured achievement among 3-5 graders, a 0.05 SD impact can be roughly thought of as "12% of a grade level" improvement.
There's just so many possible other confounding factors here that I honestly don't understand this whole discussion. It seems like you agree ("It is worth pointing out, however, that the studies do not necessarily demonstrate that books in the home themselves causally impact children’s achievement. ")
I thought it would just be worth pointing some of these other possible confounding factors .Would the studies summarized here have identified the causal power of any of these hypotheses?
* The number of books in a home are correlated with some underlying "joy-of reading" factor, which is not genetic but cultural and can be passed to children by parents, and is actually more casual in the differential outcomes.
* Same as above, but it's a "joy of learning" factor
* Maybe it's a "patience " or "long attention span" or "delayed gratification" factor. Again, not genetic, but transmittable from parent to child, and simply correlated with library size.
I agree that the non-randomized studies could very well be confounded by the "joy of learning" factor. However, this factor should not confound the randomized studies since, on average, the randomly assigned groups should be equivalent in their joy of learning.
You could argue, however, that joy of learning is a moderator of the impact of books in the home (i.e. the randomized experiment only yields a positive causal impact on the subset of students who have a greater joy of learning at baseline).
The Bert and Johnson study is the most interesting one. The other two have non-representative samples (upper-middle class families from China and families pulled from a science museum? Come on). I don't have comments about the meta-analysis.
Happy to provide additional context behind my statement. In most social science/genetics papers that I've encountered, the term "educational attainment" is specifically used w/reference to the # of years of education that an individual has acquired, and not necessarily the pedigree of the institutions attended.
Thus, when I say that "to the extent that educational attainment is genetically inherited", I'm specifically referring to genes which increase the # of years of education an individual goes on to receive (and not, say, genes which increase the probability of admission to a prestigious university).
I also want to clarify, my motivation behind spotlighting the controls for parental educational attainment was not to say "therefore, genetic confounding is successfully dealt with". Rather, it was to say "genetic inheritance isn't completely ignored/unmodeled by this study". I apologize if this motivation was not communicated clearly.
I am aware that EDU is usually a crude proxy like 'years of education' and badly fails to measure educational level. It is unfortunate, but it is what it is - and that's why it's important to remember that there is still a lot of residual confounding and it is *never* true to say that you 'controlled for education', with the bait-and-switch of EDU/education. You have controlled for EDU, yes, but only 'controlled for' a small fraction of education. (Particularly with credential inflation.)
> Thus, when I say that "to the extent that educational attainment is genetically inherited", I'm specifically referring to genes which increase the # of years of education an individual goes on to receive (and not, say, genes which increase the probability of admission to a prestigious university).
That's still wrong. The residual confounding means that you have not 'already adjusted for it' even with the bait-and-switch of a really crude educational proxy. The crude proxy will still be residually confounded in the offspring. A noisy measurement of the parental genotypes doesn't stop being noisy because you take a second noisy measurement in the offspring...
That is, if you compare two people, whose parents have the same EDU (but differ starkly on the true latent education, like my example, of 2 dropouts vs 2 elite graduates), you will still find that the offspring have different EDU, in terms of raw crude years, on average (lower vs higher, respectively). The former will be much more likely to drop out of high school or drop out of college and so achieve fewer 'years' and a lower EDU, while the later will be more likely to go all the way and achieve a high EDU etc. Thus, you have not removed the residual confounding of EDU in the offspring by controlling only EDU in the parents. (Life would be far easier if that were true, if we could avoid all residual confounding by just making sure to use the same type of measurement each time!)
Ah, I understand your point. Would the following language be more appropriate in your view?
> "Thus, the study at least tries (albeit inadequately) to deal with genetic confounding using a crude proxy variable in the form of parental educational attainment."
If so, I'm happy to update the language in the article. I'm considering writing a follow-up to this article in light of the discussion it's generated, so I may note this in the follow-up.
Yes. It's true that using EDU controls is a good idea and if it's the best you can do, you should do it. But you shouldn't fool yourself that you've controlled for more than a little of the genetic (or environmental) confounds there. And it can be of interest to note how much of the original raw correlation disappears when you add in even inadequate controls. (eg. something like "If we throw in even a crude proxy like parental years of education, a quarter of the correlation disappears. This should make us wonder how much of the rest will hold up.")
Noted. I've edited the language used to describe the educational attainment controls and added a "Corrigendum" section to the article detailing what edits have been made since publication. Thanks again for your input!
Could not wealth be a confounder, not well captured by the controls (which include profession but not income)? Richer people are more likely to be able to afford books and have the space for them and that type of family might advantage their children in other ways.
I don't have it handy, I can look for it later if you can't find it, but back when I was in grad school for my master's degree in education, I did a research paper about this topic. I don't know what the guardian study is, but I remember being startled by a study that looked at family library size in a multitude of different countries and found it pretty impressive. Didn't see it mentioned in your article so wanted to flag it.
Will be on the lookout for a paper matching this description!
What does a 0.05 SD for a question like this mean in real life?
Do you have an analogy I could use to visualise the real life impact?
Great question! Hill et al (2007) estimate that the average 3rd grader increases their reading achievement by ~0.4 SD during the course of the next year. Given that the book distribution programs I looked at primarily measured achievement among 3-5 graders, a 0.05 SD impact can be roughly thought of as "12% of a grade level" improvement.
https://www.mdrc.org/sites/default/files/full_84.pdf
There's just so many possible other confounding factors here that I honestly don't understand this whole discussion. It seems like you agree ("It is worth pointing out, however, that the studies do not necessarily demonstrate that books in the home themselves causally impact children’s achievement. ")
I thought it would just be worth pointing some of these other possible confounding factors .Would the studies summarized here have identified the causal power of any of these hypotheses?
* The number of books in a home are correlated with some underlying "joy-of reading" factor, which is not genetic but cultural and can be passed to children by parents, and is actually more casual in the differential outcomes.
* Same as above, but it's a "joy of learning" factor
* Maybe it's a "patience " or "long attention span" or "delayed gratification" factor. Again, not genetic, but transmittable from parent to child, and simply correlated with library size.
I agree that the non-randomized studies could very well be confounded by the "joy of learning" factor. However, this factor should not confound the randomized studies since, on average, the randomly assigned groups should be equivalent in their joy of learning.
You could argue, however, that joy of learning is a moderator of the impact of books in the home (i.e. the randomized experiment only yields a positive causal impact on the subset of students who have a greater joy of learning at baseline).
The Bert and Johnson study is the most interesting one. The other two have non-representative samples (upper-middle class families from China and families pulled from a science museum? Come on). I don't have comments about the meta-analysis.
I have a comment on this Post:
https://open.substack.com/pub/federicosotodelalba/p/a-comment-on-the-post-here-on-substack?r=4up0lp&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true
> First, it’s worth pointing out that the Guardian study controls for both paternal and maternal educational attainment.
>
>> We control for gender, age, parental education in years obtained by averaging maternal and paternal education and respondent's education in years
>
> Thus, to the extent that educational attainment is genetically inherited, this study is already adjusting for it.
That's not even close to true, unless you believe graduating from Caltech == 4 years at community college & dropping out without a degree.
Happy to provide additional context behind my statement. In most social science/genetics papers that I've encountered, the term "educational attainment" is specifically used w/reference to the # of years of education that an individual has acquired, and not necessarily the pedigree of the institutions attended.
Thus, when I say that "to the extent that educational attainment is genetically inherited", I'm specifically referring to genes which increase the # of years of education an individual goes on to receive (and not, say, genes which increase the probability of admission to a prestigious university).
I also want to clarify, my motivation behind spotlighting the controls for parental educational attainment was not to say "therefore, genetic confounding is successfully dealt with". Rather, it was to say "genetic inheritance isn't completely ignored/unmodeled by this study". I apologize if this motivation was not communicated clearly.
I am aware that EDU is usually a crude proxy like 'years of education' and badly fails to measure educational level. It is unfortunate, but it is what it is - and that's why it's important to remember that there is still a lot of residual confounding and it is *never* true to say that you 'controlled for education', with the bait-and-switch of EDU/education. You have controlled for EDU, yes, but only 'controlled for' a small fraction of education. (Particularly with credential inflation.)
> Thus, when I say that "to the extent that educational attainment is genetically inherited", I'm specifically referring to genes which increase the # of years of education an individual goes on to receive (and not, say, genes which increase the probability of admission to a prestigious university).
That's still wrong. The residual confounding means that you have not 'already adjusted for it' even with the bait-and-switch of a really crude educational proxy. The crude proxy will still be residually confounded in the offspring. A noisy measurement of the parental genotypes doesn't stop being noisy because you take a second noisy measurement in the offspring...
That is, if you compare two people, whose parents have the same EDU (but differ starkly on the true latent education, like my example, of 2 dropouts vs 2 elite graduates), you will still find that the offspring have different EDU, in terms of raw crude years, on average (lower vs higher, respectively). The former will be much more likely to drop out of high school or drop out of college and so achieve fewer 'years' and a lower EDU, while the later will be more likely to go all the way and achieve a high EDU etc. Thus, you have not removed the residual confounding of EDU in the offspring by controlling only EDU in the parents. (Life would be far easier if that were true, if we could avoid all residual confounding by just making sure to use the same type of measurement each time!)
Ah, I understand your point. Would the following language be more appropriate in your view?
> "Thus, the study at least tries (albeit inadequately) to deal with genetic confounding using a crude proxy variable in the form of parental educational attainment."
If so, I'm happy to update the language in the article. I'm considering writing a follow-up to this article in light of the discussion it's generated, so I may note this in the follow-up.
Yes. It's true that using EDU controls is a good idea and if it's the best you can do, you should do it. But you shouldn't fool yourself that you've controlled for more than a little of the genetic (or environmental) confounds there. And it can be of interest to note how much of the original raw correlation disappears when you add in even inadequate controls. (eg. something like "If we throw in even a crude proxy like parental years of education, a quarter of the correlation disappears. This should make us wonder how much of the rest will hold up.")
Noted. I've edited the language used to describe the educational attainment controls and added a "Corrigendum" section to the article detailing what edits have been made since publication. Thanks again for your input!
Could not wealth be a confounder, not well captured by the controls (which include profession but not income)? Richer people are more likely to be able to afford books and have the space for them and that type of family might advantage their children in other ways.