Do "books in the home" really improve academic achievement?

Tackling a classic question in the nature vs nurture debate

Table of Contents

1. Introduction

2. More Than Just Genetics

3. What Do Randomized Experiments Say?

4. Conclusion

5. Addendum: The Argument No One Made But Should’ve, So I’m Making It

6. Works Cited

Introduction

A couple weeks back, a post extolling the virtues of in-home libraries went viral on X. “Research data from 160,000 adults in 31 countries concludes that a sizeable home library gave teens skills equivalent to university graduates” it triumphantly declared. The post linked to an article by the Guardian spotlighting a study which purported to show that having a large # of books in the home during adolescence led to higher literacy and numeracy skills during adulthood.

As you might expect, such a sensational claim was not received well.

This backlash is not at all surprising and, dare I say, completely deserved. The Guardian article completely ignores key limitations of the paper including, but not limited to, omitted variables bias, measurement issues, and collider bias.¹ Personally though, I think this situation is quite unfortunate as the central question the paper attempts to answer is actually quite important: namely, how much do books in the home actually matter for children’s educational prospects?

Having comprehensively reviewed the literature on this subject, I want to stake out the following claim: books in the home have a modest causal impact on children’s academic achievement.

Let’s dive in!

More Than Just Genetics

One of the most common criticisms raised against the Guardian study is the issue of genetic confounding. An X user by the name of Jonatan Pallesen articulated this concept using the following diagram.

The diagram highlights the following relationships:

1. Genetic inheritance: High-achieving parents give birth to children who are genetically predisposed to high achievement themselves.

2. Self-selection: High-achieving parents have many books in their home for a variety of reasons (e.g. high-achieving parents are more likely to enjoy reading).

3. Genetic confounding: Thus, a correlation between the # of books in the home and children’s achievement emerges not due to a genuine causal relationship but instead due to genetic confounding.

While I agree that genetic confounding explains some of the correlation between the # of books in the home and children’s academic achievement, I’m not convinced that genetic inheritance is solely what’s going on here (nor I am saying that Pallesen holds such a view, to be clear).²

First, it’s worth pointing out that the Guardian study controls for both paternal and maternal educational attainment.

We control for gender, age, parental education in years obtained by averaging maternal and paternal education and respondent's education in years

Thus, to the extent that educational attainment is genetically inherited, this study is already adjusting for it.

But more importantly, three independent studies using genetically-informative samples cast doubt on the idea that genetic confounding is the whole story here.

---

Van Bergen et al (2017)

Van Bergen et al (2017) account for genetic confounding by leveraging parent-child trios (i.e. two parents and their child). In particular, they survey parents on the # of books in the home and then administer an oral reading fluency assessment to both parents and the child. They then estimate the correlation between the # of books in the home and children’s oral reading fluency controlling for the reading fluency of both parents to eliminate genetic confounding.³ In more visual terms, they are blocking the genetic inheritance path highlighted in the previous diagram by controlling for the reading ability of both parents.

This strategy indicates that books in the home are as powerful a predictor of children’s oral reading fluency as maternal and paternal reading fluency.⁴

Burt and Johnson (2024)

Burt and Johnson (2024) account for genetic confounding by leveraging a sample of adoptees, their biological siblings (who were raised by the biological parents), and the rearing families of both siblings.

Using this sample, they survey both biological parents and adoptive parents on the # of books in the home and then administer an assessment to adoptees and their biological siblings to measure academic achievement. They then estimate the correlation between the difference in the # of books between the adoptive and biological homes and the difference in academic achievement between the adoptees and their biological siblings, controlling for birth complications (because biological mothers were more likely to have suffered pregnancy complications when carrying the child who was adopted).

This difference score correlation is significantly positive and robust across many different specifications, demonstrating that the correlation is not merely an artifact of genetic confounding.

Zhang, Inoue, and Geourgiou (2024)

Zhang, Inoue, and Geourgiou (2024) account for genetic confounding by applying the same method as Van Bergen et al (2017). They find that the # of books in the home is significantly associated with children’s vocabulary and character recognition skills controlling for three measures of parental reading ability.

---

Though the above studies present an interesting case against the genetic essentialist viewpoint, I would be remiss if I didn’t highlight the limitations of each study. In particular:

• The study sample in Van Bergen et al (2017) consists of 101 families who visited a Dutch science museum. Given this small and unrepresentative sample, it’s not clear that this result would generalize to a broader setting.⁵

• The academic achievement test in Burt and Johnson (2024) was administered to children aged 7 years old. To the extent that environmental influences dissipate with age, their results may not persist long-term.

• The study sample in Zhang, Inoue, and Geourgiou (2024) consists of 168 upper-middle class families with children attending public school kindergarten in China. Given this small, young, and unrepresentative sample, it’s not clear that this result would generalize to a broader setting.

Nevertheless, I think these studies warrant skepticism of the genetic essentialist viewpoint given the consistency of results across a diverse collection of samples.

It is worth pointing out, however, that the studies do not necessarily demonstrate that books in the home themselves causally impact children’s achievement. As Van Bergen et al (2017) astutely point out, even these genetically-adjusted associations may still be the byproduct of confounding due to other environmental variables:

Our findings suggest that children’s reading levels are influenced by the size of the home library partly via the environment. This could be due to the importance of the number of books in itself, or the number of books could be a proxy for how much value the family places on reading.

Thus, additional evidence is needed to evaluate whether books in the home truly improve children’s achievement.

What Do Randomized Experiments Say?

Niskanen Employment Policy Fellow Matt Darling provided the following comment in response to the Guardian study:

While Darling’s comment is clearly intended as a joke, it turns out that several studies have, in fact, randomly distributed books to children - even going so far as to distribute e-readers - to estimate the causal impact of books in the home on academic achievement. What impacts do these studies reveal? Well … the devil is in the details.

In my search, I identified 10 randomized experiments which tested the impact of randomly distributing books to children on subsequent reading achievement. I then utilized a statistical procedure known as random effects meta-analysis to aggregate the results of these studies into a single effect size which represents the average impact of books in the home on children’s reading achievement. Importantly, this average effect conceals substantial heterogeneity across the studies:

• Children’s academic achievement was measured at different times. Most studies distributed books to children at the end of the school year and measured achievement at the start of the next school year. But some studies measured achievement using end-of-school-year tests administered 1 year after the intervention began.

• Not all interventions lasted the same amount of time. Most studies randomly distributed books to children over the course of a single summer. But some studies randomly distributed books to children over the course of multiple years.

• Study samples varied in socioeconomic status. Several studies focused on economically disadvantaged populations as measured by various proxy variables (e.g. eligibility for free/reduced-price lunch) while others focused on more socioeconomically representative populations.

• Study samples varied in grade level. Some studies targeted children in early elementary school while others targeted children in mid-to-late elementary school. To the extent that intervention efficacy is dependent on children already being proficient in certain reading skills (e.g. decoding), we might expect books in the home to be less impactful for younger students.

• Reading achievement was measured using different assessments. Across the studies considered, reading achievement was measured using 8 different assessments.

• Books were distributed to children using different procedures. Some studies provided children with books matched to their reading comprehension level and interests while others simply allowed children to self-select books with few constraints.

• Interventions varied in the amount of scaffolding. Some interventions included additional scaffolding designed to encourage students to read the books that they were given (e.g. postcards mailed to the students over the summer, a lesson on reading comprehension strategies given to students before the intervention) while other interventions included no scaffolding at all.

With this context in mind, we can take a look at the meta-analytic effect sizes on reading achievement and oral reading fluency.

The meta-analysis demonstrates that, on average, distributing books to children increased reading achievement by 0.05 standard deviations (p < 0.001)⁶ …

… and had a non-significant impact on oral reading fluency.

I imagine that some might be disappointed by the relatively marginal impact of books in the home on reading achievement. However, I think that there is an important silver lining hidden within these results: namely, cost effectiveness.

To elaborate, Lortie-Forgues and Inglis (2019) find that the average achievement effect size observed in rigorously-conducted randomized educational interventions is 0.06 SD - quite close to the meta-analytic effect size estimated in this article! Thus, a key takeaway in my mind is that even a low-cost intervention like distributing books for children to read during the summer may rival the impact of costlier interventions.

A curious reader might also wonder whether books in the home have a greater impact on low-income children, who are most likely to benefit from such an intervention. To investigate this hypothesis, I used a statistical procedure known as meta-regression to examine how the average effect size varied as a function of the percentage of welfare-dependent students (e.g. eligible for free/reduced-price lunch). This meta-regression yields the following result:

We can see that the effect size does, in fact, increase as the sample becomes increasingly low-income. Importantly, however, this relationship does not achieve statistical significance and should be interpreted with caution until additional future studies can be incorporated into this meta-analysis.⁷

Before concluding, I want to note that this is my first rodeo with meta-analyzing academic literature, and as a result, I’m completely open to the possibility that I’ve made a mistake at some point during this process. To that end, I invite statistically-oriented readers to look through the attached GitHub repository, and notify me of possible issues (e.g. incorrectly calculated effect sizes or incorrect meta-analytic procedures). Any and all constructive feedback is deeply appreciated.

Conclusion

Well, there ya have it folks: the best available evidence suggests that books in the home have a modest causal impact on children’s reading achievement.

Addendum: The Argument No One Made But Should’ve, So I’m Making It

While most criticism of the Guardian study focused on omitted variables bias (i.e. not controlling for confounding variables), the more devastating criticism of the study, in my view, is post-treatment bias. Post-treatment bias occurs when one controls for a consequence of the treatment in question. Montgomery, Nyhan, and Torres (2018) provide the following illustrative example:

Consider a hypothetical randomized trial testing whether a civic education program increases voter turnout in a mixed-income school. In this example, we would estimate the effect of the intervention by comparing the turnout rate among those assigned to receive the civic education treatment with those who were not.

…

Imagine, for instance, that we wish to control for political interest of the subjects (as measured after the treatment) so that we can understand the effect the civic training class independent of subjects’ political awareness.

…

Once we condition on the political interest variable by subsetting the data on political interest or including it as a covariate in a regression, we are now comparing the turnout rate of individuals who had low political interest despite receiving the civic engagement training (Group A) with those who have low political interest in the absence of the class (Group B). If the training program worked, these groups are not similar.

The basic takeaway is that it’s generally a big no-no to control for a consequence of the intervention in question when estimating the causal impact of that intervention.

Returning to the Guardian study, post-treatment bias occurs because the study controls for variables which are likely caused by the presence of books in the home:

We control for gender, age, parental education in years obtained by averaging maternal and paternal education and respondent's education in years (OECD, 2016) … We also control for respondents' occupational status in ISEI scores (Ganzeboom et al., 1992; Ganzeboom and Treiman, 2010).

Because respondent’s educational attainment and occupational status are both, to some extent, caused by the presence of books in the home during childhood, the inclusion of these variables as “controls” constitutes a glaring case of post-treatment bias.

Now, it would be one thing if the authors simply did not take into account the possibility of post-treatment bias, but what is truly mind-boggling is that the authors knew that these variables could, in fact, be caused by the presence of books in the home but chose to include them as controls in their main specification anyway.

To elaborate, Figure 3 presents a path diagram which relates each of the variables in the study to one another.

Note that this path diagram allows for the possibility that books in the home impact respondent’s educational attainment and occupational status - the very same control variables in the main specification. How the authors did not acknowledge this contradiction between their main specification and path diagram is beyond me, but it’s an indictment of the review process that led to this paper’s publication in my opinion.

Works Cited

Allington, R. L., McGill-Franzen, A., Camilli, G., Williams, L., Graff, J., Zeig, J., Zmach, C., & Nowak, R. (2010). Addressing Summer Reading Setback Among Economically Disadvantaged Elementary Students. Reading Psychology, 31(5), 411–427. https://doi.org/10.1080/02702711.2010.505165

Anger, S., Christoph, B., Galkiewicz, A., Margaryan, S., Peter, F., Sandner, M., & Siedler, T. (2024). A Library in the Palm of Your Hand? A Randomized Reading Intervention with Low-Income Children. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4970159

Borman, G. (2020). A School-Level Randomized Controlled Trial of the Impacts of the Reading Is Fundamental (RIF) Read for Success Program on Students’ Longitudinal Literacy Outcomes. Open Science Framework. https://osf.io/br7ck

Borman, G. (2022). A Multisite Cluster Randomized Controlled Trial of the Impacts of Bernie’s Book Bank on Elementary Students’ Reading Behaviors, Attitudes, and Achievement. Open Science Framework. https://osf.io/atxe9

Burt, S. A., O’Keefe, P., Johnson, W., Thaler, D., Leve, L. D., Natsuaki, M. N., Reiss, D., Shaw, D. S., Ganiban, J. M., & Neiderhiser, J. M. (2024). The Detection of Environmental Influences on Academic Achievement Appears to Depend on the Analytic Approach. Behavior Genetics, 54(3), 252–267. https://doi.org/10.1007/s10519-024-10179-w

Kim, J. S. (2006). Effects of a Voluntary Summer Reading Intervention on Reading Achievement: Results From a Randomized Field Trial. Educational Evaluation and Policy Analysis, 28(4), 335–355. https://doi.org/10.3102/01623737028004335

Kim, J. S. (2007). The effects of a voluntary summer reading intervention on reading activities and reading achievement. Journal of Educational Psychology, 99(3), 505–515. https://doi.org/10.1037/0022-0663.99.3.505

Kim, J. S., & White, T. G. (2008). Scaffolding Voluntary Summer Reading for Children in Grades 3 to 5: An Experimental Study. Scientific Studies of Reading, 12(1), 1–23. https://doi.org/10.1080/10888430701746849

Kim, J. S., & Guryan, J. (2010). The efficacy of a voluntary summer book reading intervention for low-income Latino children from language minority families. Journal of Educational Psychology, 102(1), 20–31. https://doi.org/10.1037/a0017270

Kim, J. S., Guryan, J., White, T. G., Quinn, D. M., Capotosto, L., & Kingston, H. C. (2016). Delayed Effects of a Low-Cost and Large-Scale Summer Reading Intervention on Elementary School Children’s Reading Comprehension. Journal of Research on Educational Effectiveness, 9(sup1), 1–22. https://doi.org/10.1080/19345747.2016.1164780

Lortie-Forgues, H., & Inglis, M. (2019). Rigorous Large-Scale Educational RCTs Are Often Uninformative: Should We Be Concerned? Educational Researcher, 48(3), 158–166. https://doi.org/10.3102/0013189X19832850

Melosh, G. M. (2003). Summer books! Stemming reading loss in high -poverty second graders through access to appropriate books during the summer vacation. [Doctoral dissertation, University of Florida].

Montgomery, J. M., Nyhan, B., & Torres, M. (2018). How Conditioning on Posttreatment Variables Can Ruin Your Experiment and What to Do about It. American Journal of Political Science, 62(3), 760–775. https://doi.org/10.1111/ajps.12357

Pagan, S. L. (2010). Children Reading for Pleasure: Investigating Predictors of Reading Achievement and the Efficacy of a Paired-Reading Intervention to Foster Children's Literacy Skills. [Doctoral dissertation, Carleton University]. https://repository.library.carleton.ca/concern/etds/41687j07c

Sikora, J., Evans, M. D. R., & Kelley, J. (2019). Scholarly culture: How books in adolescence enhance adult literacy, numeracy and technology skills in 31 societies. Social Science Research, 77, 1–15. https://doi.org/10.1016/j.ssresearch.2018.10.003

Smith, J. W. (1920). Children’s book week. [Photograph]. Picryl. https://picryl.com/media/childrens-book-week-november-15th-to-20th-1920-more-books-in-the-home-c27fcf

Stein, M. L. (2017). Supporting the Summer Reading of Urban Youth: An Evaluation of the Baltimore SummerREADS Program. Education and Urban Society, 49(1), 29–52. https://doi.org/10.1177/0013124516630595

Van Bergen, E., Van Zuijen, T., Bishop, D., & De Jong, P. F. (2017). Why Are Home Literacy Environment and Children’s Reading Skills Associated? What Parental Skills Reveal. Reading Research Quarterly, 52(2), 147–160. https://doi.org/10.1002/rrq.160

White, T. G., Kim, J. S., Kingston, H. C., & Foster, L. (2014). Replicating the Effects of a Teacher-Scaffolded Voluntary Summer Reading Program: The Role of Poverty. Reading Research Quarterly, 49(1), 5–30. https://doi.org/10.1002/rrq.62

Wilkins, C., Gersten, R., Decker, L. E., Grunden, L., Brasiel, S., Brunnert, K., & Jayanthi, M. (2012). Does a summer reading program based on Lexiles affect reading comprehension? https://ies.ed.gov/ncee/edlabs/regions/southwest/pdf/REL_20124006.pdf

Zhang, S., Inoue, T., & Georgiou, G. K. (2024). Is There a Genetic Confound in the Relation of Home Literacy Environment with Children’s Reading Skills? A Familial Control Method Approach. Reading Research Quarterly, 59(3), 408–423. https://doi.org/10.1002/rrq.553

1. I discuss the issue of collider bias in greater detail in the addendum.

2. Given that the causal diagram includes both genetics and upbringing, I assume that Pallesen believes this correlation is partly explained by environmental variables as well.

3. Measurement error is unlikely to undermine this result as the authors estimate a high test-retest reliability coefficient of the oral reading fluency assessment (r = 0.95). One could argue, however, that parental reading fluency is a bad control because the presence of books in the home may mediate the relationship between parental reading fluency and children’s reading fluency. The inclusion of the parental reading fluency control would then cause the books in the home association to be under-estimated.

4. The authors instead choose to frame this result in terms of the change in R^2 caused by adding books in the home as a predictor to the model. I’m personally not a fan of this approach as it depends on the order in which predictors are added (i.e. we don’t know what the change in R^2 would have been had we included books in the home as a predictor in the first step and then added parental reading fluency in the second).

5. The authors openly acknowledge this lack of representativeness in the paper:

The level of education of the FIOLA sample ranged from primary school only to post-master’s degree. The sample’s average was 0.49 SD above the national average (van Bergen et al., 2015). In sum, higher SES and skill level were somewhat overrepresented, but the sample included all socioeconomic strata and skill levels.

6. Publication bias is unlikely to explain this result.

7. Two large-scale pre-registered randomized experiments of book distribution programs are currently underway (Borman 2020; Borman 2022). Once published, the effect sizes of these interventions will be incorporated into this meta-analysis. Feeding the pre-specified traits of each study into the meta-regression model, the Bernie’s Book Distribution program is predicted to impact reading achievement between 0.09 and 0.24 SD and Books For Ownership is predicted to impact reading achievement between -0.28 and 0.24 SD.

I will be very curious to see whether these predictions bear out in the final data. To draw on my priors a bit, I anticipate that Books For Ownership prediction interval is likely downwardly biased relative to the true impact of the intervention. In other words, I expect this intervention to have an average predicted impact of ~0.05 SD, not -0.02 SD.