In an increasingly scientific and technological world, the need for a citizenry engaged in and appreciative of science, technology, engineering, and mathematics (STEM) has never been greater. Fueled largely by societal changes and new digital technologies and media, information about STEM has progressively become a part of the daily lives of most citizens, and the role of STEM in the workplace is at an all-time high and increasing every year. Historically, most of the attention related to public STEM literacy has focused on school-based efforts. However, it has become increasingly apparent to a number of policymakers, investigators and practitioners, that learning STEM neither starts nor stops at the schoolyard gates. We live at the dawn of the Knowledge Age, a time of relentless blurring of the boundaries of where, when, and how people learn STEM, as well as rapid changes in what aspects of STEM are worth knowing.

The STEM research and education communities in the U.S. have long pursued the goal of advancing the American public’s understanding of science. For over a generation, the vast majority of the rhetoric, resources, and research on this issue has focused on the perceived failure of U.S. school-aged children to excel at mathematics and science, particularly compared with children in other countries. Most policy solutions for this problem involve improving the practices of classroom teachers, particularly during the pre-college years, although there is increasing discussion about the importance of STEM education in both early childhood and the postsecondary years. The result has been an increasing focus on P–20 (pre-K through advanced degrees) solutions. That leaves the emphasis squarely on schooling. This “school-first paradigm” is so pervasive that few scientists, mathematicians, engineers, technologists, business leaders, educators, or policymakers question it, even in the absence of significant evidence to support the claim. In fact, there is a growing body of research suggesting that schooling, though important, is NOT necessarily the primary—and certainly not the only—significant mechanism by which the U.S. public learns STEM. Public STEM education in America is far deeper and broader than what happens in school. In fact, it can be seen as a comprehensive “Learning SySTEM.”

The True STEM Public Education System

Although school is indeed an important setting for the learning of STEM, it is only one of many such settings. The actual time devoted to STEM instruction in school represents a surprisingly small percentage of even school-aged children’s time, ranging, on average, from more than several hours to as little as 10 minutes per week for elementary-aged children. Pre-K and kindergarten-aged children may not receive any exposure to STEM. Outside of school, most Americans, children and adults alike, experience at least this much contact with STEM-related topics on a daily basis through the media and other discourse within the public sphere. Irrefutable evidence indicates that children and adults acquire significant STEM knowledge and develop and sustain their STEM interests as a consequence of informal experiences with STEM, such as those that take place in afterschool programs; during visits to community institutions such as science centers, zoos, and nature preserves; at home, through the use of the Internet and STEM-related books, magazines, and games; through hobbies and other special interest pursuits; and in the workplace. A recent study suggested that out-of-school educational sources collectively accounted for more of the public’s STEM literacy than schooling by a wide margin. While schooling is important for laying the foundation for STEM literacy among some members of the public, its importance may be far less than some suspect.

Data Sources: Large Scale Assessments and Individual Studies

Over the last 20 years, domestic and international tests of student performance in a variety of school subjects, including science, engineering and math, have provided a wealth of data and results that can be mined for understanding the factors that contribute to positive scores. In the U.S., the National Assessment of Educational Progress (NAEP) is administered to a representative sampling of youth throughout the country. For the past 25 years, NAEP reports have provided data on children and youth’s performance in mathematics and reading. Other subjects have been added gradually. In 2015 and 2016, NAEP reported the results of assessments in science (tested 2016), and in technology and engineering literacy (tested in 2014) in addition to the standard assessment in mathematics—all four STEM fields at the eighth-grade level. Although NAEP is designed to cover a range of topics considered relevant to broad STEM literacy, including abilities to interpret existing data rather than share factual knowledge, it tends to focus on topics assumed to be covered by the school curriculum. An exception is the 2014 NAEP Technology and Engineering Literacy (TEL) Assessment, which concerns topics that are not taught at all in half the schools sampled, but which the National Assessment Governing Board that sets policy for NAEP determined as important for everyone to learn in today’s increasingly technological world.

In addition to the basic test questions, NAEP includes so-called background questionnaires, which differ between tests and years but generally ask participating students a variety of questions about their interests and motivation, their activities outside of school, and their families’ educational status. Correlated against test performance, these background data provide, at a large scale, some indication of the types and scale of interactions that occur across settings. These results can and need to be supplemented by more focused, in-depth studies.

The Role of Family

All tests show that significant learning happens at home. For example, the 2014 TEL Assessment of eighth graders found that while 52% reported taking a course in technology or engineering in school, 63% reported that their family members taught them most of what they know about building things, fixing things, or learning how things work. In fact, nearly 90% reported that learning how to figure out why something was not working in order to fix it happened outside of school. These are not surprising findings given that year in and year out, despite repeated efforts to improve school performance, home experiences and background consistently emerge as the single greatest correlate of in-school performance. For example, 3-year-olds whose mothers nurtured their math skills through play, performed significantly better on preschool and first-grade math skills tests than those whose mothers did not engage in these activities.

The NAEP Mathematics assessment was given to 139,500 eighth graders in 2015. Correlations between performance and opportunities to learn outside of school were striking. As shown in Figure 1, 42% of the students who took the assessment discussed what they did in school with their families either every day or 2–3 times a week. These students did significantly better than their peers who did not talk with their families about schoolwork.

Figure 1: Average eighth-grade 2015 NAEP Mathematics performance scores by frequency of student-parent conversations about school work. The graph shows students’ responses to the question: “How often do you talk about things you have studied in school with someone in your family?”

In 2015, the NAEP Science assessment was given to 112,700 eighth graders. A similar percentage, about 43%, talked about their schoolwork with their families either every day or 2–3 times per week. The results were similar. Students who talked about what they did in school with their families also scored significantly higher than those who did not (Figure 2).

Figure 2: Average eighth grade 2015 NAEP Science performance scores by frequency of student-parent conversations about school work. The graph shows students’ responses to the question: “How often do you talk about things you have studied in school with someone in your family?”

Some of these differences correlate with socioeconomic variables like parental education and income. Children of parents with higher education and better financial means score considerably higher than those of parents with lower levels of education and income. For example, results from the 2015 NAEP Science test showed that fourth-grade children who were eligible for free or reduced lunch (an indication of low household income) scored on average 29 points lower (140 vs. 169 points on a 300 point scale) than all other children. This strong difference persisted at the 8th grade level (27 points, 140 vs. 167 points). In the U.S., socioeconomic status (SES) and education background are often related to race and ethnicity, but NAEP data also suggest that students’ performance on tests vary as a function of racial/ethnic background, independent of SES. For instance, the performance gap in 2015 science between 12th grade white and black children was 36 points (160 vs. 125). Again, this suggests that schools are not able to compensate for certain societal and home-related factors.

However, independent of race/ethnicity or even socioeconomic means, what is clear is that children of parents or caregivers who support their children’s learning at home achieve better, both in school and outside of school.

The Role of Outside Activities

There is also strong evidence that what happens outside the home influences STEM learning. Children and youth who engage in various kinds of science-related, free-choice activities outside of school and outside of the home are more motivated to pursue STEM-related activities in the future than are those who do not engage in such activities. The NAEP 8th grade Science assessment asked youth if they visited a museum, zoo, or aquarium to learn about science that was not on a school trip. About one third reported having done so. Their scores were significantly higher than the two thirds of students who had not (Figure 3).

Figure 3. 2015 Average eighth-grade NAEP Science performance results for the question: “In this school year, have you visited a museum, zoo, or aquarium to learn about science that was not on a school trip?”

These results are consistent with a range of emerging data from other sources suggesting the critical role that institutions like science centers, museums, zoos, and aquariums play in public understanding of science. In the most comprehensive study conducted to date, an international investigation of the contributions of science centers collected data from a random sample of approximately 12,000 adults and youth across 11 countries, including the U.S. Results showed that significant correlations existed between science center use and increased science understanding, interest, and identity. These results were significant independent of age, income, education, or prior interest.
Beyond visits to informal science education institutions, participation in activities beyond school are supportive of science learning. Another question on the NAEP Science assessment reveals a relationship between students’ science capabilities and what they do outside of school. Students who agreed or strongly agreed with the statement that they do science-related activities not required for schoolwork performed significantly better, on average, than those who said they do not (Figure 4).

Figure 4. 2015 NAEP Science performance results for the question: “Please indicate how much you disagree or agree with the following statement about science: I do science-related activities that are not for schoolwork.”

In technology and engineering, out-of-school activities emerge as particularly critical. The first NAEP Technology and Engineering Literacy (TEL) assessment was administered in 2014 to 21,500 eighth graders. This assessment asked four questions about the youths’ activities related to technology and engineering outside of school time. Their responses to all four questions turned out to be significantly correlated with performance on the assessment (Figures 5–8).

Figure 5. Average eighth-grade 2014 NAEP TEL performance results for the question: “Outside of school, how often have you ever used different tools, materials, or machines to see which are best for a given purpose?”

Figure 6. Average eighth-grade 2014 NAEP TEL performance results for the question: “Outside of school, how often have you ever built or tested a model to check a solution?”

Figure 7. Average eighth-grade 2014 NAEP TEL performance results for the question: “Outside of school, how often have you ever figured out why something is not working in order to fix it?”

Figure 8. Average eighth-grade 2014 NAEP TEL performance results for the question: “Outside of school, how often have you ever taken something apart to fix it or see how it works”

The magnitude of these differences should not be ignored. “Basic” performance on the TEL is defined as a score of 116 or above, while “proficient” is 158 or above. As indicated in Figures 6–9, youth who have had little experience with technology or engineering activities outside of school are, on average, at a basic level, while students who frequently engage in these activities perform, on average, at or near the “proficient” level.

Corroborating evidence for the value of science activities beyond the school day is available from the Program of International Student Assessment (PISA), which tests 540,000 students in 72 countries. The 2015 assessment focused on science, and revealed dramatic differences between the lowest (332) and highest scoring (556) countries. An accompanying background questionnaire asked students about their activities outside of the classroom. A recent summary report of PISA findings concluded that:

“Extracurricular activities, such as science clubs and competitions, help students understand scientific concepts, raise interest in science and even nurture future scientists. On average across OECD countries, students in schools that offer science competitions score 36 points higher in science and are 55% more likely to expect to work in a science-related occupation than students in schools that do not offer such activities; those in schools offering a science club score 21 score points higher and are 30% more likely to expect to pursue a career in science […] Across OECD countries, students who attend schools that offer science-related extracurricular activities, particularly science competitions, hold stronger epistemic beliefs, such as believing that scientific ideas sometimes change or that evidence comes from experiments” (OECD, 2016).

These findings focus at the school level. But the background questionnaire that accompanied the PISA test also revealed that a considerable percentage of 15-year olds engage relatively strongly in free-choice learning: 23% of 15-year olds regularly or very often watch science-related TV, 19% regularly or very often visit science-related websites, 16% read science articles in magazines or newspapers regularly or very often, and 15% regularly or very often follow science news on blogs or micro-blogs like Twitter. And those who engage in these kinds of science-related free-choice activities are more motivated to pursue STEM-related activities in the future.

Additional evidence of the important role that experiences beyond the classroom play in children and youth’s STEM learning comes from multiple sources. Both large-scale survey research and in-depth case studies of youth have shown that children and youth who engage in science and technology-related free-choice activities are both more STEM knowledgeable and more motivated to pursue STEM-related activities in the future.

Conclusion: Toward a Learning SySTEM

Despite this growing awareness that STEM education is lifelong and that hundreds of types of institutions and organizations contribute to the public’s STEM understanding, interest, and participation, government support continues to focus almost exclusively on the formal education system. Metrics like the NAEP results cited here are beginning to tell a different story about how the nation’s children and youth learn STEM—a story that describes a broader, more extensive ecosystem-level narrative that transcends schooling and other metrics that currently take a disproportionately school-first perspective.

As the data presented here show, STEM learning is a continuous, multidimensional phenomenon that extends across the entire breadth and depth of a learner’s experience. Although in this article we have only focused on a few settings in which children and youth learn, learning happens as a consequence of experiences within all the following settings:

• Preschool
• K–12
• Higher education
• Affinity groups
• Workplace
• Home
• Community institutions
• Afterschool
• Public sphere

Ultimately, it is important to understand what STEM learning looks like in the context of the interplay of these various settings. For example, how do experiences at home such as dinner-time conversations, or a family sitting together to watch a nature show on television, influence what happens in school? Or, how do these home experiences ultimately influence family behavior on the weekends? Are they inspired to visit a science center or go for a hike? How do youth development experiences in a 4-H or Boys and Girls Club or an afterschool or summer STEM program influence motivation to learn STEM subjects in school and pursue STEM careers? Does what an individual learns while engaged with a hobby or as part of a citizen science project (in which everyday citizens contribute data to scientific studies) build on content learned at school? How does family social and economic background influence the choices of experiences that children are afforded both in and outside of school? As the data presented here strongly suggest, these types of interactions are essential to children and youth’s STEM learning. But also apparent should be that current national metrics alone are insufficient to the task of providing sufficient depth and detail to these questions to enable wise STEM learning policy decision making.

It is time to rethink and reimagine where, when, and how the public learns STEM. The data presented above indicate the importance of taking a comprehensive, system-wide approach that places equal value on the learning that happens both in and out of school and that acknowledges the ways that experiences beyond the classroom dramatically influence lifelong STEM learning. In the 21st century, America should actively strive to support and connect STEM learning across the day and over a lifetime, and across varied settings and encounters. We need to invest in creating a network of public STEM education experiences that seamlessly incorporate learning opportunities in and out of school for all children, and which are framed increasingly around STEM lessons that support each individual learner’s desire to answer the questions that are important to his or her life. In the future, we must work toward re-envisioning STEM learning so as to fully accommodate and interconnect all available times, spaces, and ways to learn STEM. We need to adopt a Learning SySTEM perspective in policy, practice, and assessment.

References

1. STEM Education Coalition. (2016). The case for investing in out-of-school learning as a core strategy in improving science, technology, engineering, and mathematics (STEM) education. Retrieved from http://www.stemedcoalition.org/2016/04/27/the-case-for-investing-in-out-of-school-learning-as-a-core-strategy-in-improving-stem-education/
2. National Research Council (NRC). (2015). Identifying and supporting productive STEM programs in out-of-school settings. Washington, DC: National Academies Press.
3. Traphagen, K., & Traill, S. (2014). How cross-sector collaborations are advancing STEM learning (Working paper). Palo Alto, CA: Noyce Foundation
4. Carnegie Corporation of New York. (2009). The opportunity equation: Transforming mathematics and science education for citizenship and the global economy. New York, NY: Author.
5. NRC. (2011). A framework for K–12 science education: Practices, crosscutting concepts, and core ideas. Washington, DC: National Academies Press.
6. NRC. (2016). Science literacy: Concepts, Contexts, and Consequences. Washington, DC: National Academy Press.
7. Storksdieck, M. (2015). Critical information literacy as core skill for lifelong STEM learning in the 21st Century: Reflections on the desirability and feasibility for widespread science media education. Cultural Studies of Science Education, 11(1), 167–182. doi:10.1007/s11422-015-9714-4. <>
8. Bulunuz, M., & Jarrett, O. S. (2010). Developing an interest in science: Background experiences of preservice elementary teachers. International Journal of Environmental and Science Education, 5(1), 65–84.
9. Trumper, R. (2006). Factors affecting junior high school students’ interest in physics. Journal of Science Education and Technology, 15(1), 47–58. doi:10.1007/s10956-006-0355-6.
10. Trygstad, P. J. (2013). 2012 National Survey of Science and Mathematics Education: Status of elementary school science. Chapel Hill, NC: Horizon Research.
    e.g., Oregon’s commitment to P–20 education: Oregon Department of Education. (n.d.). State board philosophy and goals. Retrieved from http://www.ode.state.or.us/search/page/?id=181
11. Falk, J. H., & Dierking, L. D. (2010). The 95% solution: School is not where most Americans learn most of their science. American Scientist, 98, 486–493.
12. Falk, J. H., & Needham, M. D. (2013). Factors contributing to adult knowledge of science and technology. Journal of Research in Science Teaching, 50(4), 431-452. doi:10.1002/tea.21080.
13. Traphagen, K., & Traill, S. (2014). How cross-sector collaborations are advancing STEM learning. Palo Alto, CA: Noyce Foundation.
14. Falk, J. H., & Dierking, L. D. (2010). The 95% solution: School is not where most Americans learn most of their science. American Scientist, 98, 486–493.
15. Dudo, A., Brossard, D., Shanahan, J., Scheufele, D. A., Morgan, M., & Signorielli, N. (2011). Science on television in the 21st century: Recent trends in portrayals and their contributions to public attitudes toward science. Communication Research, 38(6), 754–777.
16. National Science Board. (2015). Science and engineering indicators: 2014. Washington, DC: Government Printing Office.
17. Azevedo, F. S. (2011). Lines of practice: A practice-centered theory of interest relationships. Cognition and Instruction, 29(2), 147–184.
18. Department for Children, Schools, and Families. (2008). The impact of parental involvement on children’s education. Nottingham, UK: Author.
19. Dudo, A., Brossard, D., Shanahan, J., Scheufele, D. A., Morgan, M., & Signorielli, N. (2011). Science on television in the 21st century: Recent trends in portrayals and their contributions to public attitudes toward science. Communication Research, 38(6), 754–777.
20. Falk, J. H., Dierking, L. D., Staus, N., Penuel, W., Wyld, J., & Bailey, D. (in press). Understanding youth STEM interest pathways within a single community: The Synergies Project. International Journal of Science Education, Part B.
21. Falk, J. H., & Needham, M. D. (2011). Measuring the impact of a science center on its community. Journal of Research in Science Teaching, 48(1), 1–12. doi:10.1002/tea.20394.
22. Fehrmann, P. G., Keith, T. Z., & Reimers, T. M. (1987). Home influence on school learning: Direct and indirect effects of parental involvement on high school grades. Journal of Educational Research, 80(6), 330–337.
23. Happer, C., & Philo, G. (2013). The role of the media in the construction of public belief and social change. Journal of Social and Political Psychology, 1(1). doi:10.5964/jspp.v1i1.96
24. Horrigan, J. B. (2006). The Internet as a resource for news and information about science. Washington, DC. Pew Internet & American Life Project.
25. Ito, M., Baumer, S., Bittanti, M., Boyd, D., Cody, R., Herr-Stephenson, B., . . . Tripp, L. (2010). Hanging out, messing around, and geeking out: Kids living and learning with new media. Cambridge, MA: MIT Press.
26. Lewenstein, B. V. (2009). Science books since 1945. In D. D. Hall (General Ed.), A history of the book in America: Vol. 5. The enduring book: Publishing in post-war America. Chapel Hill, NC: University of North Carolina Press.
27. Maltese, A. V., & Tai, R H. (2010). Eyeballs in the fridge: Sources of early interest in science. International Journal of Science Education, 32(5), 669–685.
28. McCreedy, D., & Dierking, L. D. (2013). Cascading influences: Long-term impacts of informal STEM experiences for girls. Philadelphia, PA: Franklin Institute.
29. NRC. (2009). Learning science in informal environments: People, places, and pursuits. Washington, DC: National Academies Press. doi:10.17226/12190.
30. National Science Board. (2015). Science and engineering indicators: 2014. Washington, DC: Government Printing Office.
31. Takahashi, B., & Tandoc, E. C. Jr. (2015). Media sources, credibility, and perceptions of science: Learning about how people learn about science. Public Understanding of Science, 25(6), 674–690.
32. The After-School Corporation. (2014). After-school and beyond: A 15-year history of TASC. Retrieved from http://expandedschools.org/sites/default/files/TASC_LegacyReport_for_web.pdf
33. Falk, J. H., & Needham, M. D. (2013). Factors contributing to adult knowledge of science and technology. Journal of Research in Science Teaching, 50(4), 431–452.
    https://nces.ed.gov/nationsreportcard/ and https://www.nationsreportcard.gov/tel_2014/ Retrieved December 15, 2016.
34. cf., Falk, J. H., Osborne, J., & Dorph, R. (2014). Supporting the implementation of NGSS through research: Informal science education (White paper). Retrieved from https://www.narst.org/NGSSpapers/Informal_Science_Education_June2014.pdf
35. El Nokali, N. E., Bachman, H. J., & Votruba-Drzal, E. (2010). Parent involvement and children’s academic and social development in elementary school. Child Development, 81(3), 988–1005. doi:10.1111/j.1467-8624.2010.01447.x
36. Casey, B. M., Lombardi, C. M., Thomson, D., Nguyen, H. N., Paz, M., Theriault, C. A., & Dearing, E. (in press). Maternal support of children’s early numerical concept learning predicts preschool and first-grade math achievement. Child Development. doi:10.1111/cdev.12676.
37. U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP). (2016). 2015 mathematics assessment. Retrieved from https://nces.ed.gov/nationsreportcard/
38. Bonci, A., Mottram, E., McCoy, E., & Cole, J. (2011). A research review: the importance of families and the home environment. Retrieved from http://www.literacytrust.org.uk/assets/0000/7901/Research_review-importance_of_families_and_home.pdf
39. Dubos, E. F., Boxer, P., & Huesmann, L. R. (2009). Long-term effects of parental involvement on children’s educational and occupational success: Mediation by family interactions, child aggression, and teenage aspirations. Merrill Palmer Quarterly, 55(3), 224–249.
40. NRC. (2009). Learning science in informal environments: People, places, and pursuits. Washington, DC: National Academies Press.
41. NRC. (2015). Identifying and supporting productive STEM programs in out-of-school settings. Washington, DC: National Academies Press.
42. Tai, R. H., Liu, C. Q., Maltese, A. V., & Fan, X. (2006). Planning early for careers in science. Science, 312(5777), 1143–1144.
43. Falk, J. H., Dierking, L. D., Swanger, L. P., Staus, N., Back, M., Barriault, C., . . . Verheyden, P. (2016). Correlating science center use with adult science literacy: An international, cross-institutional study. Science Education, 100(5), 849–876. doi:10.1002/sce.21225
44. OECD (2016). PISA 2015 Results. Country Specific Overviews. US (Massachusetts), p. 5. Retrieved from: http://www.oecd.org/pisa/PISA-2015-United-States-MA.pdf
45. OECD (2016). PISA 2015 Database. Retrived from: http://www.oecd.org/pisa/data/2015database/
46. Barron, B. (2006). Interest and self-sustained learning as catalysts of development: A learning ecology perspective. Human Development, 49, 193–224. doi:10.1159/000094368
47. Falk, J. H., Dierking, L. D., Staus, N., Penuel, W., Wyld, J., & Bailey, D. (2016). Understanding youth STEM interest pathways within a single community: The Synergies Project. International Journal of Science Education, Part B, 6(4), 369–384.
48. Ito, M., Baumer, S., Bittanti, M., Boyd, D., Cody, R., Herr-Stephenson, B., . . . Tripp, L. (2013). Hanging out, messing around, and geeking out: Kids living and learning with new media. Cambridge, MA: MIT Press.
49. NRC. (2015). Identifying and supporting productive STEM programs in out-of-school settings. Washington, DC: National Academies Press.

John H. Falk, Lynn D. Dierking, Jeremy Babendure, Scott Pattison, David Meyer, Monae Verbeke, Michael Coe, Sasha Palmquist, Institute for Learning Innovation

Martin Storksdieck, Nicolette Canzoneri, Oregon State University