[Return to ERIC Digest Search Page]
Increasingly, there has been an emphasis on evidence-based practice in education. The most recent incarnation of this focus is "scientifically based research" (SBR), a phrase often associated with the No Child Left Behind (NCLB) Act of 2001.
The NCLB Act, the most sweeping reform of the Elementary and Secondary Education Act (ESEA) since it was enacted in 1965, redefines the federal role in K-12 education. This legislation stipulates that federally funded programs and practices must be grounded in "scientifically based" research. School leaders who depend on federal funding are now required to be aware of the nature of the research that guides their programs and practices. In particular, federal funding decisions will be determined in part by whether programs and practices have a basis in "scientific research."
The U.S. Department of Education, among others, is developing guidelines to assist school leaders in assessing the scientific basis of programs and practices. Ultimately, the difficult decisions of selecting programs and practices will rely on the professional discretion and expertise of school leaders. This Digest describes the nature and implications of SBR.
The federal government's involvement in these efforts can be traced from the Cooperative Research Act of 1954 to the creation of the National Institute of Education in the early 1970s, which was later subsumed by the Office of Educational Research and Improvement (OERI). In the latest reorganization of the U.S. Department of Education, OERI was replaced by the Institute of Educational Sciences (IES), which established the What Works Clearinghouse. In time, after more research studies have been completed, it is hoped the What Works Clearinghouse will provide education leaders with scientific evidence about the efficacy of specific programs and practices and their effect on achievement, dropout rates, and other student outcomes.
The implications of SBR for education are the subject of heated debate among education researchers and academicians. Proponents of SBR see it as a necessary step in moving education to an evidence-based field. On the other hand, some scholars are concerned that SBR is an attempt to determine "what works" without considering all the implications, financial and otherwise.
While there is not universal agreement regarding all the nuances associated with SBR, defining characteristics have appeared across numerous sources (for example, the Coalition for Evidence-Based Policy 2002, Comprehensive School Reform Program Office 2002, No Child Left Behind Act 2002, National Research Council 2002, Raudenbush 2002). Incorporating defining characteristics that appear in these sources, scientifically based research can be defined as follows:
Persuasive research that empirically examines important questions using appropriate methods that ensure reproducible and applicable findings.
Persuasive. This attribute refers to research that is moving from "tentative knowledge claims generated at local research sites to become stabilized and transformed into widely accepted facts" (Smith and others 2002). Appropriate research design, methods, and techniques; logic and reasoning; and replicable results can all help to establish persuasiveness.
A critical element in persuasiveness is the peer-review process, in which researchers who have been trained in research methodology review and critique each other's work to help ensure that the methods used match the research questions and conclusions. Research findings published in a peer-reviewed journal can be assumed to have undergone careful scrutiny, been considered in light of alternative explanations, and deemed sufficiently "persuasive" by a panel of individuals with expertise in research methods.
Empirical. Research that is empirical is based on measurement or observation, that is, experienced "through the senses" (NRC 2002). For example, research that measures or observes the impact of school vouchers on student achievement would be considered empirical. However, there are certain questions that cannot be addressed by empirical investigations (NRC ), such as "Should school vouchers programs be enacted in my state?" Questions involving "should" are typically addressed through means other than observation and measurement.
Important Questions. This refers to questions addressed by research that build upon, add to, fill a void in, or otherwise clarify what is known and practiced. The NRC explains that the importance of a question is often determined by its relationship to prior research, theory, and relevance to policy and practice.
Appropriate Methods. This refers to the use of designs, methods, and techniques that fit the nature of the question the study is attempting to answer. However no research design, method, or analytic technique on its own makes a study or program of research scientific (NRC). If the question pertains to "Does it work?," then randomized experiments or quasi-experiments are most appropriate (Raudenbush 2002, Coalition for Evidence-Based Policy).
Simply stated, randomized experiments involve randomly assigning individuals, schools, or districts to a group that receives a particular intervention (such as class-size reduction) and to a group that does not. In contrast, if the question pertains to "What was the 'it'?," then qualitative methods (such as the case study) are most appropriate (Erickson and Gutierrez 2002). Among other things, qualitative methods provide "up-close descriptions" of what is, or is not, working; how interventions are working; and what might be facilitating or impeding the effectiveness of a particular intervention (Raudenbush).
Replicable and Applicable Findings. In general, this attribute refers to consistent, meaningful findings. The research presents sufficient detail to allow for "replication or, at a minimum, ... the opportunity to build systematically on their findings" (NCLB 2002).
Such findings are understandable, accessible, and applicable to a wide audience (Comprehensive School Reform Program Office). For example, a program of research should be designed and conducted to ensure that school leaders across the nation have a solid sense of whether they can expect to see similar results from implementing a school-reform program that has demonstrated increased student learning in another state.
Advocates assert that SBR will replace the "folk wisdom of education" with a scientific knowledge base (Whitehurst 2002). Scientific research is seen as a means for improving education and developing a knowledge-base for "what works." Whitehurst claims that "there is every reason to believe that, if we invest in the education sciences and develop mechanisms to encourage evidence-based practices, we will see progress and transformation... of the same order and magnitude as we have seen in medicine and agriculture."
Alternatively, some scholars fear focusing on "what works" may blind educators to important aspects and potentially harmful unanticipated outcomes of education processes (Erickson and Gutierrez 2002). To illustrate this point, Erickson and Gutierrez use an example from medicine that "worked" but had disastrous side effects-the use of thalidomide to treat morning sickness. St. Pierre (2002) expresses her concern that SBR represents too narrow a focus. She argues for keeping research in education open to diverse traditions of scholarship and inquiry so as not to obstruct "the proliferation of knowledge."
A comprehensive, or even adequate, database of "what works" is far from complete. In the meantime, general guidelines and recommendations are being generated by the U.S. Department of Education and other organizations to assist school leaders in selecting and implementing new programs and practices.
For example, the Comprehensive School Reform Program Office published a report in August 2002 entitled "Scientifically Based Research and the Comprehensive School Reform (CSR) Program." This report is intended to help school leaders develop an understanding of SBR and assess the scientific basis of programs and practices.
Another example is the ERIC Clearinghouse on Educational Management, which is developing guidelines for both authors and users of its products that will help clarify the nature of evidence cited. Such efforts are intended to assist readers in making informed decisions and drawing reasonable conclusions about the findings.
School leaders play a significant role in determining the implications of SBR for education. Keeping abreast of issues and becoming part of the conversation of how SBR is implemented in policy and practice will help shape the role of SBR in education. What remains unchanged is that school leaders still must rely on their professional judgment and the best information available in making decisions about the selection, implementation, and management of programs and practices in their schools.
Comprehensive School Reform Program Office, Office of Elementary and Secondary Education, U.S. Department of Education. Scientifically Based Research and the Comprehensive School Reform (CSR) Program. Washington, D.C.: Author, 2002. Available online at http://www.ed.gov/offices/OESE/compreform/appendc.pdf
Erickson, F., and K. Gutierrez. "Culture, Rigor, and Science in Educational Research." Educational Researcher 31, 8 (2002): 21-24.
Lagemann, E. An Elusive Science: The Troubling History of Educational Research. Chicago: University of Chicago Press, 2000. 302 pages. ED 457 075.
National Research Council. Scientific Research in Education, edited by R. J. Shavelson and L. Towne. Committee on Scientific Principles for Educational Research. Washington, D.C.: National Academy Press, 2002.
No Child Left Behind Act of 2001, Pub. I, No. 107-110, 115 Stat. 1425 (2002).
Raudenbush, S. "Scientifically-Based Research." U.S. Department of Education Seminar on Scientifically-Based Research. Washington, D.C.: U.S. Department of Education, 2002. Available online at http://www.ed.gov/offices/OESE/esea/research/
Smith, L. D.; L. A. Best; A. Stubbs; A. B. Archibald; and R. Roberson-Nay. "Constructing Knowledge." American Psychologist 57, 10 (2002): 749-61.
St. Pierre, E. A. " 'Science' Rejects Postmodernism." Educational Researcher 31, 8 (2002): 25-27.
Whitehurst, G. J. "Statement of Grover J. Whitehurst, Assistant Secretary for Research and Improvement, before the Senate Committee on Health, Education, Labor and Pensions." Washington, D.C.: U.S. Department of Education, 2002.Available online at http://www.ed.gov/offices/IES/speeches
-----
A Product of the ERIC Clearinghouse on Educational Management, College of Education, University of Oregon, Eugene, Oregon 97403-5207.
This publication was prepared with funding from the Institute of Education Sciences, U.S. Department of Education, under contract No. ED-99-C0-0011. The ideas and opinions expressed in this Digest do not necessarily reflect the positions or policies of IES, ED, or the Clearinghouse. This Digest is in the public domain and may be freely reproduced. The text of this Digest may be viewed electronically at http://eric.uoregon.edu EA032476.
###