Increased school accountability for all students 
The current school reform effort emphasizes the need to improve the education of all students. Assessing the academic achievement of every student is an essential part of this reform, but one that presents a challenge for most schools, school districts, and states (North Central Regional Educational Laboratory and Council of Chief State School Officers, 1996). Traditional assessment practices in many states and school districts have tended to exclude students who are learning English as a second language. As a consequence, many English language learners (ELLs) are denied access to important educational opportunities that are based on assessment results.
This digest focuses on one type of assessment system, assessment portfolios, and discusses the advantages and challenges of using an assessment portfolio system that includes ELLs as a district-wide assessment tool.
SYSTEMS FOR ENGLISH LANGUAGE LEARNERS?Inclusion of English language learners
Unlike other assessment programs, assessment portfolios do not exclude certain student populations. Many states have policies that restrict English language learners from taking commercially developed, norm-referenced and criterion-referenced tests for up to 3 years after their arrival in a U.S. school system. In contrast, all students can be included in an assessment portfolio system.
Increased school accountability for all students
At the state level, assessment information is often collected to ensure that the educational system addresses the needs of all students. Inclusion of ELLs in state and local testing programs is critical to accountability and to providing accurate data about the achievement of these students (O'Malley & Valdez-Pierce, 1996). By providing systems with a richer source of information about school learning, assessment portfolios can help school systems identify and meet the needs of diverse students, including ELLs.
A shared vision of
student goals and learning
By developing an assessment portfolio system that includes English language learners, teachers, administrators, parents, and students can shape a common vision of what students should know and be able to do as a result of their course work. By clearly articulating expectations and the criteria upon which to assess attainment of these expectations, school systems help create a shared vision of the purpose of education based on the values of the community.
Authentic picture of
learning
Assessment portfolios can be designed to measure virtually any observable skill or process or content-area knowledge needed for system-wide assessment purposes. A wide range of student products can be included in assessment portfolios as long as predetermined scoring criteria are in place. Portfolios are designed to be inclusive of all students and to provide an authentic description of what students can do.
Improved teaching and student
learning
Using assessment portfolios that include English language learners not only provides improved information about student achievement but also makes a positive impact on teaching and student learning. According to Geoff Hewitt, a writing assessment consultant, when teachers are trained to use and score portfolios based on agreed-upon criteria, they tend to move toward a more learner-centered teaching model, which encourages students to take more responsibility for their own learning (personal communication, October 11, 1996). Through such training, teachers develop an understanding of the quality of student work that meets specific achievement levels according to the scoring criteria.
Reflection
of assessment reform
Advocates of assessment reform call for new measures that provide a better understanding of student achievement, especially for English language learners. By using assessment portfolios that include ELLs, school systems could reduce the number of students excluded from system-wide assessment and possibly increase the number of teachers participating in professional development activities.
LANGUAGE LEARNERS?Lower comparability and reliability
Many performance-based tests, including some portfolio systems, do not easily or meaningfully translate into a single score or set of scores. The public has become accustomed to single scores, such as those used to describe the results of standardized or norm-referenced tests. Single scores are comparable across systems and from one year to the next. Because some districts report the outcomes of performance-based tests in words rather than numbers, some stakeholders feel the school system is less accountable for individual students.
In addition, it is difficult to implement assessment portfolios that meet the reliability requirements many school systems want. Achieving a certain degree of reliability among raters or test evaluators is important (Novack, Herman, & Gearhart, 1996). Without high inter-rater agreement figures, the usefulness of the scores as an accountability tool diminishes, because the results cannot be used to compare scores reliably between schools or over time.
Difficulty ensuring standardized testing conditions
Some states and districts are prevented by law from implementing performance-based assessments that include portfolios. Some state legislatures mandate the use of traditional norm-referenced tests because of the perception that standardized tests require students to perform under similar circumstances (Special Issues Analysis Center, 1995). When using portfolio assessments, performance conditions may vary, and teacher bias can affect students' performance. For example, the amount of support teachers provide to students, the amount of time students are allowed to spend on portfolio samples, and the extent to which student work is augmented by support from external sources have raised questions about the validity of inferences about student competence based on portfolio work (Gearhart, Herman, Baker, & Whittaker, 1993).
Cost
One hurdle to developing an assessment portfolio system is the ongoing expense of the program. Designing, implementing, and scoring portfolio items is labor intensive and therefore costly. Practitioners invest a substantial amount of time to align the assessment tasks with the curriculum and to develop the scoring criteria and scoring tools. Persuading state legislatures and school boards to agree to the additional costs involved in scoring an assessment portfolio system that includes ELLs can be difficult.
Scoring
Portfolios pose a difficult scoring dilemma for three reasons. First, developing and using scoring criteria requires labor intensive staff discussion and training. When ELLs are included in a portfolio system, the scoring must be designed to assess content knowledge and language proficiency. Second, inter-rater agreement among teachers and other scorers requires intensive staff development. Third, scoring student work using predetermined criteria is more time consuming than scoring a single-occasion, norm-referenced test. Fortunately, these same activities also lead to improved teaching and learning.
THAT INCLUDES ENGLISH LANGUAGE LEARNERS:"Decide about goals and content." Stakeholders in the school and school district, including those responsible for English language learners, decide what assessment information is needed and how that information can be provided. A group of teachers agree to lead the program.
"Design the portfolio assessment program." The lead group of teachers, including English as a second language and bilingual education teachers, administrators, and parents decide on the range of products to be included in the portfolio assessment program.
"Develop scoring criteria and standards of performance." The group decides on common goals for student learning and performance and how they will be assessed, develops scoring rubrics and checklists, and agrees on standards of performance to be attained. If possible, benchmarks that exemplify student work are articulated, including benchmarks for English language learners.
"Align tasks to standards and curriculum." The group aligns the assessment tasks to the district or state content standards and curriculum frameworks.
"Implement at pilot sites, provide staff development, and analyze results." Decide on pilot sites and provide staff development on the implementation of portfolios. Following the implementation at pilot sites for at least one full school year, score the portfolios from the pilot sites. Assess the effectiveness of the program and modify the scoring criteria based on feedback from the pilot site educators and results of the scored portfolios. Study the effects of the program on English language learners in particular to determine whether improved information is available as a result of the portfolio implementation.
"Implement at all sites." Once the program has been piloted and found to be effective, implement the portfolio program at all sites.
"Train teachers to score." Train a team of teachers to score student work using the portfolio program's scoring criteria and benchmarks. Training should include discussion of second language proficiency and its impact on student achievement. Efforts are made to reach an inter-rate reliability level of .7-.8.
"Establish guidelines for administration." Stakeholders develop guidelines for a standardized collection of student work and decide on the time, place, and manner in which standardized prompts will be given to assess students throughout the system. Accommodations for English language learners are delineated.
"Score the portfolios." Teachers score the portfolios based upon predetermined criteria. This is typically done over several days in a central location by teachers who have been trained.
"Report the results." All stakeholders receive information about the results of the portfolio assessment in a timely fashion in ways that make the results meaningful to everyone, including teachers, students, parents, and other community members.
"Evaluate the program." After one year, evaluate the effectiveness of the portfolio program and make necessary judgments.
North Central Regional Educational Laboratory and Council of Chief State School Officers. (1996). "The status of state student assessment programs in the United States." Oak Brook, IL: North Central Regional Educational Laboratory.
Novack, J.R., Herman, J.L., & Gearhart, M. (1996). "Issues in portfolio assessment: The score-ability of narrative collections" (CSE Tech. Rep. No. 410). Los Angeles, CA: National Center for Research on Evaluation, Standards, and Student Testing.
O'Malley, J.M., & Valdez Pierce, L. (1996). "Authentic assessment for English language learners: Practical approaches for teachers." New York: Addison-Wesley.
Special Issues Analysis Center. (1995). "Inclusion of limited English proficient students in state performance standards and assessment." Rosslyn, VA: Development Associates.
-----
This Digest is based on an article published by the Northeast and Islands Regional Educational Laboratory at Brown University in Perspectives on Policy and Practice (March 1999): "Creating Large-Scale Assessment Portfolios That Include English Language Learners" by Emily L. Gomez. The research for the article was completed in 1997.
This digest was prepared with funding from the U.S. Dept. of Education, Office of Educational Research and Improvement, National Library of Education, under contract no. ED-99-CO-0008. The opinions expressed do not necessarily reflect the positions or policies of ED, OERI, or NLE.
###
[Return to ERIC Digest Search Page]