Vol. 1, No. 4, December 1995
NCES 96-810 Ordering Information
The National Assessment of Educational Progress (NAEP) is the only nationwide assessment currently used that is designed to monitor students' performance in specific subjects such as reading, writing, mathematics, science, U.S. history, and geography. In early 1994, about 19,000 students in grades 4, 8, and 12 across the nation participated in the NAEP assessment in geography. This was the first comprehensive geography assessment conducted by NAEP and will provide baseline data to measure future progress. To provide a context for understanding the assessment, the Focus on NAEP series presents an overview of the framework on which the geography assessment was based. Sample questions from the 1994 assessment are also presented. The NAEP geography scores are reported on a scale that ranges from 0 to 500. Results are also reported according to geography achievement levels-basic, proficient, and advanced.
The framework that guides the assessment was developed by the National Assessment Governing Board through a year-long national consensus process involving teachers, curriculum coordinators, leading geographic educators, academic geographers, assessment experts, and the general public./1
The 1994 NAEP geography framework requires students not only to demonstrate factual knowledge but also to think critically about geographic issues, and actively apply the knowledge and skills of geography to complex problems of our society. Applying knowledge and skills to new and challenging tasks is important as the world becomes more connected through technological advancement and shared concerns about political, economic, social, and environmental issues.
The NAEP geography assessment includes
The 1994 geography assessment framework defines two dimensions for the assessment. These are summarized in table 1.
------------------------------------------------------------------------------------------------------------- Table 1.-Dimensions of the 1994 NAEP geography assessment ------------------------------------------------------------------------------------------------------------- Cognitive Content Dimension Dimension Space and Place Environment and Society Spatial Dynamics and Connections Knowing Where is the world's What mineral resources are What factors stimulate human largest rain forest? often extracted by strip migrations? mining? Understanding Why are tropical rain Explain the effects of strip Explain the motivations of modern day forests located near mining and shaft mining on the Mexican and Cubans for immigrating to the equator? landscape. the United States? Applying* Support the conclusion How can both economic and Compare current settlement and that tropical rain environment interests be employment patterns of Cuban and forests promote wide reconciled in an area of strip Mexican immigrants in the United species variation. mining? States. ------------------------------------------------------------------------------------------------------------- Applying=a range of higher order thinking skills Note: Example questions in the matrix cells are illustrative only and are not meant to represent the full array of assessment objectives. Source: Geography Assessment Framework for the 1994 National Assessment of Educational Progress (Washington, D.C.: National Assessment Governing Board, U.S. Department of Education, U.S. Government Printing Office). -------------------------------------------------------------------------------------------------------------
Content Dimension
Three broad areas make up the content dimension of the geography assessment. These content areas are represented in table 1 as Space and Place, Environment and Society, and Spatial Dynamics and Connections.
The study of Space and Place refers to knowledge and understanding of geography as it relates to particular places on Earth, to spatial patterns on the Earth's surface, and to physical and human processes that shape these patterns. Topics falling within this content area are decribed in table 2.
----------------------------------------------------------------------------------------------------------------- Table 2.-Topics within Space and Place ----------------------------------------------------------------------------------------------------------------- Fundamental place location Fundamental Physical geography Physical features and patterns of the physical Major spatial features and patterns in the natural environment such as major landforms, bodies of environment such as those relating to climate, oceans water, climate, and vegetation regions soils, landforms, and vegetation Features and patterns of the human environment such Major processes, such as atmospheric circulation as urban centers, farming regions, and political weathering and erosion, ocean currents, plate divisions tectonics, and vulcanism that shape patterns in the natural environment Fundamental geographic concepts and methods Concepts such as absolute and relative location, Fundamental human geography proximity, separation, direction, region, hierarchy, Major spatial features and patterns in the cultural density, dispersion, and methods that are used to environment such as language, religion, agriculture, describe and analyze spatial patterns and economic, political, and demograophic regions Basic spatial units of measurement such as distance Major processes, such as settlement, migration, trade, and area technological development, diffusion, and landscape transformation that shape cultural patterns Absolute location systems such as latitude-longitude and alpha-numeric grids, and relative location terms -----------------------------------------------------------------------------------------------------------------
Sample Question A (Space and Place)
Grade 4. On the map below, write the names of the North Pole, the South Pole, and the Equator in the correct location. (The actual map is large enough to write on).
Scoring Guide
Scoring rationale: Student locates and correctly labels the North Pole, South Pole, and the Equator on a map showing the Western Hemisphere.
3-Complete. The response correctly labels all three features on the map. (54.2 percent correct; standard error 1.5)
2-Partial. The response correctly identifies the location of one or two features on the map. (26 percent correct; standard error 1.2)
1-Inappropriate. The response does not correctly locate any of the three features. (7.2 percent correct; standard error 0.7)
Sample Question B (Space and place)
Grade 12. Under which of the following circumstances would you be most likely to find snow in equatorial regions?
A. In areas below sea level
B. In areas at high latitudes
*C. In areas at high elevations
D. In winter
*Key: C. (67.5 percent correct; standard error 1.3)
Environment and Society focuses on relationships between the natural world and the people who inhabit it. Through knowledge of environment-society interactions, geography helps students learn how people depend upon, adapt to, are affected by, and modify the natural environment. Many modifications, such as planting trees to reduce erosion from winds, may have positive consequences. Other modifications, such as locating a landfill over a ground water source, may have negative consequences. The continually developing force of technology requires that society give even greater attention to the results and potential outcomes of environment-society interactions. Topics falling within this content area are described in table 3.
--------------------------------------------------------------------------------------------------------------- Table 3.-Topics within Environment and Society --------------------------------------------------------------------------------------------------------------- Unity Positive and negative consequences of human Interaction among Earth's natural systems such changes to the environment such as over-grazing as the hydrosphere, lithosphere, biosphere, and and plowing arid land that may temporily increase atmosphere food production but, over time, contribute to desertification Interactions among Earth's human systems such as urban, agricultural, political, economic Human systems affected by the characteristics of and transportation sytems natural systems such as weather, plate tectonics, and vulcanism Interactions among natural and human systems such as a forest and a recreation area Changes in one system that lead to changes Implications of technology within the system and in other systems such as Uses of technology that result in changes to the impact of El Nino on commercial fishing or environment both intended and unintended. the effects of drought on forest fires Transporting oil or chemicals by ship or rail for example, can harm the environment if an accident Changes in a system that impact locally occurs regionally, and/or globally such as the erruption of Mount Pinatubo in the Philippines and its Positive and negative consequences of the uses of effect on the world's climate technology on the environment and society, such as automobiles enhancing human mobility but car Limits exhaust decreasing air quality The environment's limits in absorbing the impact of human activity such as the impact of the imported Mediterranean fruit flies on Perspectives California's produce production or Peoples differing perceptions of the same over-hunting on the elephant population and environment based on their experiences and interests tourism industry of Kenya (For example, a developer and an environmentalist may view the use of forest land at the edge of a city Human adaptations to, or modifications of the very differently. A farmer and an urban dweller may environment influenced by the characteristics perceive the construction of a chemical fertilizer of specific environments such as weather and plant outside of town very differently.) climate, landscape features, and natural resources People's changing perceptions of environmental modification over time, such as perceptions of industrial smokestacks of the 1920s versus the 1990s or the use of forest resources in the 19th and 20th centuries --------------------------------------------------------------------------------------------------------------
Sample Question C
Grade 8. Nuclear energy can be developed for peaceful purposes.
What is one argument in favor of developing nuclear energy for peaceful purposes?
What is one argument against developing nuclear energy for peaceful purposes?
Scoring Guide
Scoring rationale: Student demonstrates knowledge of an argument for and against developing nuclear energy.
3-Complete. The response presents one significant argument in favor of nuclear power and one significant argument against nuclear power for peaceful purposes. Correct answers may be drawn from the list given or include some other appropriate response.
2-Partial. The response presents a significant argument either for or against nuclear power. Other argument, if present, is insignificant or incorrect.
1-Inappropriate. The response fails to present a significant argument for either side of the issue. It may provide personal opinions or inaccurate assertions. Examples are that nuclear reactors are ugly, or that they are completely safe.
(47.3 percent correct; standard error 1.4)
Sample Question D
Grade 4. Most air and water pollution is caused by
A. ocean currents
* B. people
C. earthquakes
D. animals
*Key: B. (78.3 percent correct; standard error 1.3)
Spatial Dynamics and Connections
Geography's spatial perspective helps students understand the dynamics of connections among people, places, and regions. Connections are made when people travel from place to place, when ideas and beliefs such as capitalism or Islam spread across the world, or when products such as petroleum and automobiles move from producing to consuming areas.
Connections among people and places are influenced by a wide variety of factors including trade relationships, political tensions and changes, human migration, and technical change. Their effects may be positive in providing expanded opportunities and progress. They also may be negative, for example, in the case of disease or military conflict. Or they may combine both positive and negative consequences in the sense of advantages for some and hardships for others. (Topics falling within this content area are described in table 4.)
---------------------------------------------------------------------------------------------------------------------- Table 4.-Topics within Spatial Dynamics and Connections ---------------------------------------------------------------------------------------------------------------------- Spatial dynamics Geographic factors that contribute to conflict and The effects of a variety of factors on the cooperation in social, political, and economic organization and identity of regions such as a settings on a variety of scales, such as neighborhood, a metropolitan area, or the neighborhood youth and their perception of a local American Midwest park as their territory or the varying national claims on seabed resources The effects of factors such as proximity and distance on relationshipds between and among places Trans-regional organizations-alliances, cartels, and formal international organizations-that are The effects of concepts of specialization and formed international organizations-that are comparative advantage on the location of economic spatial characteristics such as the creation of the activities such as U.S. specialization in commercial Organization of Petroleum Exporting Countries airplane production. The United States dominates (OPEC) to influence the international price of in this field and therefore has a comparative petroleum advantage over other nations Movement The effects of diverse cultures onthe characteristics Natural and cultural phenomena that are spread by of places and regions such as ways in which diffusion throughout the world such as coffee, American grid cities differ from Islamic cities or cocaine, and capitalism ways in which various cultures construct housing Voluntary and involuntary human migration Connections patterns such as Russian Jews to Israel, or Concepts that are related to connections between Mexicans to the United States people, places, and regions such as systems and networks Unequal distribution of resources that generate trade, encourage interdependence and shape Changes in information systems, communications economic patterns such as U.S. export of lumber networks, and transportation technologies that to-and import of electronics from Japan increase connections such as in the building of supertankers or in the completion of the Trans-Alaska Pipeline Living conditions Standards of living that relate to regional economic District patterns of function in urban, suburban differences and relationships such as cities to farms and rural regions such as land use and service the Rust Belt to the Sun Belt, or the United States requirements to Canada and Mexico ----------------------------------------------------------------------------------------------------------------------
Sample Question E
Grade 12. Maddieville is building a new shopping center. There is a disagreement in the city council over whether to build the shopping center at site A or at site B on the following chart. As a resident of the city who would like to shop at the shopping center, write a letter to the mayor in support of either site A or site B. Give three reasons why the site you support is better than the other site.
Scoring Guide
Scoring rationale: Student demonstrates the ability to judge the placement of a new shopping center based on evidence given on a map.
4-Complete. The response identifies a site and supports the choice with three reasons. (14.9 percent correct; standard error 1.1)
3-Essential. The response identifies a site and supports the choice with two reasons. (39.8 percent correct; standard error 1.3)
2-Partial. The response identifies a site and supports the choice with one reason. (28.7 percent correct; standard error 1.4)
1-Inappropriate. The response may or may not identify a site, but it does not offer any support for either location. (9.8 percent correct; standard error 0.8)
Sample Question F
Grade 8. What is an important reason that skyscrapers were built in American cities?
A. Construction companies liked to build structures that made residents feel as if they lived in single-family houses.
B. Skyscrapers allowed tenants to create a closer community.
* C. Skyscrapers allowed people to use small amounts of land more efficiently.
D. Citizens wanted to be safe from street crime.
*Key: C. (79.5 percent correct; standard error 0.9)
Cognitive Dimension
Three cognitive areas displayed as horizontal rows in table 1 specify areas of thinking expected of students as they embrace specific geography content. These cognitive areas are defined as knowing, understanding, and applying. The cognitive dimension tests the student's ability to perform mental tasks in these areas and expects students in grades 4, 8, and 12 to be able to think geographically in all three ways as they work with the content that is appropriate to their grade level.
(What is it? Where is it?)
In the area of knowing, students should be able to perform two related functions with respect to information: an observation function, such as observing different elements of the landscape; and a recall function, such as the name of a place or a resource indigenous to a particular country (see sample questions A and F).
(Why is it there? How did it get there? What is its significance?)
Understanding refers to the ability to see connections between diverse bits of geographic information, and to use that information to explain existing patterns and processes on Earth. For example, students may understand the concept of differential heating and cooling of air over land and water well enough to explain what is occurring in the atmosphere to cause this phenomenon (see sample questions C and D).
(How can knowledge and understanding be used to solve geographic problems?)
Applying refers to a range of higher-order thinking skills. Students are genuinely competent in geography when they are able to apply the knowledge and understanding of the discipline to real-life situations, allowing them to make personal decisions and seek solutions to societal problems. Examples of contemporary issues are the spread of diseases like AIDS or the suitability of different waste disposal programs to the needs of a particular urban center. To function in the Applying mode, students must be able to classify, hypothesize, analyze, synthesize, and evaluate information (see sample questions B, C, and E).
The results of the assessment will be reported in the fall of 1995 in The NAEP 1994 Geography: A First Look report and The Geography Report Card. The results in these publications will be interpreted in the context of background variables such as television watching, number of geography courses taken, teacher qualifications, time spent teaching specified skills and topics, and computer availability.
1/Geography Assessment Framework for the 1994 National Assessment of Educational Progress (Washington, DC: National Assessment Governing Board, U.S. Department of Education, U.S. Government Printing Office).
2/Short constructed-response items usually require a phrase or a few sentences as an answer. Extended constructed-response tasks usually require in-depth answers. Both are scored according to specific criteria.
The Focus on NAEP series briefly summarizes information about the ongoing development and implementation of the National Assessment of Educational Progress. The series is a product of the National Center for Education Statistics (NCES), Gary W. Phillips, Associate Commissioner for Education Assessment. This issue was written by Sheida White from NCES and Christine O'Sullivan from Educational Testing Service.
For questions, and for more information about NAEP and to order NAEP products see the NAEP staff directory