There can be no doubt that technology has had a major impact on our daily lives. We read the news online, we shop online, we communicate with each other online, we are entertained on line, and we learn online. Education has been a prime beneficiary of technology's power. More than 95 percent of all public schools are equipped with computers, with a majority having Internet access (Market Data Retrieval, 1999). Distance learning is becoming more and more commonplace, not only in laboratories with computers connected to the Internet, but also on our desktops at home and work. We can find information on nearly everything at the stroke of the keyboard and listen and see events of historical proportion that take place thousands of miles from our home.
Educators have long understood the potential of technology in supplementing the instructional process, including the use of technology in the area of assessment. Technology-delivered assessment is being implemented in a variety of settings and grade levels for a variety of uses in education, business, government, and the private sector (Sistek-Chandler, 2000). Under the right conditions and with proper use, a marriage of technology and assessment can shine like a diamond. Used without proper care or forethought, however, it can be a rock.
* accessibility: Individuals can take various tests by computer or on-line for many purposes including college entrance, course placement, certifications and licensure, career decision-making, academic preparation, military selection and classification, and personality assessment. As a result, individuals can have access to information that they may not have had before. Bill Gates, in his book The Road Ahead (1995), suggested that individuals who are "wired" have access to the same information, resulting in a virtual equity more achievable than real-world equity.
* immediate feedback: The potential for immediate test scoring and feedback is a key advantage of technology-delivered assessment which can be a significant motivator for persons taking assessment instruments. Individuals can learn their status on assessments quickly and use that information to make decision or take immediate action.
* more efficient testing: The use of computer-adaptive testing, as opposed to computer-administered testing, allows persons to take tests that are targeted more accurately to their ability levels (Heubert & Hauser, 1999). Use of technology in combination with the increasingly popular item response theory can determine an individual's performance level using fewer questions. Under certain conditions, significant time and money can be saved by using computer-adaptive testing, particularly in large-scale assessment situations.
*judging portfolio information: The capability of placing ones work, experiences and/or educational history on a floppy disk or on a website and making that information available for review and judgment is empowered by technology. Writing samples, art work, letters of recommendation, journals, test results, transcripts, certificates and certifications, descriptions of community service, listing of club memberships and activities, and special project work can be saved electronically, transported easily (either physically or electronically), and made readily available for evaluation by others.
* ability to assess higher order skills: Use of technology permits test developers to use techniques and create situations that are difficult or impractical to construct in regular paper/pencil assessments. Consequently, and with some creativity through the use multi-media, interactivity, simulations, etc., assessments can reflect more authentic conditions. Crafted carefully, the assessments may provide the capability of tapping into higher order cognitive skills than cannot be easily tested with a standard paper/pencil instrument (Chung & Baker, 1997). Test developers can construct situations that simulate the real world. For instance, test items on computer can be designed to simulate events in biology, economics, architectural design, structural engineering, etc., asking students to exhibit skills under real or more realistic conditions. Language assessments can use audio and video to simulate various situations that could be encountered in a foreign country.
* helping persons with disabilities: Use of assistive technology in assessment can be significant in helping persons with disabilities. Text readers can help persons with visual impairment gain access to testing situations. Physically challenged persons can take advantage of voice recognition technology to answer test items and even to dictate long responses to essay questions. Those with difficulty controlling fine motor skills can use a touch screen or smart board to respond to assessment items. Persons who are housebound or not able to travel to a test site can take a test over the Internet from their home. Technologies are being created that can respond to slight movements of the head or eyes. Not only does technology provide testing accommodations, but access to testing opportunities that were not available before.
* lack of accessibility: The Department of Commerce has shown in its most recent study that access to computers and the Internet is highly dependent on income, racial and ethnic group, and urbanicity (U.S. Department of Commerce, 1999). Other studies indicate that access varies greatly by income, age, education, and technology optimism. (Forrester Research, 2000). In addition to basic computer access, persons may not be able to afford the cost of the tests themselves.
* test security problems: Not unlike paper/pencil instruments, those available in electronic formats are vulnerable to compromise. Compromised tests, can result in unfair advantage to test takers who might obtain the questions prior to taking an exam or be able to find answers to questions on the Internet. Solutions range from securing removable hard drives to tracking usage, and blocking access to certain Internet addresses.
* concerns about test taker identity: Users of test results need to be sure that the person taking the assessment has accurately represented his or her identity. Special care must be taken to ensure that the person completing a licensure test for credentialing, for example, is the person that is seeking this certification. Solutions can range from desktop video teleconferencing to fingerprint and retinal recognition systems.
issues of privacy/confidentiality: As with paper/pencil assessments, information about an individual's technology-delivered scores and results should be kept confidential and be made accessible only to those individuals who have a rightful need to know. This is particularly critical for sensitive assessments that are answered over the Internet
* lack of information on the quality of the instrument: It can be mistakenly assumed that any test made available on computer or via the Internet meets professional testing standards. This can be a risky assumption. It is quite possible for the instrument to be deficient in the requisite technical information necessary to show the degree of quality and suitability of the test. In fact, many assessments have been posted for easy access via the Internet where little or no technical information has been made available to the consumer.
* problems with test comparability: If an instrument is available in both paper/pencil and computer-delivered format, it can be mistakenly assumed that test developers have produced parallel forms that provide the same scores regardless of administration format. It is not unusual for some high-quality tests, offered in the paper/pencil format, to result in different scores when those same items are administered via computer or Internet. Without assurance of comparability, a test taker may have an advantage by using one format rather than the other. Disparate results from noncomparable assessments can transpire for any number of reasons including speededness, point size of the words, monitor resolution, use of color, physical arrangement of the items, inability to change completed items, comfort with the equipment, and response mode.
* gender, racial, and ethnic disparities: That females, persons of color, or individuals of different ethnic backgrounds may be disadvantaged in certain testing situations has been a long-standing concern in paper/pencil testing. This can be exacerbated with tests delivered via computer or the Internet. If a particular group has disproportionate access to computers and technology, disparity could be created by the medium alone.
* reporting and interpretation: Immediate feedback on tests is clearly desirable. Without appropriate interpretation, however, there is danger that the test taker will take actions that are not warranted by the test results. The potential exists for interpretations to appear to be so definitive that test takers fail to understand the need for caution due to the lack of accuracy of the scores. Also, feedback can be so extensive that the test results can paralyze a person's actions.
* lack of human contact: With technology-delivered assessments, meaningful human contact and intervention to assist with test score interpretation may be lacking or unavailable. A skilled counselor can help a test taker to sort out his or her results and use them in the context of other knowledge and experiences.
*issues of familiarity with technology: There are some studies that show that students perform better on tests that use the same medium in which they were taught. For example, if students use the computer to write essays and papers, they tend to score higher on writing assessments that allow them to use that technology to take writing assessments and more poorly if that technology is not available for testing purposes. Therefore, students who learn on computers may have an advantage on computerized assessments.
* American Educational Research Association, American
Psychological Association, and National Council on Measurements in
Education. (1999). Standards for Educational and Psychological
Testing. Washington, DC: American Educational Research
Association.
* American Association for Counseling and
Development. (1989). Responsibilities of Users of Standardized
Tests. Alexandria, VA: Author.
* American School Counselors
Association and Association for Assessment in Counseling. (1998).
Competencies in Assessment and Evaluation for School Counselors.
Alexandria, VA: Author.
* Committee on Professional Standards and
Committee on Psychological Tests and Assessment. (1985). Guidelines
for Computer-Based Tests and Interpretations. Washington, DC:
American Psychological Association.
* Dahir, C. A., Shelton, C.
B., and Valiga, M. J. (1998). Vision Into Action: Implementing the
National Standards for School Counseling Programs. Alexandria, VA:
American School Counselor Association.
* Joint Committee on
Testing Practices. (1988). Code of Fair Testing Practices in
Education. Washington DC: National Council on Measurement in
Education. This document is being reviewed and should be revised for
publication in 2001.
* Joint Committee on Testing
Practices. (2000). Rights and Responsibilities of Test Takers:
Guidelines and Expectations.
* National Board for Certified
Counselors. (1998). Standards for the Ethical Practice of Web
Counseling. Greensboro, NC: Author.
* National Career Development
Association. (1997). NCDA Guidelines for the Use of the Internet for
the Provision of Career Information and Planning Services.
Alexandria, VA: Author.
* U.S. Department of Labor Employment and
Training Administration. (1999). Testing and Assessment: An
Employer's Guide to Good Practices. Washington, DC:
Author.
Forrester Research. (2000) The Truth About the Digital Divide. Forrester.com.
Gates, W. (1995). The Road Ahead. New York: Penguin.
Heubert, J. P., & Hauser, R. M. (Eds.). (1999). High Stakes: Testing for Tracking, Promotion, and Graduation. Washington, DC: National Academy Press.
Market Data Retrieval. (1999). Technology in Education, 1999. Shelton, CT: Author.
Sistek-Chandler, C., Online Assessment: Changing the Way You Test, Converge, November, 2000.
U.S. Department of Commerce. (1999). Falling Through the Net: Defining the Digital Divide. Washington, ED: Author.
-----
Janet E. Wall is Assistant Vice President of ACT-Educational Technology Center in Hunt Valley, Maryland.
ERIC Digests are in the public domain and may be freely reproduced and disseminated. This publication was funded by the U.S. Department of Education, Office of Educational Research and Improvement, Contract No. ED-99-CO-0014. Opinions expressed in this report do not necessarily reflect the positions of the U.S. Department of Education, OERI, or ERIC/CASS.
###
[Return to ERIC Digest Search Page]