Ida Hill, Editor

K-12 Web-Based Assessment Implementation

Efforts to make our schools more accountable, the belief that no child should be left behind, and visions for technology as a catalyst for change and education improvement have contributed to an emerging K-12 testing and technology “bubble.” Test designers and administrators in collaboration with educators, politicians and business leaders, have turned to technology as a delivery and information retrieval tool for assessment. This union is of critical importance to 21st century education.

On-line assessment and web-based assessment are terms used at times interchangeably and at other times to imply a difference. On-line assessment, to some educators, is downloading questions from a server, responding to them and sending responses back to a central server which may be located at a vendor site. Web-based assessment refers to the process of having questions come directly to and completed at an individual computer terminal through a centralized source. There is also a tendency to use the term web-based assessment to refer to “web-enabled systems that improve standards instructional, remedial and testing capabilities” (Virginia DOE, 2000). In this brief, on-line assessment refers to Internet-based delivery and scoring of tests, while web-based assessment will incorporate subsequent instructional and remedial capabilities in addition to the Internet based delivery and scoring of tests.

Information concerning the implementation of on-line or web-based assessment is summarized in this document. The purposes of the summary are to reveal what supporters and critics have to say about the movement, to provide insight on what is practiced in schools, to review related research and the status of accessible technology, and to cite support needs. Additionally, some policy decision questions will be raised to stimulate research, debate, and evaluation. This summary complements the on-line assessment policy brief under the heading “Assessment” by McMillan (2001).

Supporters of web-based assessment and critics alike readily cite reasons for support and potential barriers to success with on-line use. Assessment of standards statewide at the K-12 level is relatively new, but use of technology for delivery of standards tests is newer. Implementation is largely at the pilot stage. Thus, the process of implementation of a web-based program is evolving and its novelty makes it an attractive springboard for creative solutions and speculation. Supporters for the use of technology for delivery and retrieval of assessment information speak about political and business support, access, databanks, technology integration, infrastructure efficiency, student equity, and student acceptance of use of technology for assessment.

Political and Business Support. Politicians and business leaders interested in sustained and predictive funding for technology acknowledge that online assessment is one way of ensuring constituents that investments in technology yield positive results. Some also acknowledge that the resulting student training will produce more qualified students for technology-related jobs in business.

Speedy Access to Test Results. Central delivery and management of data make possible simultaneous and rapid aggregation of test results across schools, grades, classes, and students.

Databank Availability. Implementation of a large variety of studies, analyses, and observations will be possible at state agency and school levels. Such investigations that posed challenges previously will soon be available. Through electronic manipulation of data in “tanks,” useful information can be brought closer to the test administration site. Such manipulation, when short term, will benefit on-going teaching and learning.

Technology Integration to Achieve Broader Education Goal. Harnessing technology for web-based assessment will provide opportunities for schools to receive an abundance of resources and information. Classroom-to-classroom, school-to-school and school-to-state Internet, T1, and cable connections ensure access to better and improved learning resources for all content areas and other school programs. Statewide standards for infrastructures, for example, will provide technology access to all students including those tested with learning disabilities, those who are English language learners and are low SES students, and those who experience high stakes test taking problems.

Efficient and Consistent Use of Technology Funds. Access to and sophisticated use of technology has become a major priority in many states. It is the primary focus in as many as six states that have made a commitment to on-line assessment. Schools assisted by state agencies and Legislatures through provision of guidelines and grants will put in place those technologic systems necessary for delivery, feed back and aggregation of assessment data.

Equity. Proven equipment necessary for on-line assessment includes (a) an Internet- ready local area network hooked up to speedy T1 lines or cable modem,(b) up-to-date computers, and(c) high speed, high bandwidth capability. The presence of this technology in all schools will ensure a foundation level that is currently unavailable in some schools. When properly implemented, the foundation will allow for growth through simultaneous connections of multiple devices.

Other examples of support for on-line assessment include public support for technology expenditures across several years and a reduction in test administration requirements in schools. Legislators and government officials are among the first to admit that funding has not been at desired levels and they feel that the public may have supported more funding for technology if its use could be directly linked to improvement in student performance. One obvious reduction in test administration is paperwork such as handling and screening test booklets.

Potential barriers to the use of technology in assessment programs include existing infrastructure and computer and related technology challenges, transition from traditional assessment to on-line assessment, time requirement for implementation, on-going financial commitment, and situation- specific problems.

Existing Infrastructure Challenges. Schools have been in the process of developing networks for several years. Because of differences in engineering, systems vary in capacity, speed and technology. Some systems were developed with professional assistance and others were “self-engineered” by local school personnel.

Classroom Technologies. For several years, schools have been the recipients of purchased and donated computers and related equipment. While most have been well received and continue to serve instructional purposes, many are now dated, worn out, obsolete in terms of locating repair parts, and incapable of performing at the desired level for network connectivity.

Traditional Assessment Transition to On-line Assessment. Large-scale K-12 assessments were developed initially as pen and paper tests. A move to electronic testing will require certain accommodations; and the resulting compromise may not take full advantage of technological enhancements such as multimedia.

Time Requirement for Installation and Implementation. The number of computers in schools range from large computer to student ratios such as 30 to 1 to ratios of less than 5 to 1, from low bandwidth capability to high speed, high bandwidth capability, and from no Internet service to full Internet service. Additionally, aging and obsolete equipment is evident in classrooms. labs, and libraries. A reliable, efficient statewide infrastructure may require several years of readiness before it is useful for assessment in all grades and classes in each school.

On-Going Financial Commitment. Implementing systems to meet instructional, remedial and testing needs require specific facilities, network and computer/workstation platforms. Technology acquisition, installation, management, operation, and training requirements are costly and cannot be satisfied with one-time up front investments. Large-scale and on-going financial investments in technology may be perceived by some policy makers, funders, and critics as investments that will drain resources from other projects and programs.

Situation Specific Problems. Problems such as viruses, cracked security, poor software, and on-site placement of equipment that are specific to the use of the Internet, local networks and computers are certain to become on-line assessment problems. An issue for security, for example, is “how does the computer know that I am who I say I am?” Another may relate to the trade-off between security and the maintenance level of school administration.

Barriers not mentioned that might or might not surface due to the changing landscape are student anxiety, insufficient teacher training, and system failure. Each will be impacted by experience. Because more and more students and teachers have access to computers and the Internet and the number of homes equipped with Internet-connected computers are increasing at a rapid pace, these barriers may have little or no significance in years ahead.

Because of its infancy, on-line assessment of K-12 standards within states has not been practiced sufficiently long to provide time for researchers to study the program. As expected, there is widespread interest and variety in practice. A commitment to a formal initiative is evident in less than ten states. There, however, are some observations and developments that appear to have significance to a growing collection of information.

“The standards movement and high stakes testing present both opportunities and risks to students of color, English language learners, and students with disabilities (Heubert, 2000).” Opportunities afforded by technology include the use of print enlargement, use of multimedia, and interactivity.
Some advocates for students with special needs are not aware of technology that assists testing. In 1999, a group of Oregon parents filed a lawsuit against the state arguing that their children were not permitted to use technology tools when they took standardized tests. Now, on a case-by-case basis, students are allowed to use spell- checking software, dictation machines and other technology. (Bushweller, 2001).

A report on Internet-connected classrooms in Maryland revealed that 72% of the state’s classrooms are presently connected. Only 58% of the classrooms were connected in 1999. But, in Maryland, students in wealthier schools are three times more likely to use technology to perform measurements and collect data (Maryland DOE and Business Roundtable for Education, 2000). Observations such as this one may provide insight about what use is made of the computer by select populations.

Research is mixed on whether there is a gender gap in the use of technology. Organizations such as the American Association of University Women believe that “there is a need to cultivate girls’ interest in infusing technology concepts in subject areas.” Cisco Systems has implemented a program to encourage girls to enter computer fields. Others argue that the difference between males and females in choosing to pursue a career in computer fields is a natural process. They believe males and females should be free to make choices and not be pressured into technology-related careers (Kleinfeld, 1999). Investigations in this area, however, will reveal interesting information about anxiety and skill in technology use in testing in addition to gender information.

Broadband access at least the quality of T1 is presenting connection problems for remote and sparsely populated communities and schools. Connections simply do not exist in some communities. In North Dakota, the state Legislature is working to provide such facilities to its unserved communities and schools (Zehr, 2001).

Goal number 5 of the newly revised National Educational Technology Goals is “digital content and networked applications will transform teaching and learning.” This goal supports the building of networked infrastructures to serve diverse needs. The intent is to have infrastructures “wired and wireless, desktop or handheld, that allow multiple devices to connect simultaneously to the Internet throughout every school and community in the nation (Executive Summary, 2001).” This type of infrastructure has great potential for implementation of on-line assessment and other forms of evaluation.

School personnel and staff training for the use of technology must incorporate web-based training for testing if the expected results are to be obtained. Eighteen percent of teachers say that training is a great barrier to their use of technology. An additional 48% say it is a moderate and small barrier. History indicates that the more time teachers spend in training, the more comfortable they become with the use of technology and the more likely they are to use it in the classroom. (Meyer, 2001).

Many state educators and politicians have not yet focused on the need to accelerate a movement to web-based assessment. The inattention is not due to a lack of awareness of the potential of using technology for this purpose, but rather attention has been given to more pressing needs considered prerequisites to web-based assessment. State allocations of funds are, at present, being exhausted in three areas. They are acquisition of technology and connection to the Internet, use of technology to improve the way teachers teach, and provision of access to on-line resources to students and teachers.

Widespread efforts and initiatives funded by Federal, state and local agencies have resulted in tremendous gains nationwide in computer and Internet access. The number of schools and classrooms connected to the Internet via T1 or cable modem has increased substantially in virtually every state. Nationwide, the ratio of students to computer according to Market Retrieval Data was 19.7 to 1 in 1997 and 7.9 to 1 in 2000. US DOE surveys reveal that almost all schools are connected to the Internet and 63% of the nation’s classrooms are connected to the Internet.

Educators and community leaders who work in schools are quick to point out that these indicators of success, while encouraging, do not reveal deficits that will delay statewide implementation of on-line assessment across schools, grades, and classes. Cited often are the number of computers in classrooms, differing ratios for students to computers from class to class, Internet access in rural and high poverty schools, a growing inventory of obsolete and dated hardware, lack of funding in some states for technology, varying levels of local network efficiency, and unavailabl telecommunications services in some schools.

Another deficit is a lack of computer use by teachers and limited on-line use by teachers. Teachers admit that there has been insufficient training for teachers and other instructional personnel. Viewed as a part of school readiness, teachers say they lack support in integrating telecommunications into the curriculum. Additionally, students in grades 7-12 state that computers are not used often to help them understand new concepts. They also say that they have learned most of what they know about computers at home (Education Week, 2001). On the other hand, gains in the number of states training teachers to use technology are impressive.

North Carolina, Massachusetts, Tennessee, Texas, Wisconsin and West Virginia are among 26 states that provide consistent training or require teachers to engage in several hours of technology training. North Carolina, Virginia and Idaho are among states requiring training before new teachers or practicing teachers can renew teaching credentials. Idaho, Michigan and North Carolina administer a technology test as a requirement for initial licensure. Training is much less impressive in high poverty and rural schools where training is strictly a local option. It should noted be noted however, that through use of funds other than state funds, some of these schools have developed exceptional training programs.

Some students benefit from teachers who know how to harness the powers of technology and others do not (Panas, 2001). Some students benefit from system managers who know how to build and maintain efficient networks and some do not. Some students benefit from technology- rich leadership and some do not. Thus, pockets of disparity in training are still the concern of those who desire to use high-risk applications of teaching and learning resources. Misuse of technology by untrained school test administrators and others will unfairly impact student performance and technology integration.

Provision of web-based resources to all teachers and students requires access to specific software and time to learn, practice and plan. A US DOE, National Center for Education Statistics, Fast Response Survey (1999) indicated that 82% of teachers mentioned insufficient time for them to receive training, to practice, and to plan as the greatest barrier to their use of technology. Additionally, 71% of the teachers say that they do not have good instructional software. These two findings closely relate to training. If teachers are to use the technology as a teaching tool, it must be handy like a cell phone and they must be comfortable with it in all applications including on-line assessment. Comfort comes with knowledge and experience. North Carolina, Georgia, North and South Dakota, Virginia and Washington are among states providing web-based testing, instructional, and administrative experiences prior to implementation of on-line assessment. A few states are using the Virtual Education Space Project considered significant in the area of high stakes testing.

Other on-line testing concerns include transition from pencil and paper to an electronic medium, data security, virus control, and student computer identification. As mentioned earlier, these concerns are more speculative than actual. Actual concerns arising out of experience will provide information for research. A Standards of Learning Demonstration Project in Virginia, for example, is already identifying problems. Feedback from vendors piloting implementation of on-line assessment is being documented for decision making.

Because full implementation of web-base assessment of standards in high, middle and elementary schools will require substantial resources especially fiscal and human, related issues will extend outside education. Use of these resources, however, will be of benefit to programs inside and outside education. Government agencies, higher education institutions, and business are expected to play important roles.

One issue is should the acquisition, maintenance and replacement of computers and related technologies be negotiated for a single purchase for all state agencies, higher education institutions and schools? While several states have contracted with vendors for mass purchases of computers and other technologies and passed reduced product costs onto the schools and agencies, fewer states have tackled the negotiation of maintenance and replacement of computers and other technologies long term. As schools and educational institutions become more accountable, it is likely that this issue will move front stage.

A second issue is how will additional money be obtained for viable educational ventures such as on-line assessment in seemingly exhausted budget cycles? State commissions and councils engaged in continuous budgeting research will be called upon to study and make recommendations related to creative uses of existing funds and to identify new funding options should on-line assessment remain popular.

A third issue is how will schools compete with business to recruit and retain qualified technical personnel in an already saturated market? Businesses acknowledge that the pool of applicants for high tech jobs is insufficient. As school networks become more complex and dynamic, personnel capable of managing them will be the same people sought by business. States such as Georgia and Iowa are already reporting that hiring technically qualified personnel is difficult even when money is available.

A fourth issue relates to a need to better coordinate all of a state’s technology programs. Obvious areas of study are overlap, duplication, and need. Decision and policy makers will turn to research institutions, commercial research agencies and schools for assistance. Additionally, issues concerning on-line assessment may be found in a CEPI on-line assessment policy brief by McMillan(2001).

Holding all students, teachers and administrators to high standards of education is the focus of the “standards movement.” Testing advocates are looking for promising ways to assess student learning, technology advocates are looking for ways to efficiently use technology, and educators, in general, are looking for resources that will assist instruction and testing to improve student performance.

One promising trend is the use of technology as an assessment tool. A change in process from the cumbersome and time-consuming data and information collection system presently used to a more reliable, efficient data gathering and data manipulation system encompassing instructional information for on-going classroom use is desired. As several states move in this direction, observers recognize the need for differing approaches and some experimentation. States’ internal structures while alike in many ways, differ in engineering, organization, and methods of funding.

And since the use of web-based assessment is still in the novelty stage, who is in a position to say which approach to testing is best or which infrastructure is most efficient? Thus, in the months ahead, policy makers and educators will continue to identify positive consequences resulting from high stakes assessment; and will simultaneously identify positive consequences resulting from high stakes assessment within an electronic environment. In the Virginia demonstration project mentioned earlier, questions were raised when on-line assessment was carried out electronically in nine schools. Three vendors were contracted to demonstrate solutions including industry best practices. Vendors were also asked to develop and build on technical skills teachers possess to operate systems as a part of on-going professional development. A larger scale project involving all high schools is planned for completion in 2003.

It is certain that projects such as the Virginia and Iowa assessment projects will contribute to discussions that will serve as impetus for questions leading to policy development. Commitment by states to web-based assessment of standards is a course in “uncharted” waters for K-12 schools. Questions for research and policy will likely include the following:

1. Will the move to on-line testing encourage teachers to assess learner needs and personalize instruction with greater frequency throughout the year?

2. Will use of computer software and school infrastructures for testing accelerate or restrict the frequency of use of on-line resources for instructional and remedial purposes?

3. What impact will on-line testing have on the performance of special student populations such as minorities, students with disabilities, students from rural areas, and students from low SES backgrounds?

4. Will the use of technology for assessment result in improved teaching and teacher training?

5. Will negative publicity associated with information made public as a result of centralized databanks lead experienced teachers to leave high poverty and rural schools?

6. Will on-line assessment and resulting information on schools and school performance contribute to a political backlash against high stakes testing programs and the use of technology for such purposes?

7. Will on-line assessments provide new treatment alternatives?

8. What does “ready” or “prepared” for on-line testing mean?

9. Can test information be secured and free of contamination?

10. Will what is learned at the from on-line assessment programs in high schools be transferable to elementary and middle schools?

11. How many computers are enough for on-line assessment?

12. What investments must be made to ensure on-going student, teacher and school readiness for on-line assessment?

13. How much training for teachers is enough for testing and use of resources on-line?

14. Will student performance be negatively impacted by student anxiety related to testing and student anxiety related to the use of technology?

15. Will the Allen amendment in the recently approved Federal tax bill (May, 2001) provide the necessary boost to state and local governments to supplement purchases of educational computer, related equipment, software and Internet access needed to support web-based assessment and other on-line programs?

Click here for summary of recent Virginia Legislative history of Online/Web-Based Assessment.

Web-based SOL Technology Initiative. Virginia, 2001.

Implementing Electronic Government in the Commonwealth of Virginia, Executive Order 65, 2000.

http://www.pen.K12.va.us/VDOE/Technology/Technologysoltech/soltech.html

http://www.ed.gov/Technology/elearning/index.html

http://edStandards.org/standards.html

Bushweller, Kevin. Beyond Machines: Technology Counts. Education Week, 2001.

Doherty, Kathryn and Orlofsky, Greg. Student Survey Says: Technology Counts. Education Week, 2001.

Editors, Dividing Lines. Education Week, 2001.

Fine, Lisa. Special Needs Gaps: Technology Counts. Education Week, 2001.

Gehring, John. Not Enough Girls: Technology Counts. Education Week, 2001.

Hardin, Peter. Allen amendment in bill. Richmond Times Dispatch, May, 2001.

Heubert, Jay. High Stakes Testing: Opportunities and Risks for Students of Color, English Language Learners, and Students with Disabilities. Teachers College, Columbia University, 2000.

Interviews with Lan Neugent and Susan Susbury, Virginia DOE, 2001.

Johnston, Robert. Money Matters: Technology Counts. Education Week, 2001.

Klienfeld, Judith. Quote in “Not Enough Girls” by John Gehring, Education Week, 2001.

Maryland DOE and Business Roundtable for Education. 2000.

Market Data Retrieval. Technology in Education 2000 and Unpublished Tabulations for MDR’s 1999-2000 Public School Technology Survey, 2001.

Meyer, Lori. New Challenges: Technology Counts. Education Week, 2001.

Panas, Glenn. Quote in “Beyond Machines” by Bushweller. Education Week, 2001.

Schnauberg, Lynn. State of the State: Technology Counts. Education Week, 2001.

What Research Says About Student Assessment. Improving America’s Schools: A Newsletter on Issues in School Reform, Spring, 1996.

Zehr, Mary. Rural Connections: Technology Counts. Education Week, 2001.

Click cepi@vcu.edu to provide comments or additional information. Please indicate in an e-mail the copyright source and contact information for new inclusions.

Back to Top

Copyright © CEPI 2000
CEPI grants permission to reproduce this paper for noncommercial purposes if CEPI is credited.