CEPI - Commonwealth Educational Policy Institute

View Print Friendly Version | Close Window

Public Accountability: School Report Cards

One of the important components of the standards/assessment driven reform movement is holding schools accountability for student performance. An increasingly popular approach to school accountability is use of school “report cards” or “accountability reports” to inform the public about how well the school is doing. Based on the long-standing use of student report cards, the purpose of school report cards is to convey information about the school and student performance in a familiar format that increases parent involvement and facilitates public understanding and appropriate use of the information. In short, policymakers hope that school report cards will make schools more accountable and, in turn, more effective.

School report cards have been used throughout the country since the mid-1980s. By 2001, 45 states (including Virginia) will publish “standardized” annual report cards on individual schools (including 14 of 15 Southern Regional Education Board states). In addition, many school divisions have their own school report cards. Over the past ten years school report cards have changed to focus more on student performance and less on school inputs or resources. This is because of the increased importance and consequences of student performance at the school level. Another trend is that many school report cards are showing the adequacy or consequences of student performance rather than, or in addition to, test data or other student performance data. According to Education Week, however, current statewide efforts “show that most are skirting the edges of a serious accountability system” (Quality Counts ’01, Special Report). The Education Week survey study found that:

Only 27 states (including Virginia) publicly rate the performance of all schools or identify low-performing ones.
Only 14 states have the power to close, take-over, or overhaul chronically failing schools.
Only 20 states provide monetary rewards to high performing schools.
Only 24 require high school graduation tests.
Only 3 link student promotion to test scores.
Only 2 have attempted to tie the evaluation (will rise to 7 in 2003) of teachers to student performance.
Only 13 states (including Virginia) require school reports to be sent home to students’ parents.

In many states, school report cards are part of a larger accountability system, including, for example, school accreditation, high school graduation, imposing sanctions, for giving awards and financial incentives. There are also unintended uses of school report cards that are not part of a prescribed accountability system (e.g., ranking schools, using results to market a community or community area). No two states follow the same formula in the way the report cards are designed and used.

School report cards now communicate two important messages to the public. First, accountability at the school level, not just the division level, is important. Second, schools are to be judged by what is produced, not on the basis of school resources, efforts, and other “inputs.” The current focus is to “show me the results.”

Proponents of standardized school report cards (ones that contain the same or nearly the same information for each school in the state) see the following benefits:

Easing state regulations and leaving decisions about how education is provided to the local school and community.
Greater school and community accountability.
Greater focus on student performance, especially as tied to desired levels of performance.
Improved public confidence in schools as reports show improvement.
Greater clarity in the goals for public education.
Improved communication with the public concerning school performance.

As is the case with high-stakes testing, advocates of standardized school reports include politicians and business and civic leaders. The widespread adoption and use of school report cards suggests that there is general acceptance of the approach. In Virginia, The School Performance Report Card, which has been issued since fall, 1999, attempts to have the following positive effects:

To build communication and promote understanding between schools, parents, and the community.
To provide information for the public to convey results and accountability.
To provide a snapshot of school and student achievement to parents and to the community.

Critics of school report cards, while in general agreement with the concept and goal, believe the devil is in the detail. Parents, teachers, and researchers tend to maintain that the manner in which report cards are prepared and the specific nature of the information provided often results in the following negative consequences:

Too much emphasis on standardized test scores as a measure of school effectiveness.
Encouragement of a ìone size fits allî mentality that is inconsistent with reality.
Less emphasis on other measures of school performance, such as the percentage of graduates attending college, employer feedback, and noncognitive impacts resulting from student participation in co-curricular activities.
Encouragement of unfair school to school comparisons.
Disproportionate negative impacts on schools in urban and low socioeconomic areas.
Teachers and schools being held accountable for test scores that are influenced by factors that they can not control, such as parental involvement, home environment, aptitude, and outside-of-school activities.
Late reporting of information that, as a consequence, is not very helpful to schools or teachers.
Given the minimum benefits derived, school report cards are not a wise use of resources.
Narrowing of curriculum to focus only on tested areas.

It is not so much a matter of pros and cons about whether school report cards should be used. The issue is manifest in the nature of what is included on the card and how it is shared with the public.

Information on all states’ school report cards can be found at the Center for Education Reform

Of the 36 states requiring school report cards in 1999, 26 made the report cards available on the Internet and only 13 states required that report cards be sent home to parents (Quality Counts ’99)
Recent focus group research confirms that parents want school report cards that downplay demographic data, include comparisons with other schools, state and national averages, and performance from previous years, as well as fixed standards, all in a concise, easy to read format (Reporting Results, 1999).
80% of parents, taxpayers, and educators indicated a desire to have longer, more detailed school reports as well as short, concise reports (Reporting Results, 1999).
Out of a possible 21 indicators, parents indicated that school safety, teacher qualifications, class size, and graduation rates were more important than statewide test scores (Reporting Results, 1999).
Survey research of adults indicates that indicators such as teacher qualifications and course offerings are more important than student test scores or student demographic data (Jaeger, Gomey, & Johnson, 1993; McMillan, 1996).
Educators typically find school report cards less useful than parents (Kochan, Franklin, Crone, & Glascock, 1994).
Poorly designed or inadequate school report cards tend to be ignored by parents and the community (Herrington, 1993).
Principals tend to believe that requiring schools to report performance data by race has a negative impact on the educational system, increasing racial tensions (Herrington, 1993).
Only a third of parents and a quarter of educators believe that test scores should be used as the main measure to hold schools accountable (Quality Counts ’99).

Because school report cards have been used for more than a decade there is some research and a wealth of experience that provides helpful information about best practice and related policy. The issue is best delineated into three critical components: what information is reported, how comparisons are made, and how information is reported.

What Information is Reported?

The second step in developing a school report card, after the purpose is clarified, is to specify the information that will be reported. Obviously student achievement data are critical, but several studies have found that other kinds of information are very important to parents, usually more important than standardized test scores (Caldas & Mossavat, 1994; Gaines & Cornet, 1992; Jaeger, Gorney, & Johnson, 1994; McMillan, 1996; Reporting Results, 1999). Often, what the public views as helpful and informative is not consistent with what policymakers and school officials are seeking. For example, while policymakers tend to emphasize learning outcomes as demonstrated in test score data, parents are typically more interested in school safety and teacher qualifications. This highlights the importance of being clear about purpose — is it to inform the public about student performance as mandated by state policy, or is it intended to give parents information they want to judge their childrens’ schools? Education leaders and policymakers need to remember that what they may want to tell the public about schools is not what the public wants to hear.

The information contained on school report cards can be categorized into three major types: inputs and school contextual factors, process indicators, and outcome indicators. Inputs and school contextual factors include the following kinds of data:

Demographic characteristics of students, such as the percentage of students eligible for free and reduced price lunch (socioeconomic status of the students), racial percentages, limited English proficiency percentages.
Per pupil spending.
Division and community characteristics.
Student mobility.
School facilities.
Parental involvement.
Class size
Pupil/teacher ratios.
Resources provided.
Special education rates.

In recent years, with greater emphasis on student performance, reporting most of these input and contextual factors has become less important (class size is an exception). However, most educators maintain that accurate and appropriate interpretations of student achievement results require an understanding of context. Despite the currently popular notion that “all students can achieve high standards,” educators experience first hand the influence of such factors as poor parental involvement, poor readiness skills, large classes, and frequent student transfers to different schools.

Process indicators are those school factors that directly effect student performance. They are important because information about them may suggest where improvements are needed (Eisner, 2001). Common process indicators include:

Teacher qualifications and years of experience.
Course and program offerings.
Co-curricular activities.
School environment.
Hours of homework per week.
Student attendance.
Opportunities for students to work cooperatively.

In studies of parental preferences about what to include in school report cards two of these factors, teacher qualifications and course/program offerings, were found to be as if not more important than student outcome data such as test data. (Jaeger, etal., 1994; McMillan, 1996)

Outcome indicators are measures of student performance, accomplishment, and behavior. These indicators are the “outputs” that are increasingly used as the majorbasis for judging school quality, particularly by policymakers. Student performance on test scores is a major outcome, but most report cards include other indicators, as illustrated in the following:

School safety, such as number of arrests or violent incidents.
Graduation and/or dropout rates.
Suspension rates.
College attendance rates.
Grade promotion rates.
Grades.
Percentages of students receiving different types of high school diplomas.
Employer satisfaction.
Accreditation ratings.
Special awards received.
College scholarships received.

In addition to these measures that are clearly a result or outcome, many secondary school report cards also include the numbers of students attending advanced placement, college-level, and international baccalaureate courses.

Policy decisions about which of these many indicators should be included need to consider conflicting needs to be concise on the one hand, limiting what is presented, and, on the other hand, to provide sufficient data so that the information is accurate and useful. That said, there are some elements that are essential, including data on school safety, teacher qualifications, student socioeconomic status, test scores, and graduation rates. In Virginia, the following indicators are included in school report cards:

Input

School enrollment
AttendanceProcess
Teacher training
Advanced courses and program enrollment and AP course scores

Output

Accreditation rating
SOL test results
Percentage of students taking SOL tests
School safety
Diplomas received (high school)
Dropouts (high school)

Nature of Comparisons

All school report cards include some kind of comparison. Schools are typically compared in one or more ways: 1) with other schools, division totals, state totals, or national totals, 2) with set standards, and 3) with past performance. Currently, only six states include comparisons with other schools, while nearly all of the states have school report cards that compare school averages with either state, division, or national averages. Nineteen states compare schools with set standards, such as percentage of students in a school obtaining passing scores (will rise to 25 in 2002). About half of the states include data for two or more years to show trends over time. In Virginia, school report cards include comparisons with division and state averages, comparisons with set standards, and comparisons with previous years.

A few states, such as North Carolina, Kentucky, South Carolina, Oklahoma, Indiana, and Tennessee, “adjust” scores so that comparisons take inputs into account. Typically, either some kind of “expected” gain based on previous achievement is compared to actual gain, or like schools are clustered together so that comparisons are made among schools with similar characteristics. The first of these approaches is sometimes referred to as “value-added” accountability (The Tennessee Value-Added Assessment System, developed by Dr. William Sanders, is best known; value-added accountability for Virginia is argued by Wheat (2000)). These approaches are controversial. Some believe comparisons with like schools or with consideration of where students are when entering school make sense because either apples are compared to apples, in the case of like schools, or improvement in performance is stressed. It is argued that improvement or change in performance is what schools can best impact through instruction (rather than measuring student entering quality, which has a significant impact on status measures at the end of the year). Others maintain that what is most important is the level of performance at the end of the year compared to set standards, regardless of the starting point or what similar schools are doing. The major tenet of this argument is that expectations are then the same for all students (i.e., “high standards for all students”), eliminating the potential negative impact of lower expectations.

Whether formally or informally, comparisons with like schools are generally helpful in examining strengths and weaknesses. Such comparisons are not healthy if local context is ignored (even like schools are different in important ways), and if there is an over reliance on test scores (test scores show the achievement of only some important school goals). It may well be that different kinds of comparisons are most helpful for different audiences. Local school board officials and policymakers may need to have comparisons of like schools and trend data over many years to set policy and allocate resources, while parents may be most interested in national comparisons with desired standards that provide a holistic perspective of school performance. SchoolMatch is a school research and database service that provides comparable data from different districts and states. Comparisons can be customized to show differences between like districts and schools nationwide.

How Information is Reported

School report cards present information in many different ways, using tables, graphs, percentages, percentiles, frequencies, narratives, and ratings. Some report cards are short, perhaps only two pages, others are much longer. Given the variety of formats, different numbers of indicators used, and level of specificity, what policy decisions need to be considered to ensure accurate interpretation and maximum utility?

Research and experience suggest that school report cards are most effective when four principles for presenting data are incorporated:

Do not include too much information. Report cards need to find the appropriate balance between too much and too little information. There is some research to suggest that reports from two to four pages are best (Jaeger, Gorney, and Johnson, 1994). The greatest challenge is not presenting so much information that readers are overwhelmed and confused. In New York, for example, the 11 page report was viewed as containing much too much information (Quality Counts ’99, 1999). One effective approach is to present a brief overview of findings in two or three pages, followed by more detailed information. Another approach is to customize reports to different audiences. Reports given to parents could be more concise and include more narrative, while longer reports with less use of narrative and more use of tables and graphs could be prepared for professional educators and policymakers.
Use effective graphic design. People are more likely to read a report that is well-organized and uncluttered. Attractiveness, appropriate use of headlines, and layout are important considerations. The worst case is trying to cram as much information as possible into one or two pages. An example of what an effective format could look like is shown in a prototype report, found at Education Week.
Include appropriate narrative. Short narrative explanations in each major section help readers put the information in context and facilitate understanding. Without these narratives, tables and graphs are more likely to be misinterpreted, particularly by non-expert readers. Narratives also help place numbers in context.
Limit information to what is important. School report cards are most effective when the information that is presented is limited to what readers of the cards want and need to know. Thus, information that does not contribute to targeted interpretations and conclusions should not be presented. For example, it may not be very important to report the number of male and female students in a school, or to report both raw and scaled test score results.
Distribute using a variety of dissemination methods. To get school report cards in the hands of parents it is best to use several methods of dissemination, including mail directly to the homes, having reports available from the school, sending reports home with students, and having the reports available on the Internet.

The development and distribution of school report cards is closely related to consequences for schools. Since the purpose of the report cards is to provide information to the public about how well schools are functioning, how much of the total picture should be reported? Are there other data that are released that also help inform parents? If tied to school accreditation, should be report cards indicate whether schools are accredited? Since report cards issued by the state can not include all pertinent information, divisions and schools need to decide what additional information to provide. Divisions also need to decide how additional information is provided.

As with high-stakes testing, school report cards can consume considerable resources. Because of this, it is important to conduct appropriate research and evaluation studies to be sure that the results are worth the expense. If it is found, for example, that the majority of parents do not use or accurately interpret the information on the reports, then changes are needed. It may also be unnecessary to issue yearly reports. By having a more comprehensive report every three or four years, for example, with specific school improvement implications, may be a better use of resources than briefer yearly reports that may not provide sufficient information.

Statewide school report cards imply some level of standardization in public education. This suggests that local educators need to be clear about how their schools are unique and how their public receives information that is targeted to the division’s more specific goals and objectives. It would be helpful for state level policymakers to recognize this need in the state-developed report cards, and to perhaps provide resources to divisions to develop local school report cards to ensure that a balanced, complete picture of each school is presented.

School report cards are integral to standards-based, high-stakes testing accountability as one means of informing the public about the progress of students and schools. There are many improvements that can be made, however, particularly as data from multiple years are available. Perhaps the most important function of these reports is to show how improvements are being made across several years. Careful consideration will need to be made with respect to report card design to show data over years, and comparisons with division or state averages may need to give way to comparisons over years to stress the importance of longitudinal analyses.

It is clearly not sufficient to simply prepare reports and distribute them. There is a need for proper support to ensure appropriate use. One kind of essential support is making sure that teachers and administrators understand and can accurately interpret the reports. Teachers, because of their direct contact with parents and generally high credibility with parents, need these competencies. This requires appropriate staff development. Training is also needed to help school administrators and teachers use the data for school improvement.

At present, we do not know very much about how school report cards are used. There is a clear need to invest resources to determine whether the purposes are being met. Studies need to be conducted to examine the impact of the reports. How many parents read them? What do the reports mean to parents? How do parents interpret the data? How do schools and policymakers use the report cards? How do high school report cards help the public know if students are prepared for higher education and the workplace? These kinds of questions need to be asked and answered to have more complete information on the consequences of school report cards.

Finally, there are limits to the information that can be provided as well as to the nature of the interpretations that are reasonable. With these limitations in mind, information accompanying school report cards should include clear statements about what should, as well as what should not, be concluded. For example, while it may be accurate to make conclusions related to student performance over several years, it would be incorrect to assume that school report card data is the only set of indicators of school success.

Recently Education Week summarized ten reasonable, helpful recommendations by A-Plus Communications, for more effective use of school report cards (Quality Counts ’99):

Plan ahead - and avoid wishful thinking. Be clear about audience, indicators, report format, and mechanisms for collecting feedback.
Test scores provide only part of the picture.
People want to know about safety, teacher qualifications, and other measures.
Comparisons count. Parents and taxpayers want to know how “their” school measures up to other schools, as well as changes over time.
Be cautious about the labels you assign to schools. Parents do not like labels such as “exemplary,” “adequate,” or “low performing” since the reports typically do not contain all information to apply such labels.
Downplay demographic data. Parents rate indicators such as race, primary language, or percentage of students receiving free and reduced price lunch low ratings as helpful school report card indicators.
Make the report easy to read.
Use credible messengers.
Help people understand how to use the information.
Use report cards as a tool to engage your community.

In Virginia, reference to these recommendations will be helpful in resolving concerns generally created by such reporting systems. Specifically, it may be prudent to consider more research and evaluation of the effect of the report cards as well as the effectiveness of different reporting formats, including clear narrative about what the information on the card does and does not mean, and consideration of deleting some currently provided information, such as numbers of students enrolled and students included and excluded from SOL testing. Additional information could be provided on teacher quality and trends across time and school divisions need to supplement external assessment results with local data that provide more specific information.

Click here for summary of recent Virginia Legislative history of “School Report Cards.”

Brown, R. S. (1999). Creating school accountability reports. AASA Online School Administrator

Caldas, S. J., & Mossavat, M. K. (1994). A statewide program assessment survey of parents’, teachers’, and principals’ perceptions of school report cards. Paper presented at the Annual Meeting of the American Educational Research Association. ED374159.

Eisner, E. (2001) What does it mean to say a school is doing well? Phi Delta Kappa 82 (5), 367-372.

Gaines, G. F., & Cornett, L. M. (1992). School accountability reports: Lessons learned in SREB states. Atlanta: Southern Regional Educational Board.

Herman, J. L. (2001). Accountability bottom up, The CRESST Line (Winter), 1-2, 4.

Herrington, C. D. (1993). Accountability, invisibility, and the politics of numbers: School report cards and race. Paper presented at the Annual Meeting of the American Educational Research Association. ED 371976.

Jaeger, R. M., Gorney, B. E., & Johnson, R. L. (1994). The other kind of report card: When schools are graded. Educational Leadership , 42-45.

Jaeger, R. M., Gorney, B. E., & Johnson, R. L. (1993). The nation’s schools report to the public: An analysis of school report cards. Greensboro, NC: University of North Carolina – Greensboro, Center for Educational Research and Evaluation.

Kochan, S. E., Franklin, B. J., Crone, L. J., & Glascock, C. H. (1994). Improving Louisiana’s school report cards with input from parents and school staff. Paper presented at the Annual Meeting of the American Educational Research Association.

McMillan, J. H. (1996). Public perceptions about what to include in school report cards. Paper presented at the Annual Meeting of the American Educational Research Association.

Quality counts ’99. (1999). Bethesda, MD: Education Week.

Quality Counts ’01. (2001) Bethesda, MD: Education Week.

Reporting results (1999). Washington, DC: A-Plus Communications.

School report cards state by state around the nation. (2000). Washington, DC: The Center for Education Reform.

Wheat, D. (2000). Value-added accountability: A systems solution to the school accreditation problem. Springfield, VA: Thomas Jefferson Institute for Public Policy.

Organizations

Achieve, Inc

National Center for Research on Evaluation, Standards, and Student Testing

Click to provide comment or additional information. (to: cepi@vcu.edu) Please indicate in e-mail copyright source and contact info for new inclusions.

View Print Friendly Version | Close Window