The Numbers We Need: How the Right Metrics Could Improve K–12 Education, by Frederick M. Hess and Jon Fullerton

“Educators lack the data necessary to pinpoint concerns and successes in schools. A focus on collecting student achievement data in the past ten years has increased the amount of information school districts have, but these data are insufficient. Successful organizations use extensive data analysis to guide decisions, but few K–12 districts have the metrics needed to do the same. This Outlook outlines several steps that, if implemented, could make data-driven management in education a reality and lays out key measurements districts should collect to make data analysis an effective tool for improving education.” (p. 1)

“Domain One: Tracking Student Outcomes. The most important measures are those tracking student outcomes.
Just a decade ago, most districts had abysmal systems for tracking achievement and school completion. Today, most can provide coherent data on how well students are doing on state assessments, but outcome metrics beyond state assessments can be difficult to come by. Key data include:
• Performance of students on various substrands (for example, number sense or spatial relations on the math test) of state tests with results accessible to the classroom teacher.
• Item-level analysis at the individual student and classroom level. This allows teachers to analyze whether all or most of their students miss the same test items, and then to adjust their teaching strategies.
• Employment or enrollment status of students after high school.” (p. 4)

American Enterprise Institute, February 2010

College- and Career-Ready: Using Outcomes Data to Hold High Schools Accountable for Student Success, by Chad Aldeman

In this report, Aldeman analyzes the effect that including postsecondary outcomes data in high school accountability indices has on whether schools are labeled as success or failures.

Aldeman finds that an accountability index that includes postsecondary outcomes is a more accurate gauge of high school quality.  Schools that produce students more likely to succeed in college are our best high schools, and yet AYP measures alone sometimes gauge these schools as failures.  For example, on average, graduates of Oregon high schools that met AYP had lower first-term college GPAs and lower one year college retention rats than graduates of schools that failed to make AYP.  By including postsecondary outcome data, like CER and credit accumulation, states will avoid mischaracterizing some of our best high schools as “failures.” 

The analysis of Florida high school data showed that including postsecondary outcomes yields a more stable and more consistent measure year after year, avoiding the wild fluctuation in scores that undermine public confidence in accountability systems.

A high school accountability index that includes postsecondary outcomes also has the advantage of encouraging alignment between K-12 and higher education. 

Education Sector, January 2010

Data Quality Campaign Annual Progress Report on State Data Systems

The Data Quality Campaign has developed a set of 10 essential elements necessary to making a functioning longitudinal data system. They also perform yearly surveys, evaluating which elements each state possesses. The essential elements are:

1. A unique student identifier that connects student data across key databases and across years
2. Student-level enrollment, demographic and program participation information
3. The ability to match individual students’ test records from year to year to measure academic growth
4. Information on untested students and the reasons they were not tested
5. A teacher identifier system with the ability to match teachers to students
6. Student-level transcript data, including information on courses completed and grades earned
7. Student-level college readiness test scores
8. Student-level graduation and dropout data
9. The ability to match student records between the P–12 and postsecondary systems
10. A state data audit system assessing quality, validity and reliability

Data Quality Campaign, November 2009