Are Our Students Performing As Well As We Think?

This week the federal government released the latest scores from the National Assessment of Educational Progress (NAEP), also called the nation’s report card. The NAEP tests a representative sample of students in each state. While 4th graders in the United States overall improved their reading performance in 2007, North Carolina 4th graders actually declined slightly in their percentage of higher level (proficient and advanced) reading scores.

State assessment tests tell us that North Carolina students are performing well and improving each year. Yet national standardized tests are more rigorous and paint a picture of much poorer achievement and slower progress. Much of the difference comes down to the fact that our state tests compare today’s students to a curve based on students 15 years ago, and a lot has changed in since then.

In 2001, Harvard University made the national news by reporting that it issued grades of A or A- to nearly half its students. A flurry of articles and reports brought the issue of grade inflation to the forefront. The central issue was whether half of the students at Harvard and other Ivy League schools actually deserved As, or whether something had gone awry. Researchers asked whether the courses were appropriately challenging, and if professors’ expectations were too low.
 
In North Carolina, it’s time we asked our State Board of Education the same questions. According to state assessments, 46 percent of our fourth graders are reading above grade level. Another 40 percent are reading at grade level. Before we pat ourselves on the back for a job well done, we need to know if the job has indeed been done.

In addition to state assessments, North Carolina students, along with students in the rest of the nation, take the National Assessment of Educational Progress (NAEP) tests. These tests provide a secondary, external check of student proficiency. According to NAEP results, fewer than one-tenth of our fourth graders have advanced reading skills, fewer than one-fourth are proficient, and fewer than one-third have at least "an inconsistent mastery" of grade-level reading skills (basic).

Last year, 4th grade reading achievement on state assessments (end of grade tests) reached a new high, and 45.8 percent were categorized as advanced. All told, 85.4 percent of fourth graders were considered to be at least proficient in reading. An in-depth look at the way the state tests are constructed and scored helps to explain the discrepancy between NAEP and state assessments.

The State Assessments
 
In North Carolina, a "proficient" fourth grader is one who performs as well as the average fourth grader in 1992. The tests are North Carolina-centric, and the results have been compared to an echo chamber:

Achievement Levels

Level I:  Students performing at this level do not have sufficient mastery of knowledge and skills in this subject area to be successful at the next grade level. Level II:  Students performing at this level demonstrate inconsistent mastery of knowledge and skills that are fundamental in this subject area and that are minimally sufficient to be successful at the next grade level. Level III:  Students performing at this level consistently demonstrate mastery of grade level subject matter and skills and are well prepared for the next grade level. Level IV:  Students performing at this level consistently perform in a superior manner clearly beyond that required to be proficient at grade level work. Source: N. C. Department of Public Instruction

• The tests were designed to assess whether students are learning the standard course of study, so they assume that the standard course of study reflects what students actually should know at each grade.
• Test items were designed by North Carolina teachers (with training).
• North Carolina teachers identified which students were proficient to determine the initial grading curve.
• The bottom line is that what is "proficient" is based on the judgment of teachers and performance of students in North Carolina 15 years ago.

In 1992, North Carolina teachers were asked to categorize their students into four categories (see Achievement Levels for details), known as Levels I (below basic) through IV (advanced). The State administered the first round of assessment tests and collected student scores. The percentages of students placed in each category by teachers were essentially used to create a grading curve. For example, if 10 percent of students had been classified as Level I by their teachers, then the bottom 10 percent of test scores would be considered "Level I" scores.

This curve remains the basis for the "cut points" in today’s state assessment scores. The highest scores that were determined to be Level I, II, III, and IV in 1992 were set as the break points for future years. The result, not surprisingly, is that reports of student achievement have improved each year. Today’s fourth graders read at a higher level than fourth graders in 1992. But whether they read well enough for today’s challenges is a question these tests cannot answer.

Comparison to the National Assessment of Educational Progress

North Carolina is not alone in its high passage rates compared to NAEP. Studies indicate that in many states the NAEP "basic" is comparable to the state test’s "proficient" category. However, North Carolina’s "proficient or better" is 20 points higher than the NAEP "basic or better," and several states have been successful in creating tests that more closely mirror the NAEP results. This means that their state scores look worse – in Massachusetts, for example, 47 percent of students were proficient on the state exams – but they may be a better reflection of reality.

Student Promotions

State assessments serve another purpose in the "gateway" grades of 3, 5, and 8. End of grade tests in those grades are the key to promotions. For most school districts, a student must be within one standard error of measurement of Achievement Level III (proficient) in order to advance to the next grade. Since 2001-02, fewer students have been required to repeat a grade each year. Not only have the percentage of students achieving Level III and above increased, but the percentage of students who do not pass the tests and yet are promoted to the next grade has increased as well. In 2001-02, 57 percent of 3rd graders who did not meet reading or math standards were promoted anyway. By 2005-06, that number had climbed to 87 percent, an increase of 30 percentage points. The most common cause of the promotions was the decision of a principal to override the test score.

The Bottom Line

The information provided by North Carolina standardized tests is limited and can be misleading. At best, it gives only an overall picture of how student knowledge has improved over time. At worst, it contributes to a false sense of complacency. The student proficiency benchmark is not tied to the demands of today’s society or workforce. Other states have made tough decisions to sacrifice good marks for a more rigorous (and accurate) measure of student proficiency. Without such a measure, we cannot tell whether we have truly done the job we set out to do 15 years ago. Our job is not to prepare citizens for standards of the past, but for challenges of the future.

 

Sources: All data and information about the state’s standardized tests and the state’s performance on NAEP tests can be found on the Department of Public Instruction’s website, www.ncpublicschools.org.