2017 NAEP results present mixed bag for North Carolina students

Today, the US Department of Education published results from the 2017 National Assessment of Educational Performance (NAEP). Known as “The Nation’s Report Card,” NAEP is administered every two years to a representative sample of students in each state. NAEP results are help policymakers identify trends in student performance and offer the ability to make comparisons of school performance across states.

While very useful, one must be extremely careful before using NAEP test results to draw broad policy conclusions. A few things to keep in mind:

• NAEP assesses a different cohort of students each year, so the 4th grade cohort in 2003 might differ from the 4th grade cohort tested in 2017. Even though both cohorts were representative of the student body at that time, the share of students from low-income families or for whom English is not their first language might have changed.
• Related to the point above, changes in trends for all students can mask trends for different student cohorts. In many states, overall performance has plateaued even though performance of white students and students of color has improved. Seems paradoxical. But if the share of lower-performing subgroups is becoming a greater share of the total student population, total performance may look flat even though achievement levels of all subgroups are improving.
• Remember that correlation is not causation. Just because some policy happened (or didn’t happen) during the time that NAEP scores increased (or decreased) does not necessarily mean that policy is good (or bad).
• Beware those who claim that low proficiency levels on NAEP are evidence that our schools are “broken.” Scoring “proficient” on NAEP is not synonymous with performing “at grade level” – it is a much higher standard.
• Cross-state comparisons of student achievement levels should take into account demographic differences. While North Carolina should aspire to Massachusetts-levels of student performance, North Carolina’s higher share of students of color, English language learners, and students from low-income families make it more difficult for North Carolina to reach the same level of achievement.

With those caveats in place, below are a few observations from the 2017 results:

4^th grade math:

• North Carolina one of just 10 states to experience a significant decrease in 4^th grade math scores from 2015.
• Average score of 241 means we’re no longer above-average in 4^th grade math.
• Performance for both white students and Black students fell in 2017, but performance of Black students fell further, widening the Black-white achievement gap in 4^th grade math.
• Only 16 states have a wider 2017 Black-white achievement gap in 4^th grade math than NC.
• North Carolina one of 21 states with widening Black-white achievement gap in 4^th grade math since 2011.
• Gap between students qualifying for free or reduced lunch (FRL) and non-FRL students actually narrowed in 2017 for 4^th grade math, and is now significantly narrower than the national average.
• 40 states now have a wider income-based achievement gap than NC when it comes to 4^th grade math.
• NC is one of just 8 states to have narrowed its income-based achievement gap in 4^th grade math since 2011, but it has done so through falling scores for both students who qualify for FRL and those who don’t.

8^th grade math:

• North Carolina among 48 other states/jurisdictions with no significant change in 8^th grade math scores from 2015.
• Average score of 282 puts NC exactly at the US average for 8^th grade math.
• Performance for white students in 8^th grade math improved in 2017, but performance of Black students fell, widening the Black-white achievement gap.
• Only 6 states have a wider 2017 Black-white achievement gap in 8^th grade math than NC.
• North Carolina one of 26 states with widening Black-white achievement gap in 8^th grade math since 2011
• 8^th grade math achievement gap between FRL and non-FRL students increased slightly in 2017 and is somewhat larger than the national average.
• Just 6 states have a wider 2017 income-based achievement gap in 8^th grade math than NC.
• NC is one of 40 states to have experienced a widening of its income-based achievement gap in 8^th grade math since 2011, with scores falling for both students who qualify for FRL and those who don’t.

4^th grade reading:

• NC’s 2-point decrease in 4^th grade reading achievement places it among 43 other states/jurisdictions with no statistically significant change from 2015.
• Average score of 224 puts NC above (but not significantly different from) the US average for 4^th grade reading.
• Performance for both white students and Black students fell in 2017, but performance of Black students fell further, widening the Black-white achievement gap in 4th grade reading.
• 30 states have a wider 2017 Black-white achievement gap in 4^th grade reading than NC.
• Researchers and policymakers should use caution when comparing NC’s long-term trends in 4^th grade reading, as the retention of struggling 3^rd grade readers via NC’s Read to Achieve policy has only been reflected in NAEP results since 2015.
• 4^th grade reading achievement gap between FRL and non-FRL students narrowed slightly in 2017 and is somewhat larger than the national average, but it has done so through falling scores for both students who qualify for FRL and those who don’t.
• 33 states have a wider 2017 income-based achievement gap in 4^th grade reading than NC, though NC’s income-based achievement gap does not significantly differ from the US average.

8^th grade reading:

• North Carolina among 41 other states/jurisdictions with no significant change in 8^th grade reading scores from 2015.
• Average score of 265 puts NC above (but not significantly different from) the US average for 8^th grade reading.
• 8^th grade reading scores for both white and Black students improved in 2017, with the greater performance increase of Black students narrowing the Black-white achievement gap.
• 24 states have a wider 2017 Black-white achievement gap in 8^th grade reading than NC.
• NC’s Black-white achievement gap in 8^th grade reading remained unchanged since 2011, compared to a majority of states (27) that experienced a widening of their Black-white achievement gap.
• 8^th grade reading achievement gap between FRL and non-FRL students narrowed in 2017 and is now in line with the national average.
• Just 15 states have a wider 2017 income-based achievement gap in 8^th grade reading than NC.
• NC is one of 31 states to have experienced a widening of its income-based achievement gap in 8^th grade reading since 2011.

Overall, these results are neither dramatically better nor worse than national trends. However, the results are still disappointing. North Carolina’s education system is not producing broad-based student achievement gains across all subjects.

The results also point to a number of questions worth further analysis:

• Has North Carolina’s focus on third grade literacy harmed instruction in math?
• Except for 8th grade reading, performance is trending down for Black and Hispanic students, and those who qualify for FRL. These results should inspire policymakers to focus on the needs of children from these communities.
• Performance improved across nearly all student groups in 8th grade reading. This is incredibly surprising and deserves further investigation.
• North Carolina’s achievement gaps remain persistently large (and shameful). There’s no subject for which North Carolina’s achievement gaps are significantly narrower than the national average. The 2017 NAEP results will hopefully spur the General Assembly to focus on addressing these gaps.