I raised that question in an earlier post, based on the fact that the raw scores needed to be advanced or proficient have dropped substantially over the past few years.

Now there is a new study from the National Center for Education Statistics — a government agency — comparing how students did on their own state tests to how they did on National Assessment of Educational Progress (NAEP) tests.

Lo and behold, out of the 24 states nationwide whose tests were comparable to NAEP, Arkansas was one of the few states wherein children performed higher on state tests in 2007 than in 2005, but their performance was not higher by the NAEP standard.  Indeed, if you look at Table 24, Arkansas is the only state in the entire study that saw all four categories (4th and 8th grade reading and math) make suspiciously large improvements on state tests between 2005 and 2007.

For specifics, see pages 41-46. Arkansas 4th grade reading scores had a 4.6 percentage point discrepancy in 2007 compared to 2005 (that is, 4.6% more students scored “proficient” in 2007 than they would have if the state tests remained more consistent with NAEP). Arkansas 4th grade math scores had a 9.5 percentage point discrepancy. Arkansas 8th grade reading scores had a 5.8 percentage point discrepancy. And Arkansas 8th grade math scores had a startling 12.1 percentage point discrepancy — more than any other state in the study.

The logical possibilities are these:

1) NAEP got harder between 2005 and 2007. This is not the case.

2) Students in Arkansas spend an inordinate amount of time practicing for Benchmark tests, but whatever skills and concepts they learn don’t carry over to a slightly different math and reading test. Possible, but not likely either. 8th grade math should be similar no matter who is testing it, and the U.S. Department of Education thought Arkansas’ tests were indeed comparable to NAEP. And in any event, there’s no evidence that Arkansas students are spending more time in state-specific test prep than anyone in any other state.

3) Arkansas students are more motivated to do well on high-stakes Benchmark exams, but brush off the low-stakes NAEP test. The problem is that there is no evidence for this theory. From the student’s perspective, neither of these tests are high-stakes in any way whatsoever. Moreover, the “motivation” theory doesn’t explain why Arkansas tests would see an increasing discrepancy with NAEP as time passes (rather than having the same discrepancy all along), nor does it explain why Arkansas would have more discrepancies than any other state (whose students are presumably subject to the same differential motivations).

4) State tests were dumbed down.

Unfortunately, the last possibility is looking the most likely at this point. Recall that the number of raw points Arkansas required to be “proficient” in 8th grade math was 39 points in 2006, but this has now dropped to a mere 28 points out of 80, which seems awfully low to be called “proficient.” And then recall that of all the states in the study, Arkansas 8th graders in Arkansas had the greatest discrepancy between state tests and NAEP — 12 percentage points. This is a coincidence?

So far, the Arkansas Department of Education has had no explanation for these results. A recent Democrat-Gazette story said:

Julie Johnson Thompson, a spokesman for the Arkansas Department of Education, attributed that to the fact that Arkansas results on the National Assessment of Educational Progress have been flat in recent years while the results improved on the state exams.

But Thompson here is just restating the problem in circular fashion, not providing an explanation of anything.

The Arkansas Department of Education should explain these discrepancies in a clear and straightforward manner that the public can understand. Otherwise, it looks like Arkansas’ academic “progress” in recent years was questionable.