Lots of people criticize standardized tests for focusing too much on math and reading, claiming that tests end up forcing schools to narrow the curriculum to those two subjects.

Curriculum narrowing may occur from time to time, but it is far from necessary. Indeed, it’s counterproductive for a school just to focus on a separate subject called “reading” at the expense of other background knowledge in many other subjects. After all, most reading tests require a student to read various essays, and essays have to be about something.

Banal, I know, but this is the key point: students who have a lot of background knowledge about the essay’s subject, whatever that might be, will do better at “reading.” Past the point of decoding letters, reading is inseparably tied to background knowledge.

As cognitive scientist Dan Willingham says:

Remarkably, if you take kids who score poorly on a reading test and ask them to read on a topic they know something about (baseball, say, or dinosaurs) all of a sudden their comprehension is terrific—better than kids who score well on reading tests but who don’t know a lot about baseball or dinosaurs.

In other words, kids who score well on reading tests are not really kids with good “reading skills.” The kids who score well on reading tests are ones who know a lot about the world—they have a lot of prior knowledge about a wide range of things–and so that whatever they are asked to read about on the test, they likely know something about it.

Thus, as E.D. Hirsch wrote, state reading tests should be explicitly tied to material that students have been learning in their particular grade. The problem is that the tests usually are “random,” and “not aligned with explicit grade-by-grade content standards.” Instead, “children are asked to read and then answer multiple-choice questions about such topics as taking a hike in the Appalachians even though they’ve never left the sidewalks of New York, nor studied the Appalachians in school.”

Hirsch notes that if “reading passages on each test were culled from each grade’s specific curricular content in literature, science, history, geography and the arts, the tests would exhibit what researchers call ‘consequential validity’ — meaning that the tests would actually help improve education.”

For example, if kids in 4th grade have been learning about how the telegraph and the pony express affected westward expansion in the United States (that’s content standard H.6.4.16 in Arkansas), then a good reading test might have an essay on that very topic, or any of the many other topics studied in the 4th grade curriculum.

If that were the case, not only would reading tests be more fair — as they would cover material that everyone had been taught in school — they would actually reinforce all of the other subjects, rather than narrowing the curriculum. The best way to prepare for a good reading test would be simply to teach all the things that kids need to learn about history and science and the arts.

The question I’ll address in the next post is whether Arkansas Benchmark reading tests come anywhere near this ideal.

On the national front, the UCLA Civil Rights Project released a report today that claims to show that charter schools are more segregated than traditional public schools. The report finds:

…that charter schools, particularly those in the western United States are havens for white re-segregation from public schools; requirements for providing essential equity data to the federal government go unmet across the nation; and magnet schools are overlooked, in spite of showing greater levels of integration and academic achievement than charters.

It looks like, based on a quick pass through the report, their main finding is based on demographic comparisons  between charter schools and traditional public schools at the state level. This method of comparison likely leads to inaccurate conclusions due to the fact that charter schools are overwhelmingly an urban phenomenon. The correct comparison is between charters and the demographics of their immediate geographic area. We have discussed this topic as it relates to Little Rock at length here.

The Economist’s take on this report is concise, to-the-point, and spot on.

In plain English, there are a lot of black kids in charter schools. This is because charter schools tend to get set up in neighbourhoods where the public schools are terrible, such as south-eastern Washington DC or the rougher parts of New Orleans. These neighbourhoods are disproportionately African-American. Charter schools are popular with poor black parents because their other choices are so awful. There are very few charter schools in rich white suburbs with nice public schools, because there is no call for them.

The important question about charter schools is: do they give kids a better education than they would otherwise have received? The answer is yes. Nothing else matters.

You can find information specific to Arkansas from the Civil Rights Project report here.

ADDITION: I had some additional thoughts that I wanted to add to this post, so , here goes.

First, to drive home the inappropriate nature of the report’s method of comparison, I would like to make an reductio ad absurdum argument. In any given state it is necessarily true that some traditional schools will have a higher percentage of black students than the state average and some will be lower . Take for example the West Memphis and the Little Rock here in Arkansas. Both have a very high percentage of black students compared to the state average, 80% and 68% respectively compared to the state average of 21%.  Would the authors of this report advocate regulating these districts, or perhaps, dictating where people live within the state so as to equalize the demographic makeup of all schools.  And, what happens when they realize that Arkansas is whiter than Mississippi and blacker than Oklahoma?

Second, I find it curious that opponents of choice see black kids choosing charter schools and blame the charters.  While it seems more likely to me that this represents a population of students being severely underserved by their traditional public schools who are making a choice to leave. Shouldn’t we be more concerned about those traditional districts that are failing to meet their student’s needs?

A little closer to home, Cynthia Howell of the Dem Gaz has an article today providing blow by blow coverage of the latest from the North Little Rock School District desegregation hearings. Some of the more interesting discussion focused on discipline.  Here are a couple of excerpts.

John Walker quizzed Bobby Acklin, assistant superintendent for desegregation,and Francical Jackson, director of student affairs, about why black students, particularly boys, are disciplined at greater rates than their white classmates and how students and parents are to know that the district will pay for field trips for students from poor families if those messages aren’t in writing.

**********

The district’s enrollment for the past several years has been 59 percent black. In 2006-07, 83 percent of 3,709 suspensions in the district were to black students. And 91 percent of the 1,079 out-of-school suspensions were given to black students.

**********

Jackson said she has not found racial bias in the discipline recommendations from schools but, in response to Walker, she said she has not prepared any written analyses on that matter.

She said that black students “misbehave more often than whites,” and that playful roughhousing among black students can be misinterpreted by district employees as fighting.

Walker noted that the district at times assigns students with behavior problems to privately operated day treatment programs at a cost of $232,000 per semester. He questioned whether Jackson had determined whether those students, most of whom are black, perform better upon their return to the North Little Rock schools.

“The culture in North Little Rock is to not put anything in writing and you don’t do anything to remediate students,” Walker told Jackson at one point.

“We may not have it in writing but we are making progress,” Jackson responded, noting that the district has increased the number of alternative education services for students who are not succeeding in the regular classrooms.

Education Secretary Arne Duncan sent a letter to education stakeholders last week touting the Presidents plans for the coming year.  I am encouraged to see continued support, at least lip service, for reform, but I am also a little disconcerted by the administration’s penchant for supporting programs based on political popularity/expediency rather than evidence and results.

In his letter, Secretary Duncan emphasizes two programs which have a rather shaky track record for results.  The first is increased student aid for higher ed.  While it is true that the cost of tuition has been rising quickly over the past decade, it is certainly not clear that more student aid will make college more affordable.  There is some evidence that universities soak up any increases in student aid through tuition increases, leaving the affordability of a college degree relatively unchanged. And, there is further evidence that student aid does not help low-income families to the extent that we would like. Navigating the complicated application process serves as a barrier to many of these needy students. Reforming the student loan process and simplifying the aid application could go a long way to mitigating the system’s deficiencies, but more money will mostly help college’s bottom lines.

It is also important to note that the returns to a college degree have also been rising at a rapid rate over the past decade. In other words the economy is placing greater and greater value on a college degree, and this has a direct positive  impact on lifetime earnings. The promise of greater lifetime earnings should not only induce more people to attend college, but should also increase their ability to pay for college by providing them with more income to repay student loans.

Evidence suggests that college readiness is a much larger problem than college affordability. In 2001 only about 32% of the nations high school students “leave high school qualified to attend four-year colleges.”  This report also found that nearly all those who were qualified to attend a four-year institution did. Given this evidence, shouldn’t we be focusing our money and attention on college readiness rather than affordability?

Mr. Duncan also touts the administration’s commitment to early learning (preschool) programs, the most notable of which is Head Start. But, there is a growing body of evidence showing that early childhood education programs produce marginal results that fade out alarmingly fast (here is a review of the latest study of Head Start).  I am not making the claim that early childhood education is not important, but I am questioning whether we are getting the best bang for our education buck.  Head Start gets an increase in funding in spite of poor results while programs with a demonstrated record of success like the D.C. Opportunity Scholarship Program get cut.

It’s time President Obama lived up to his claim of evidence based governance. Both the Race to the Top competition and the reauthorization of NCLB (now ESEA) will be great tests of the administration on this front.  We will be watching with our fingers crossed, but not holding our breath.

UPDATE: Rick Hess and Andy Rotherham have some interesting things to say about the President’s reauthorization strategy. Here is another article about ESEA from EdWeek.

Our friend Jay Greene has a great post over at his blog today.  It seems the NEA is making the claim that , as their headline puts it, “Teachers Take ‘Pay Cut’ as Inflation Outpaces Salaries.  Average teachers’ salaries declined over the past decade.” Interestingly their own data do not support this claim.

Jay takes a look at their report and finds that:

In Table C-14 “Percentage Change in Average Salaries of Public School Teachers 1998-99 to 2008-09 (Constant $)” we see that salaries increased by 3.4% nationwide over the last decade after adjusting for inflation.  The increase in average salary outpaced inflation in 36 states, which is very different from the claim that  ”Inflation over the past decade has outpaced teachers’ salaries in every single state across the country…”  Check for yourself, the table is on p. 20 of the report, which is p. 38 of the pdf.

I can’t find a single table or figure in the report that would justify the headline and claims in the press release.  But when the Ministry of Truth speaks who are you supposed to believe — them or your lying eyes?

Ministry of Truth indeed.

P.S. Check out this video from NEA president Dennis Van Roekal. On a technical note, it looks like they made this mistake by comparing the average salary growth in constant dollars (inflation adjusted dollars) to the inflation rate.

UPDATE:  So, it looks like the NEA has realized their error and have yanked the video I linked to and altered their report. Jay has a follow-up post here.  Here is the video I tried to link to just in case you missed it:

President Obama delivers his State of the Union address tonight, and new education initiatives are likely to be featured prominently in the speech. Today’s Washington Post has an article discussing some of the details that have leaked.

Administration officials are saying that the federal education budget may increase by as much as $4 billion or 6.2%. The bulk of the new funding would split between Race to the Top and No Child Left Behind. It is also expected that in addition to the funding increase, President Obama will announce that his administration will be trimming some of the excess fat at the Department of Ed. The Post reports that:

Obama is expected to propose the consolidation of federal education programs. The budget he submits next week will collapse 38 K-12 programs into 11 and eliminate six programs, senior White House aides said.

The next few years could be very exciting for education reformers. The Race to the Top competition has already spurred a number of promising reforms in many states. Choice has been expanded and new data and accountability systems have increased the focus on education outcomes. This new wave of spending could result in even greater gains. As the Post put it:

Obama has signaled that he wants tougher academic standards but more flexibility for schools to reach them. His administration has pushed for innovations such as public charter schools, teacher performance pay and stronger data systems to track student growth from pre-kindergarten all the way to college.

However, my optimism is tempered by the realization that there are large interests arrayed against any real and lasting reform effort.  The teachers unions, in particular, spend a lot of time, effort, and money impeding any real change, and they have proven to be quite skilled at this task.

I hope the administration has the resolve to stick to its reform agenda. New money should be targeted to specific reforms rather than simply spread around existing programs. States and districts should be held accountable for specific education outcomes, but be given the freedom to implement local strategies to meet these goals.

I look forward to tonight’s speech, but will be watching closely over the coming months.  Hopefully this will mark the beginning of real change, and not simply be another well delivered speech full of empty ideas and wishful thinking.

Here at Mid-Riffs we are no fans of education consultants and their edubabble. Especially hucksters who are willing to use their academic credentials to hustle taxpayers for money. These people charge exorbitant sums for their services while doing very little to improve actual classroom learning. So we were a bit concerned to learn that the Little Rock School District recently employed education hacks bandits consultants Picus and Odden (of Arkansas educational adequacy case fame) to compile a strategic plan for the district. A recently released draft of their report has been met with the approval of the community group the district formed to review the report and something bordering on revulsion from the Dem Gaz editorial staff. You can read the Op-Ed here if you have a subscription. Here are a few exerpts:

Somebody needs to translate this document into plain English, the way George Orwell once translated the political idioms and idiocies of his time in a brief but incisive essay that has become a classic, “Politics and the English Language.” Now the same service should be performed for education in ours. And in much the same way-first by repeating its pretentious newspeak, then saying what it all really means.

*********

Eye-rolling would be a more natural reaction to this highflown piffle. Do not be misled by the terms “minority” and “majority” for black and white students even if black students may now be in the majority in the school district. These euphemisms are used in place of black and white, which are considered dirty words in educanto and so must be censored. In a pinch, to avoid complete confusion, multisyllabic terms like African American and Caucasian may be used instead.

*********

Never refer to how well or how poorly students perform on tests but the “student performance situation.” Much as one would refer to war as the “human conflict situation.” Speak of “achievement gap” rather than how we have failed to educate our children, especially our black children. That way, somebody might have to take responsibility.

*********

The achievement gap between Caucasian and African American students in mathematics at the advanced levels has grown from about 20 to about 40 percentile points.

Translation: After all the millions of dollars We the taxpaying People have spent, after all the decades of wasteful litigation that have benefited only the lawyers involved, and after all the gibberish our “educators” have talked, our public schools are getting worse, not better.

*********

But to speak like this is to violate all the rules of educanto, a language designed not to transmit meaning but obscure it. Feel free to go through this report and pick out your own unfavorite part to translate into English, or attempt to. You’re fully entitled to do so. This study in educanto cost you, the innocent taxpayer, $200,760.

Don’t get me wrong, creating a strategic plan can be a useful exercise; however, the kind of cut-and-paste consultant speak  that Picus and Odden peddle is not worth the paper it’s printed on. To date Arkansans have given the dynamic duo (don’t ask me which one is Robin) $1 million plus for the privilege of listening to their drivel. Their latest iteration of gasbaggery cost taxpayers about $200k.

For a detailed critique of Picus and Odden’s work you can read Stanford economist Eric Hanushek’s particularly scathing piece which appeared in Education Next in the Winter ‘07 issue. You can find a copy at his website here. Here are a few highlights:

So similar are the studies that at times it seems the copy function of the Microsoft word processor deserves to be listed among the authors.

*********

Picus and Odden … provide “scientific evidence” to support the claim that a specific set of policies can shift average student performance upward by three to six standard deviations, an extraordinary gain … If one then assumes a cumulative impact from giving students not just a single application but continuing treatment through grade 12, the gains reach astronomical proportions, somewhere in the range of 23 to 57 standard deviations … This, of course, is the stuff of science fiction novels, not research-based school policies.

*********

Why do different states and organizations pay ever-increasing amounts to see this research review when Google would bring up the most recent version immediately and without expense? The answer is simple. Clients want a bottom-line statement … Few people care about the “studies” on which consultants base their reports, or even their validity, because nobody really expects schools to implement these specific programs … Clients simply want a requisite amount of scientific aura around the number that will become the rallying flag for political and legal actions.

So why do people continue to pay for this ridiculous junk?  Because they lack either the ability or the temerity to lead.

It’s time our education leaders stopped passing the buck and started doing the job we pay them to do. And, if they must go to an oracle to know they need to do a much better job educating the district’s students, then I strongly suggest it’s time the people of Little Rock begin looking for new education leaders.

There were some positive developments at the State Board meeting in Little Rock yesterday concerning the proposed Little Rock Urban Collegiate Public Charter School for Young Men.  Arkansas Education Commissioner Tom Kimbrell deserves a lot of credit for standing up for common sense.  He argued against the restrictions the LRSD has been asking for and, as reported by the Dem-Gaz,  “said the department’s current process for reviewing applications and existing charter schools is inadequate, and he announced plans to establish a charter school review council that will be made up of Education Department staff members, including himself and assistant commissioners.”  Naturally, Chris Heller is upset.  Here are some excerpts from the article:

Restrictions lifted on school for boys

Charter won’t fill seats by scores, income, as favored by LR district

LITTLE ROCK — An independently run charter school for boys that is set to open in August in Little Rock no longer has to meet state-imposed restrictions on family income and student achievement levels.

The Arkansas Board of Education voted 5-0 Tuesday to remove the conditions it approved last month requiring the new Little Rock Urban Collegiate Public Charter School for Young Men to serve mostly poor and low-achieving students.

The Little Rock School District had urged the state board to set the enrollment restrictions.

An attorney for the district said Tuesday that removal of the restrictions could be a setback in ongoing negotiations between the district and the state on how to phase out nearly $70 million in state desegregation aid to the three Pulaski County school districts.

The state board lifted its restrictions at a meeting in which it approved a new conversion charter school in Forrest City, denied two proposed conversion charter schools in the Pulaski County Special School District and delayed a decision on a conversion charter program at Little Rock’s Cloverdale Middle School until that proposal is rewritten.

Arkansas Education Commissioner Tom Kimbrell told the board that enrollment limits on the Urban Collegiate school – which is designed to serve up to 696 boys in kindergarten through eighth grades – were counterproductive, a logistical nightmare to carry out and not good for children.

Kimbrell, who became commissioner in September, recommended removing the restrictions. He said he should have done so at last month’s meeting but hesitated because he was new to the job.

“From a practical and professional perspective, it is my opinion that putting these conditions on the charter school won’t lead to anybody’s success,” Kimbrell told the board.

He said the department’s current process for reviewing applications and existing charter schools is inadequate, and he announced plans to establish a charter school review council that will be made up of Education Department staff members, including himself and assistant commissioners.

The council will evaluate in a more systematic manner than now exists the applications for both independently operated and school district run charter schools.

The council also would immediately evaluate already operating charter schools on an annual basis to determine whether they are conforming to the terms of their charters, which are five-year contracts with the state Education Board. Those reviews would focus on enrollment in the schools, their finances and any new programs or decisions that are under way.

“We would come to you about applications and existing charters with recommendations for modifications, revocations or continuations of those charters,” Kimbrell said about the council.

*****

Minutes from the state Education Board’s Dec. 19 meeting said that the board required at least 80 percent of the school’s enrollment each year to be from low-income families or below-proficient on the state Benchmark Exams.

But Kimbrell said in an interview that a review of a tape of the board meeting indicated it had to be both – not one or the other.

The tape showed that the board wanted at least 80 percent of the students to be from low-income families and below-proficient on state exams, a tougher requirement.

Kimbrell told the state board that the school planners would not have access to records showing whether school applicants qualify for free- and reduced-price school meals, which is an indicator of low family income, until after July 1. Nor would they have access to Benchmark results until late summer.

In both cases, the information wouldn’t be available until well after students would enroll for the August start of classes, he said.

Additionally, he said, some students from home schools and private schools do not have state test records and, as a result, can’t be included in the 80 percent. And he said that some families decline to fill out an application for subsidized school meals for their children even though they would qualify for the benefit.

Kimbrell also said a school populated with virtually all low-achieving students would leave the school devoid of student role models and peer mentors. As well, the school would become a permanent fixture on the state’s list of schools in need of improvement because of low test scores

There will be 12 open-enrollment charter schools in Pulaski County next year.

Little Rock School District officials have objected to most of them, in part because they say such charter schools hinder efforts to attract students to the district’s magnet schools, which are a component of federal court approved plans to racially desegregate schools in Pulaski County.

Charter schools also tend to attract higher-achieving students, district representatives argued, leaving the traditional district with a higher proportion of low-achieving students and fewer financial resources to serve them.

*****

Chris Heller, an attorney for the Little Rock district, tried to address the state Education Board on Tuesday but was not recognized.

The district is in negotiations with the state over the possible phaseout of state desegregation money. Unconditional approval of charter schools by the state board has been an issue in those negotiations.

“Up until today it appeared we were making progress and some of our concerns were being heard,” Heller said in an interview about the talks. “This is a significant reversal.”

*****

Arkansas’ application for Race to the Top funds can now be found here on the ADE website.  I’m sure composing it was a long and tedious process for those involved, and I’m impressed that there are people out there who can navigate through such intense bureaucratic jargon.

We haven’t fully digested it yet.  At first glance, it reads more like a brag sheet of what Arkansas has done for the past decade.  That’s to be expected.  Unfortunately, the document seems to be pretty thin when it comes to demonstrating any new specific innovations or any new legislation that has been forged that will lead to future reforms.

One item that is worth mentioning shows up in the section that defends Arkansas’ charter school cap (charter caps are discouraged by RTTT goals).  Arkansas’ application states that:

As of the date of this submission, no applicant for an Open-enrollment charter has had its application denied due to the existing “cap” of 24. While no applicant State or State agency can guarantee the future adoption of any type of legislation, we anticipate that, as has occurred in the past, when the number of high-performing Open-enrollment charters approaches the current “cap” the question of increasing or eliminating the cap to accommodate additional high-performing Charter will be given serious consideration.

This might be news to State Board members.  As we’ve mentioned before, State Board member Brenda Gullett told the press that the charter cap was was the basis for how selective she is when considering applicants.

I also thought it was entertaining when, in response to a question on the RTTT application that asked specifically for:

“The number of charter schools allowed under State law and the percentage this represents of the total number of schools in the State”

Arkansas’ application answers:

“24 open-enrollment charters (8 percent of the total number of LEAs – 244).”

Of course,  the number of total public schools is more than 4 times the number of LEAs.   The actual correct answer is around 2.2%.

And, since we’re getting technical, 24 is 10 percent of 244, not 8.

Tomorrow (January 19th) is the deadline for phase-1 Race to the Top applications.  As our readers know, the so-called Race to the Top program is a federal program that aims to disperse around $4 billion in education funding to states.  We describe the program here, and we’ve been worried for months about Arkansas’ ability to compete against other states that seem to be more enthusiastic and knowledgeable about education reform.  While some states have been busy tuning-up their engines,  it seems that Arkansas is still fumbling around in the trunk looking for a pair of jumper-cables.

We’ll review Arkansas’ application once it becomes public.  In the meantime, the group Democrats for Education Reform (DFER) has been keeping tabs on what other states are doing.  You can check out DFER’s own blog here.  They’re doing a great job of monitoring RTTP activity.  Here’s a sample from their January 18 post:

After months of legislative work around the country, states are scrambling to meet the Jan. 19th deadline for legislative and policy changes as part of the “Race To The Top.” What gets negotiated and signed in the next 12 days will largely determine which states cross the finish line with the kind of gusto it is going to take to win some serious federal dinero for public schools.

California – Lawmakers there yesterday shifted power away from bureaucrats and foot-dragging school boards and placed it in the hands of parents, passing new “Race To The Top” legislation which allows parents from failing schools to yank their kids out and/or vote as parents to take drastic measures at their local zoned schools. Kudos to our friends, Ben Austin, at Parent Revolution, and Sen. Gloria Romero (our September education reformer of the month – http://www.actblue.com/page/dferseptember09) for their inspiring leadership on this one. The Governator will quickly sign the bills into law to make the Jan. 19th RTTT deadline.

Massachusetts — Early this morning, the House passed RTTT legislation that will allow Boston school officials to shutter failing schools and convert them to new charter schools, without having to allow the teachers union to decide which schools could and couldn’t be closed. Boston Mayor Tom Menino told the Boston Globe “this bill was made in Heaven.” (Which probably means we better read the fine print!) The Senate previously passed its version of the RTTT bills and conferencing on the two bills was expected to start today. Legislators were on track to pass the bills into law by Jan. 14^th, so that the state would have ample time to prepare its application by the Jan. 19^th deadline.

New York — Regular DFER readers know we have been frustrated for much of the year by the Empire State’s unwillingness to take the reform competition seriously – particularly at a time when the state’s coffers are beyond bare. But things have been moving quickly in the last month, starting with bold action by the Board of Regents in passing a series of K-12 reform recommendations. The Senate and Assembly are looking at what they can do to allow New York to apply, and earlier today Gov. David Paterson introduced a program bill that would eliminate the cap on charter schools, kill a law that bans using student performance in teacher tenure decisions, and allow the Regents to take control of persistently low-performing schools, among other things. The next week will be very interesting, as New York decides whether it wants to try to take the necessary steps to become a national leader. Stay tuned.

Tennessee — Gov. Phil Bredesen is calling for a special session of the legislature next week in order to change several laws in time for the Jan. 19^th application deadline. “The (the feds) don’t want any promises for the future, they want things in law,” Bredesen told reporters. One change high on Bredesen’s wish-list: a change in state law to allow the use of student achievement in teacher evaluations.

Education Week recently released their annual Quality Counts Report. You can see their nifty graphics here and some more detailed information here. Arkansas received a B- this year (it’s worth mentioning that we received the same grade last year as well). To create a report like this it is necessary to define what quality in education is. It’s obvious from the data items included in this report that, for the Ed Week editors, quality is almost entirely about inputs. While it is true that better ingredients tend to make a better product, good ingredients are not sufficient. You could give a bad cook the highest quality ingredients but the end product would still be inedible.  Further, education research has shown time and again that the inputs we think matter are at best loosely correlated with achievement. So why focus on inputs?… because it is easy. While I think the right inputs are important, I would much rather see that Arkansas’ students are achieving at the highest level regardless of what input mix got us there.

And what does our achievement grade look like? Ed Week gives us a D where it counts!

Stuart Buck wrote a great critique of the Quality Counts Report over at Jay Greene’s blog last year. You can see it here. But for those of you who are lazy here is a little taste.

…imagine a state that managed to produce A-level achievement even though its population was poor and disadvantaged (and thus got a lower grade on the “Chances for Success” measure).  Under any rational grading system, we should give that state the highest possible rating.  But the Quality Counts method would actually downgrade the state for having too many poor children.  By the same token, Quality Counts would upgrade a poor-achieving state that happened to have a privileged and rich student population, even though that state’s education system would obviously be far more incompetent and inefficient.  If anything, the “Chances for Success” ranking should be counted inversely as compared to all the other measures of a state’s education system.

Margaret Raymond from CREDO writes on the Ed Next blog that the variation in the report’s Chance-for-Success Index can almost entirely be explained by state demographic changes rather than changes in education quality. Here is the money quote.

Until the measures that are incorporated into the Quality Counts ratings are more clearly tied to education outcomes, we are likely to see continued shifts in rankings that bear little resemblance to actual changes in education quality.

Margaret and her CREDO team present revised estimates of the index here. Arkansas moves up ten spots, from 45th to 35th, but is still in the bottom third of the ranking.

Anyway you look at it, Arkansas has a lot of work to do to improve the quality of its schools.