There has been a great deal of media attention over the past several months focused on the question of whether Universities should be paying salaries or stipends to student-athletes. Some, including Jay Paterno (the son of Penn State football icon Joe Paterno) have opined that the players are already getting a great deal that other students around campus “would gladly take”. Many others argue that top-level college sports is indeed big business, but relatively little money flows down to the athletes. This question has recently become even more interesting as big-name football programs like USC and Ohio State have been sanctioned for payments to players or profit-seeking activities by the players that are against NCAA regulations.
So, what’s the truth? A basketball-buddy of mine, who is himself an aficionado of college sports, Flint Harris added his $.02 to this debate recently on his Holy Turf web site (quite a good site — check it out). Flint’s article sheds useful light on this question by simply providing the data regarding how much money athletes actually receive (or can receive within the existing rule structure) during their college career. In my view, any discussion is more useful with real data as the foundation. It turns out, according to Flint, that players can receive approximately $17K each year in living, eating, and clothing expenses (this does not include scholarship money that goes toward the cost of tuition and books and fees).
This information is helpful, but does not by itself settle the question of whether players should be paid additional dollars for their efforts. But it does allow us to dismiss the exaggerated claims that we simply must pay stipends otherwise these players can’t afford to buy a pizza or go on a date. This seems not to be the case. However, it certainly seems reasonable that college athletes, like most college students, may mismanage their funds and at times find themselves out of pocket money. But this situation, in and of itself, is not a justification for paying players stipends.
There is a widespread belief in education circles that the grades that students “earn” in college (and also in graduate school) may not represent the level of academic achievement that they once did. Observers of K12 education in Arkansas have been paying attention to this issue for a while. Indeed, our state has published a few “grade inflation” reports (first in 2005) in which researchers have identified schools where many students would earn A’s in their coursework but would nonetheless earn low marks on the state’s End-of-Course (EOC) exams in the same subject. The most recent “grade-inflation” report identified many Arkansas schools where the students’ grades in Geometry and Algebra simply did not match up to their performance on the standardized assessments.
Source: Stuart Rojstaczer and Christopher Healy
The Economix blog on theNew York Times web site today highlights this problem in higher education. It turns out that, over time, students in college are far more likely to earn A’s, or at least professors are far more likely to give A’s, than in previous years. The blog is highlighting the work of two authors — Stuart Rojstaczer and Christopher Healy — who have been studying the concept of grade inflation for years.
It seems clear, from the data provided by the authors and illustrated in some neat graphs over at the Economix site, that it is becoming easier to get an A. Should this bother us? Well, according to the authors, as grading standards may become even looser in the coming years, it will become increasingly difficult for graduate schools and employers to distinguish between excellent, good and mediocre students. This seems to be problematic.
More disturbing, they argue, are the potential deleterious effects on educational outcomes from these declining standards and expectations.
“When college students perceive that the average grade in a class will be an A, they do not try to excel. It is likely that the decline in student study hours, student engagement, and literacy are partly the result of diminished academic expectations.”
It’s too bad that the obviously-flawed Western Michigan study targeting KIPP received so much media attention. Author Gary Miron (Fellow at NEPC) was covered in Education Week, the New York Times, and numerous other lower-profile outlets throughout cyberspace. It was even covered here in our Democrat Gazette on April 1 (“Study assails charter school network’s feats; KIPP calls data flawed”). KIPP provided its official response here very quickly after the Miron report came out.
So, who should we believe? Where should we turn for the truth? KIPP – of course, it seems clear that KIPP has an interest in presenting a positive picture. As for Miron – it is at least the case that Miron’s past work has been critical of charters with potentially shaky methodology and that he belongs to groups that are typically antagonistic to charters and choice-based reform (Fellow at NEPC). Given all of that, let’s skip past the criticisms of Miron and the responses of KIPP and look for answers elsewhere, by somebody with no dog in this hunt.
Fortunately, just this past week, I was at AERA (American Educational Research Association) in New Orleans and attended two presentations at which researchers from Mathematica Policy Research , or MPR, (a well respected third party research firm) presented results from its multi-year evaluation of KIPP schools across the country. Here are the key findings, taken verbatim from report:
KIPP Schools Get a Thumbs-Up from Mathematica!
***
In the first report from that evaluation, “Student Characteristics and Achievement in 22 KIPP Middle Schools” (Tuttle et al. 2010a), we presented preliminary findings from a matched, longitudinal analysis designed to estimate KIPP’s effect on student achievement in a nationwide sample, the first study of its kind. Students entering these 22 KIPP middle schools typically had prior achievement levels that were lower than the average for the local school districts. Still, for most of the KIPP schools studied, quasi-experimental impacts on students’ state assessment scores in mathematics (18 of 22) and reading (15 of 22) were positive, statistically significant, and educationally substantial.
Second, the report examined rates of attrition from KIPP schools relative to attrition from nearby traditional public schools. The data in that report showed that students do not leave KIPP middle schools before completion (finishing eighth grade) at higher rates than do students in local district schools. The cumulative attrition rate (defined below) in KIPP was 34 percent, compared to 33 or 35 percent, depending on how one defines the comparison group of “local district schools.”
***
Mathematica will report next year on a random assignment study of KIPP schools to further investigate the academic impacts of KIPP. But as of now, it looks like KIPP is doing well by its students. And these good results are not due to smoke, mirrors, or attrition; it looks like these results stem from strong leadership, great teaching, and hard work!
Each spring, there are a few certainties across the US. Many of us will procrastinate and hide from our tax forms. Others will get out in the nice weather to ride bikes, run, or otherwise attempt to get in physical shape while enjoying the Spring air. At the same time, our public schools will be testing our students’ mental fitness with annual state standardized tests (in Arkansas, we have our benchmark exams). And the information provided by these exams is useful and important for students, parents, educators, and the general public. Contrary to claims of critics, the results don’t tell us whether our children “test well”; they tell us if our students have learned what we hoped they would learn.
Unfortunately, the Spring air is also filled with cries of the critics bemoaning the “oppressive” standardized testing imposed upon our young students and claim (with NO evidence) that it is ruining our once-great system of public schools. But don’t worry … these claims are either exaggerated or just plain wrong.
“In fact, even the students in our most heavily tested grades (5 and 7) participate in standardized assessments for only 12 hours of the approximately 900 instructional hours availed to the students each year. This represents only about 1% of total instructional time. As a point of comparison, state law mandates that 5th grade students spend 36 hours each year in physical education classes.”
These frameworks have been developed by Arkansas policymakers (along with committees of educators and other members of the community) to outline what our educators and citizens think children ought to be learning in each grade (some version of this happens in every state);
It seems reasonable that the job of teachers is to teach a broad range of academic content (much of which will end up on the test) outlined in the Curricular Frameworks. Ideally, the tests should require no additional preparation beyond what the teachers and students do each day. Indeed, as many good teachers have shared with us in interviews, they should not do ANYTHING different because of the existence of exams. Good teachers will teach to the curricular standards, whether or not there are state exams.
Unfortunately, some educators might choose to “teach to the test” in harmful ways. Critics of testing will share stories of teachers who put their students to sleep with repetitious drilling of one test item after another; we hear of schools in which students might be encouraged to “stay home” on test day; and we even hear of outright cheating on the state assessments so that school results can appear better than they might have been. Of course, these practices are unhelpful for our students and even unethical. Nevertheless, the gamesmanship and cheating reported each year are isolated examples — we have no evidence that these problems are widespread. In any case, the problem here is the bad pedagogy, not the exams.
How strong can an argument be if it is based on a small minority of teachers who adopt poor practices in an effort to “beat the tests”? These teachers — and the chorus of testing critics — are missing the point.
The best way to “beat the test” is to spend the academic year ensuring that our students have the skills and content-knowledge they need to score well on them. And that isn’t called gamesmanship, cheating, or even “teaching to the test” …. that’s just called teaching, and good teachers do it every day!
The defenders of the status quo in the heated debate over school reform have increasingly adopted a new strategy. Many of these folks — who incidentally criticize the “reformer” crowd for attacking teachers — have decided that it is a good idea to attack KIPP schools. What would a critic say, you might ask, about the network of KIPP schools that spring up in the poorest of areas to serve the most disadvantaged of students and instill in them realistic dreams of success and college graduation??
Well, the argument goes, KIPP educators and students are succeeding because KIPP only takes on the best students (in these extremely disadvantaged areas). That is, KIPP’s success is due to high attrition and the fact that the “weakest” students leave. This is not only silly, as has been shown by credible academic work, but it is also surprising and borders on mean-spirited.
Why would a researcher, who could choose to study anything, spend his or her time looking to poke holes in the good work done at KIPP? Many of us presumably got into this field of K12 education because we care about kids. Perhaps we worked in schools with disadvantaged students and thought we might make a more important positive impact for kids by attempting to influence policy for the better (my story). And, I imagine I am not the only one who sometimes feels far removed from the students who spurred me to enter this field … all of that to say, it is more than inspiring to marvel at the great work of dedicated educators working 12-hour days so that poor students can have access to post-secondary education (the same access that researchers like me take for granted for our own children… and likely so do you if you are the type of person who does research or reads a blog for fun!).
Motivation aside, if you haven’t heard, the recent KIPP-critique was published by Gary Miron of Western Michigan University and colleagues; it is entitled: ”What Makes KIPP Work? A Study of Student Characteristics, Attrition and School Finance“. Miron claims that KIPP succeeds because of abnormally high attrition and abnormally high levels of school funding. Sadly, KIPP officials had to be spend some time last week rebutting some of this silliness and did so coherently and concisely — and graciously — on their own web site. The holes in Miron’s argument are obvious and easy to expose. He claims that KIPP suffers from high attrition by comparing the attrition rate at KIPP SCHOOLS to the same rate at the nearby school DISTRICTS. Obviously, the appropriate comparison would be between schools and schools; Miron’s analysis ignores any mobility BETWEEN traditional public schools within a district and thus significantly understates the attrition rates within traditional public schools.
Miron may or may not be aware of this obvious flaw in research design – he previously published a report called “Schools Without Diversity ...” in which he criticized various charter schools for being far too racially segregated based on comparisons between charter SCHOOLS and traditional public school DISTRICTS. Here, again, this analytic strategy neglects to consider the obvious racial segregation within individual traditional public schools.
Back to KIPP ….. it is one thing to have an intellectual debate on the various benefits and costs of charters in general. However, it seems another thing altogether for those in our field, ostensibly working so that kids have better life opportunities, to put so much effort into explaining away the obviously good work of those smart and hard-working educators who serve KIPPsters across the country each and every day. We academics and policy geeks can and should continue to argue about the details of school reform, but let’s not spend our time denigrating teachers and leaders who are doing each day what the rest of us are not — improving the lives of poor students.
And make no mistake about it — KIPP schools are doing just that. If you like hard evidence with real numbers and scientifically rigorous comparisons, check out the recent study done by Mathematica Policy Research. The only knock against Mathematica is that their impartial research designs are so conservative that they almost never find positive results. But, here is what Mathematica found when applying a rigorous research design to KIPP:
“ For the vast majority of KIPP schools studied, impacts on students’ state assessment scores in mathematics and reading are positive, statistically significant, and educationally substantial. ”
And if you’re not into numbers and data, check out these journalistic accounts of the good work being done by KIPP … produced by typical news organizations, by no means school reformers or so-called enemies of public schools. Try this story on PBS, or this one on MSNBC, or the video below by NBC. Or, if you like more music, check out this KIPP Welcome Video.
Diane Ravitch has it out for Texas in her latest post at Ed Week:
I am sure you recall that when No Child Left Behind was under discussion, there was a great deal of publicity about “the Texas Miracle.” I remember newspaper accounts of the wonders that had been accomplished by the simple strategy of testing and accountability.
Soon after the election of George W. Bush as president, we learned that he was the architect of this miracle in Texas. The miracle occurred because of this strategy: the state tested every child every year in grades 3-8; disaggregated their scores by race, ethnicity, and other characteristics; published the scores; and then honored the schools where scores went up and shamed the schools where they did not. Mirabile dictu, it worked! Or so a credulous press told us. Test scores went up, graduation rates went up, and the achievement gap began to close.
. . .
But what we now know is that there never was a Texas miracle. At best, it was wishful thinking. At worst, it was a lie.
On the National Assessment of Educational Progress of reading, 8th grade students in Texas had exactly the same score in 2009 as they had in 1998. No progress, period.
By coincidence, Ravitch managed to pick the one NAEP score in which Texas students didn’t show improvement over the past decade or so. Look at how Texas students improved on NAEP in 4th grade reading, 4th grade math, and 8th grade math:
8th grade math: In 1996 (no data for 1998, Ravitch’s starting point), 41% of Texan eighth graders were below basic, and 21% were proficient or advanced. In 2009, only 22% were below basic and 36% were proficient or advanced. In other words, the number of kids below basic was cut almost in half, while the number who were proficient or advanced went up by 15 percentage points.
4th grade math: In 1996, 31% were below basic; only 25% were proficient or advanced. In 2009, 15% were below basic; 38% were proficient or advanced.
4th grade reading: In 1998, 42% were below basic; 28% were proficient or advanced. In 2009, 34% were below basic; 32% were proficient or advanced.
Now, as social scientists know, you can’t tell much of anything from trends in NAEP scores, without adjusting or controlling for anything. So Ravitch’s attempt at analysis is bogus in the first place. But if she had applied her analysis fairly — rather than by cherry-picking a single item — she would have to say that the huge gains in 4th and 8th grade math, and smaller gains in 4th grade reading, prove that Texas’s accountability system worked.
Over at the OEP blog, Gary Ritter does a little fact-checking on a recent report about charter schools making the rounds here in Arkansas, written by the Arkansas Public Policy Panel (and co-sponsored by the Arkansas Education Association and Arkansas Advocates for Children and Families). Here’s an excerpt:
“In short, the report (co-sponsored with AEA and our friends at AACF) is based on analyses and data that are quite simply ….. well .. bad. In my comments in the Dem Gaz article, I provided a few of my concerns with the report. However, with the extra space provided here in cyber-land, I will elaborate a bit on my concerns and organize them within two primary categories. First of all, the data used and analyses conducted are inappropriate. Secondly, and perhaps more importantly, the conclusions drawn by the authors are not supported by evidence.
Faulty Comparisons Using Bad Data
The regression analyses are based on grade levels (instead of individual students), use only one year of data with no attempt to measure student growth (and that year is not even the most recent year available), and include very strange choices for comparison schools.
Among the examples of inappropriate comparisons are the use of Farmington as a comparison district for Haas Hall (which has been located in Fayetteville for several years) and the use of Little Rock SD as a comparison for the Arkansas Virtual Academy (which has students from all across the state).
Moreover, the report erroneously suggests that Haas Hall and the Virtual Academy serve NO low-income children. This is because neither school has a cafeteria and thus neither serves any free lunches. This does NOT mean that there are 0% low-income children in these schools (school leaders at these schools claim to serve about 50% poor students).
Conclusions Are Unrelated to Any Evidence
The report concludes that charters cause a problem for students left in “under-resourced” traditional public schools. I am not exactly sure what under-resourced means here. For example, in 2009-10, traditional public schools in Arkansas spent $11,717 per student, while charters spent $9,417 (data gathered from ADE Annual Statistical Reports).
Finally - and this is especially relevant this week – the authors conclude (without any connection to the data they used) that the state needs more accountability to close failing charters. It should be clear to any observer of education in Arkansas that charter schools actually face more and stricter accountability. First of all, they receive no funding if no students make the active choice to enroll there. Charters must also go before the State Board for renewal regularly. And, of course, the State Board on Monday voted to shut down a Little Rock charter school — immediately — due to financial problems (click here for more on the reaction of UCPC students and parents). How much stricter should our state’s charter policies be?
Throughout, the report is also laden with internal inconsistencies … on page 1, the authors cite a Stanford study from 2009 showing that AR charters outperformed traditional public schools. Based on this, the authors give credit to the state board’s careful screening process for charters. Nonetheless, the authors then conclude that the evidence suggests that the state needs more criteria and accountability for charter schools. Which is it — do charters perform well or not? Do we have a good screening process or not?
It is good that this group tried to assess student performance; it is not so good that the conclusions were entirely unrelated to the data and were likely drawn up well before any statistical analyses were conducted!
In fact, the authors’ interpretation of the student performance data (despite the flawed analyses) ended up being pretty reasonable and in line with what others have found. Charters likely perform just a little bit better than their traditional public school peers. Some do great work (e.g. KIPP), and some do a lousy job. This is also the case with traditional public schools — most are pretty good, some are great, but some are not very good. The difference is that the State Board generally allows under-performing traditional public schools to stay open and the students in these schools do not have the option to go elsewhere.
I am becoming more convinced that this whole debate is counter-productive in our state. In my view, it would be far more productive if those of us in the education establishment would spend less time trying to limit the growth of charters and spend more time trying to improve the education we deliver to the 95% of students who attend traditional public schools. Every minute that we spend lobbying policymakers to fight charter schools is a minute we’re NOT trying to help our teachers come up with even better strategies to serve all students across Arkansas.”
There were a couple of pieces over the weekend that make a great followup to the conversation MPetty and I were having in the comments section the other day. We were discussing the increasing income inequality in the US, and I attributed the increase mainly to the increasing skills premium. Low skilled workers are facing very serious wage pressure, from both machines and foreigners, in the globalized economy, and thus their wages have not grown as quickly as high skilled workers. On the other side, the demand for high skilled workers worldwide has gown more quickly than the supply, and thus their wages have grown more quickly than those of low skilled workers. My solution to the problem was to better educate our children to compete in the global labor market.
Interestingly, some of my reading over the weekend has called into question how I would have previously categorized low and high skilled workers. Here is a NY Times piece that talks about how lawyers are becoming obsolete (can’t say I’m too disappointed). And here is a Paul Krugman piece that summarizes economist David Autor. The point of the piece can be summarized as follows:
…the crucial difference in terms of possible replacement of humans by machines was one of routine versus non-routine, rather than white-collar versus blue-collar, and that computerization was if anything likely to increase demand for some “low-skill” occupations and reduce demand for some traditionally well-paying white-collar jobs.
Those of you who still held out hope that Paul Krugman had not gone full tilt into political hackery FOX News style might want to check out one of his recent columns here and the subsequent takedown by blogger iowahawk here. And here is a little taste of iowahawk’s post:
Paul Krugman. The Times’ staff economics blowhard recently typed, re the state of education in Texas:
And in low-tax, low-spending Texas, the kids are not all right. The high school graduation rate, at just 61.3 percent, puts Texas 43rd out of 50 in state rankings. Nationally, the state ranks fifth in child poverty; it leads in the percentage of children without health insurance. And only 78 percent of Texas children are in excellent or very good health, significantly below the national average.
Similarly, The Economist passes on what appears to be the cut-’n'-paste lefty factoid du jour:
Only 5 states do not have collective bargaining for educators and have deemed it illegal. Those states and their ranking on ACT/SAT scores are as follows:
South Carolina – 50th
North Carolina – 49th
Georgia – 48th
Texas – 47th
Virginia – 44th
If you are wondering, Wisconsin, with its collective bargaining for teachers, is ranked 2nd in the country.
The point being, I suppose, is that unionized teachers stand as a thin chalk-stained line keeping Wisconsin from descending into the dystopian non-union educational hellscape of Texas. Interesting, if it wasn’t complete bullshit.
As a son of Iowa, I’m no stranger to bragging about my home state’s ranking on various standardized test. Like Wisconsin we Iowans usually rank near the top of the heap on average ACT/SAT scores. We are usually joined there by Minnesota, Nebraska, and the various Dakotas; Vermont, Maine, New Hampshire…
… beginning to see a pattern? Perhaps because a state’s “average ACT/SAT” is, for all intents and purposes, a proxy for the percent of white people who live there. In fact, the lion’s share of state-to-state variance in test scores is accounted for by differences in ethnic composition. Minority students – regardless of state residence – tend to score lower than white students on standardized test, and the higher the proportion of minority students in a state the lower its overall test scores tend to be.
Please note: this has nothing to do with innate ability or aptitude. Quite to the contrary, I believe the test gap between minority students and white students can be attributed to differences in socioeconomic status. And poverty. And yes, racism. And yes, family structure. Whatever combination of reasons, the gap exists, and it’s mathematical sophistry to compare the combined average test scores in a state like Wisconsin (4% black, 4% Hispanic) with a state like Texas (12% black, 30% Hispanic).
So how to compare educational achievement between two states with such dissimilar populations? In data analysis this is usually done by treating ethnicity as a “covariate.” A very simple way to do this is by comparing educational achievement between states within the same ethnic group. In other words, do black students perform better in Wisconsin than Texas? Do Hispanic students perform better in Wisconsin or Texas? White students? If Wisconsin’s kids consistently beat their Texas counterparts, after controlling for ethnicity, then there’s a strong case that maybe Texas schools ought to become a union shop.
My collegue and coauthor, Robert Costrell, has a great op-ed in the Wall St. Journal today discussing public employee benefits in Wisconsin. Here is a section from his piece:
The showdown in Wisconsin over fringe benefits for public employees boils down to one number: 74.2. That’s how many cents the public pays Milwaukee public-school teachers and other employees for retirement and health benefits for every dollar they receive in salary. The corresponding rate for employees of private firms is 24.3 cents.
Gov. Scott Walker’s proposal would bring public-employee benefits closer in line with those of workers in the private sector. And to prevent benefits from reaching sky-high levels in the future, he wants to restrict collective-bargaining rights.
The average Milwaukee public-school teacher salary is $56,500, but with benefits the total package is $100,005, according to the manager of financial planning for Milwaukee public schools.
So, what’s the point? While I do not agree with the way that Gov. Walker has handled the situation in Wisconsin (actually I think the Gov. is being kind of a jerk), I think it is important for the public to fully understand the compensation packages that public employees have secured through collective bargaining. Unlike others, I do not think that the collective bargaining is the real problem in Wisconsin. The biggest reason benefit rates are so high is the opaque and deferred nature of of the cost. Politicians have an incentive to give public employees deferred compensation because it provides them with a political benefit at little to no cost to the current budget.
Over at the 
My collegue and coauthor, Robert Costrell, has a 