4/13/2005

Statistics

On my weekly stroll through the Carnival of Education at The Education Wonks blog, Kimberly Swygert over at the Number 2 Pencil blog informed us of a provocative study that challenges the assumption that private schools do a better job than public schools -- at least if you look at the NAEP math scores in a certain way.

CHAMPAIGN, Ill. - Students do better in private schools, according to common wisdom -- and some well-regarded data now more than two decades old.

But a recent study of standardized math scores in more than 1,300 public and private schools says the opposite may be true, according to Sarah and Christopher Lubienski, education professors at the University of Illinois at Urbana-Champaign.

Public school students from similar social and economic backgrounds tested higher in a national math achievement test than their peers in private schools, the Lubienskis say in an article to be published in the May issue of Phi Delta Kappan, an influential education journal.

They also are presenting their findings at the annual meeting of the American Educational Research Association (AERA), being held April 11-15 in Montreal.

"These results call into question common assumptions about public and private school effects, and highlight the importance of carefully considering socioeconomic differences in comparisons of school achievement," the Lubienskis wrote.

The achievement and survey data used in the researchers' study came from the 2000 National Assessment of Educational Progress (NAEP), the most recent annual assessment for which raw data were available to researchers.

"These results are significant because all the most prominent reforms right now assume that private schools do better, and that if you take a disadvantaged kid and give that kid an option to go to a private school, that will boost their achievement," [Christopher Lubienski] said.

The research at least provides grounds to question that assumption, especially since the data upon which it is based is more than two decades old, he said.

"We can't make claims about the effects of schools on individual students," Chris Lubienski said, "but there's reason here to question the overall assumptions behind a lot of the private-market choice proposals being promoted right now."

Full text: http://www.news.uiuc.edu/news/05/0407school.html

Whoa Nelly!

Helpfully, Kimberly illustrates Dr. Lawrence Gray's statement that "all data are suspect, especially data used to sell something." Kimberly explains how something called Simpson's Paradox can turn the results of studies such as this on its head, depending on how the subjects in the study are grouped (for example, by socioeconomic status).

It's also interesting to note that this research was based on the results of the National Assessment of Education Progress (NAEP) tests, a.k.a. "The Nation's Report Card," which assesses how well schools are toeing the line of federally-supported methods and curricula such as integrated math, which is accepted more widely in public schools than in private schools.

Check out the comments to Kimberly's post, too. Two examples:
Much like politics, all education is local. General statements that public or private are better are pretty meaningless. What is relevent as a parent is what public school my child would attend and what private school alternatives exist.

and

...the article studied "standardized math scores". They refer to the NAEP test, which is very trivial. They also refer to "average math achievement scores", but do not elaborate. There are distinct problems with using an average number for simple tests. What does an average value mean when the top end of the bell curve is chopped off (a different shape)? I had many professors in college that always made sure that the tests were hard enough to keep the average in the 60's.

The results could mean that private schools aren't great; public schools are worse, and parents make up for a lot at home.

We will need help from some impartial statisticians when the Wayzata School District surveys its graduates about the effectiveness of its Chicago Math-Connected Math-Core Plus Math track. Parents, stay involved with your kids' education, and watch out for statistics.