The Republic ran a story, picked up by the Star, about a study saying Phoenix is the worst city in the country at retaining its college grads. It turns out the study forgot one minor major detail. Phoenix is the home of the University of Phoenix which has students all over the country. Naturally, lots of them don't end up living in the Phoenix area since they never set foot in the city while they attended college. Take them out of the equation, and Phoenix moves up considerably. It's a terrific example of why results of studies, in education and elsewhere, can't be taken at face value. "Buyer Beware" is always the best policy.

The original study concluded that only 36.3 percent of college grads stayed in the Phoenix area, last in the nation and less half the top rate for large metro areas of 77.7 percent. It's even worse when the study only looks at grads of four year colleges: 18 percent retention for Phoenix, eight points below the nearest city and one quarter the retention rate at the top of the scale.

Here's the problem with the Phoenix numbers.

MAG [Maricopa Association of Governments] claims Brookings used data pulled from LinkedIn profiles to determine where students of Arizona and other institutions were living today. That included all of the online students from University of Phoenix and Grand Canyon University — more than 200,000 students in 2014 alone — into the category of Arizona residents while students.
 
[Anubhav Bagley, MAG’s data guru] said only 8,400 University of Phoenix students of its 187,000 enrollment are Arizonans. Over the past 15 years, University of Arizona and Arizona State University between them graduated over 284,000 students.

The authors of the original study went back and revised the Phoenix data to 56 percent retention for all colleges and 41 percent for four year colleges, moving Phoenix out of the bottom ten metro areas. But they didn't change their original report, they just stuck an "UPDATE" about some "good suggestions" they received at the bottom. It's not referenced in the original, so unless you read the report to the end, you'd never know they revised the numbers. 

In response to this post, Rothwell and I received a number of good suggestions about how to deepen and refine our analysis in the future. Several pointed out that Phoenix is home to the University of Phoenix, with its large online student body, many of whom don’t live in the Phoenix metro. When Rothwell redid the numbers taking this into account, Phoenix's retention rates improved to 56 percent for two- and four-year institutions and 41 percent for four-year institutions.

The Republic ran another article noting the error in the original study. I haven't seen a similar correction in the Star.