{"id":653,"date":"2008-06-30T16:39:03","date_gmt":"2008-06-30T16:39:03","guid":{"rendered":"http:\/\/scientopia.org\/blogs\/goodmath\/2008\/06\/30\/market-based-college-evaluations\/"},"modified":"2008-06-30T16:39:03","modified_gmt":"2008-06-30T16:39:03","slug":"market-based-college-evaluations","status":"publish","type":"post","link":"http:\/\/www.goodmath.org\/blog\/2008\/06\/30\/market-based-college-evaluations\/","title":{"rendered":"&quot;Market based&quot; College Evaluations"},"content":{"rendered":"<p> I&#8217;m a bit late to the party on this, but I couldn&#8217;t resist<br \/>\nsaying something.<\/p>\n<p> A rather obnoxious twit by the name of Richard Vedder has set up a<br \/>\nfront-group called <a href=\"http:\/\/www.collegeaffordability.net\/aboutus.php\">&#8220;The Center for College Affordability and Productivity&#8221;.<\/a> The goal of this group is purportedly to apply market-based mechanisms to the problems of higher education<br \/>\nin America. When you take a look at their &#8220;research&#8221;, you&#8217;ll quickly recognize that this is astroturf, plain and simple.<\/p>\n<p> A typical example of this is described in <a href=\"http:\/\/www.forbes.com\/opinions\/forbes\/2008\/0519\/030.html\">an article Dr. Vedder recently wrote for Forbes magazine<\/a> about a supposed research study done by his organization on college rankings. According to Dr. Vedder, the popular &#8220;US News and World Report&#8221; college rankings are no good, and that market-based principles can produce a better, more meaningful ranking. The rationale for this new ranking system is that the standard rankings are based<br \/>\non the <em>input<\/em> to the schools: schools are ranked based on the quality of students admitted. Dr. Vedder wants to rank schools based on <em>outcomes<\/em>: how well the school achieves the goal of<br \/>\nturnings its students into educated, successful people after they<br \/>\ngraduate. According to Dr. Vedder, his ranking system tries to rank<br \/>\nschools based on several &#8220;output&#8221; measures: &#8220;How do students like their courses?&#8221;, and &#8220;What percentage of students graduate?&#8221;, &#8220;How many awards do the students recieve?&#8221;, and &#8220;How successful are students after they graduate?&#8221;<\/p>\n<p><!--more--><\/p>\n<p> Ranking schools based on their graduates rather than their enrollees<br \/>\nis an interesting idea &#8211; and it&#8217;s a worthwhile goal. But it&#8217;s pretty hard to<br \/>\ndo well. Just to point out one obvious problem, how do you define success in a quantifiably way?<\/p>\n<p> Just to drive that point home, let&#8217;s compare a few basic graduates, each<br \/>\nof which is arguably highly successful.<\/p>\n<ol>\n<li> Me. I&#8217;m a software engineer at Google. It&#8217;s a great job, at an amazing<br \/>\ncompany which is incredibly selective in its hiring process. I make<br \/>\na good living, but I&#8217;m not rich, and I probably never will be. <\/li>\n<li> A friend of mine who graduated from college the same time I did. He<br \/>\nwent and took a job with Salomon Brothers on Wall Street. He became<br \/>\nextremely rich. He also became an alchoholic and totally burned out.<br \/>\nLast I heard, he was still an alchoholic, but had quit working, and was<br \/>\nliving on the money he made in his 10 years on Wall Street.<\/li>\n<li> The husband of my PhD advisor. Terrific guy. Graduated from college,<br \/>\nand got a job working for an oil company. Hated it. So after a few years,<br \/>\nhe quit, and became a high school science teacher. Now, last I heard,<br \/>\nhe was incredibly happy with what he was doing, and his students loved him.\n<\/li>\n<li> Larry Page. Dropped out of grad school at Stanford to start a business<br \/>\nwith his friend Sergei. Now he&#8217;s one of the richest people in the world. I haven&#8217;t met him personally, but from everything I&#8217;ve heard, he&#8217;s a pretty happy guy, and he helped start the amazing company that I love to work for.<\/li>\n<li> George W. Bush. Graduated from great schools, which he got into through parental connections. Before he got into politics, he ran several terribly<br \/>\nunsuccessful businesses, and lost a huge amount of money. He also became<br \/>\nan alchoholic. Then he straightened himself out, and got into politics, and got elected first governor of Texas, and then president of the US.<\/li>\n<\/ol>\n<p> Which of those people are successful? Am I? I&#8217;ll never be rich. I&#8217;ll never own my own business. I&#8217;ll never be famous outside of a very small community of my peers. Is my college friend? He&#8217;s filthy rich. Even if he never works<br \/>\nanother day in his life, he&#8217;ll be able to live comfortably, as long as he&#8217;s a little bit smart about where he puts all that money. But he&#8217;s an alchoholic. Larry? I think everyone would agree that Larry Page is an extremely successful guy. The science teacher? He&#8217;s doing something valuable, which he loves doing, but the pay is garbage. Someone with a master degree in geology can do a hell of a lot better, money wise. How about our president? A man known for his utter lack of intellectual curiosity, who before getting into politics was<br \/>\nan unquestionable failure at business; who was, by almost any standard, an utter failure before getting into politics?<\/p>\n<p> Which ones are successful, and which aren&#8217;t? Which is most successful? How can you quantify it?<\/p>\n<p> Of course, Dr. Vedder doesn&#8217;t worry about questions like that. To people like him, people who believe in what <em>he<\/em> means by &#8220;market based approaches&#8221;, money is all that matters. Bush is a success &#8211; because no matter how much money he lost, he always had the ability to raise more through his family connections &#8211; so he&#8217;d be rich, no matter what, and wealth is success. Likewise, the alchoholic is a success &#8211; he&#8217;s rich. I&#8217;m moderately successful, but not very. Larry is obviously successful. And the science teacher is clearly <em>not<\/em> a success, because the outcome of his education was getting a job that he hated, and ending up becoming a teacher.<\/p>\n<p> I&#8217;d say that just ranking on wealth obviously doesn&#8217;t work. But how can<br \/>\nyou quantify real success? You could probably find a way to do it &#8211; but it<br \/>\nwould be a significant amount of work. And people like Dr. Vedder aren&#8217;t<br \/>\ninterested in doing real work. They&#8217;re interested in getting lots of publicity<br \/>\nin order to (A) get really big paychecks, and (B) advance their political<br \/>\nagenda. Doing real research doesn&#8217;t qualify under either of those.<\/p>\n<p> So how does Dr. Vedder generate his outcome based college rankings? He<br \/>\nuses two inputs:<\/p>\n<ol>\n<li> Course quality: The average ranking of the school&#8217;s courses on &#8220;ratemyprofessors.com&#8221;.<\/li>\n<li> The percentage of students who having listings in &#8220;Who&#8217;s Who in America&#8221;.<\/li>\n<\/ol>\n<p> He also <em>claims<\/em> that they use the number of students who win awards like Rhodes scholarships and the graduation rate, but in the fine print, you&#8217;ll find that the rankings they&#8217;ve released don&#8217;t include that data yet.<\/p>\n<p> So&#8230; We&#8217;ve got two inputs. One is an anonymous teacher ranking system. It includes 5 basic scores for each professor: easiness, helpfulness,<br \/>\nclarity, hotness, and overall quality. Yes, you read that right: one of the<br \/>\nmajor things that they ask you to rate a professor on is &#8220;hotness&#8221;. That&#8217;s the best thing that Dr. Vedder could find for evaluating course quality.<\/p>\n<p> The &#8220;graduate success&#8221; ranking is no better. For those of you who haven&#8217;t heard of &#8220;Who&#8217;s Who&#8221;, it&#8217;s an elaborate vanity scam. Basically, this publisher sends you very fancy, formal, personalized letter explaining how &#8220;Who&#8217;s Who&#8221; publishes an extremely selective list of the most successful people in America &#8211; and they&#8217;d like to include <em>you<\/em>. All you need to do is write up<br \/>\na your biography for them, summarizing your (doubtless) highly impressive<br \/>\ncareer, and they&#8217;ll include you in the next edition of the book. Of course,<br \/>\nas a member of the Who&#8217;s Who directory, it&#8217;s absolutely essential for you to have a copy! So just send in your biography, along with a nice hefty check,<br \/>\nand presto! You&#8217;re in! (The last time I got a Who&#8217;s Who offer was about 10 years ago, and at the time, they wanted $150 for the &#8220;prestigious leather bound volume&#8221;.)<\/p>\n<p> So&#8230; According to Dr. Vedder, you can evaluate colleges and universities<br \/>\nbetter that US News and World Report, using &#8220;market based principles&#8221;; and that market-based principles dictate that successful people are likely to be taken it by vanity scams.<\/p>\n<p> Yeah.<\/p>\n<p> This is what I call obfuscatory mathematics. The idea is that you have a predetermined outcome, and you want to find some way of using numbers to<br \/>\nproduce that outcome. It doesn&#8217;t matter whether it makes any sense. In fact, the point of it isn&#8217;t to make sense &#8211; it&#8217;s to <em>appear<\/em> to make sense. If you look at the press that Dr. Vedder has gotten, they uniformly talk about how his rankings are &#8220;outcome based&#8221;. They almost never go into any detail on what that means, and they <em>never<\/em> go far enough to explore the validity of his data sources. The data is <em>garbage<\/em>, utterly worthless for drawing any meaningful conclusion. But Dr. Vedder doesn&#8217;t <em>want<\/em> a meaningful conclusion. What he wants is to make an argument that higher education in America is hopelessly screwed up, and that only ultraconservative &#8220;market based principles&#8221; can possibly save it. In order to support that argument, he has to show that what we consider to be the top elite colleges aren&#8217;t as good as we thought. Using college reviews that include inspector &#8220;hotness&#8221; will produce that result &#8211; and so he uses it.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I&#8217;m a bit late to the party on this, but I couldn&#8217;t resist saying something. A rather obnoxious twit by the name of Richard Vedder has set up a front-group called &#8220;The Center for College Affordability and Productivity&#8221;. The goal of this group is purportedly to apply market-based mechanisms to the problems of higher education [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[1],"tags":[],"class_list":["post-653","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p4lzZS-ax","jetpack_sharing_enabled":true,"jetpack_likes_enabled":true,"_links":{"self":[{"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/posts\/653","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/comments?post=653"}],"version-history":[{"count":0,"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/posts\/653\/revisions"}],"wp:attachment":[{"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/media?parent=653"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/categories?post=653"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.goodmath.org\/blog\/wp-json\/wp\/v2\/tags?post=653"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}