I’m on my way back to The Hague, returning from the EAIR conference in Copenhagen. Although lots of interesting new studies and findings have been presented there (some of them I’ll discuss in later posts), I actually want to talk about a conference I visited last July in Berlin.

This conference (Transparency in Diversity – Towards a European Classification of Higher Education Institutions) presented the results from the second stage of the project Classifying European Institutions for Higher Education, a project that might turn out to have a major impact on European higher education policy. This project was initiated in 2005 (see this previous post) and is now supported by for instance the European Commission (DG Education) and the German Hochschule Rektorenkonferenz. It’s run by an international team led by Frans van Vught.

The project can be seen as a response to two trends (at least, that’s my interpretation). First of all, there is the emergence of the European higher education area, the objective of the Bologna process. If there’s one space, we need to know what types of institutions are occupying that space and hence, we need a classification or typology.

Secondly, there is the proliferation of ranking and league tables. As I’ve discussed many times before, these rankings present a very uni-dimensional view of the contemporary higher education institution. Basically they only look at the - science heavy - traditional research university. Through this they neglect the quality of a very wide range of other institutions which might be very good at the things they are supposed to do. Here one can think of mono-disciplinary institutions (e.g. colleges of fine arts; schools of economics and business), teaching oriented institutions (like the American liberal arts colleges) ore more professionally and vocationally oriented institutions (like the German and Austrian Fachhochschulen, the Dutch Hogescholen, etc.).

A multidimensional classification of European higher education institutions can on the one hand create more transparency in European higher education, while at the same time clarify which institutions can be compared with each other (so we can compare apples with apples and pears with pears). If you are interested in how they intend to do this, I suggest you have a look at the presentations of the conference. See Frans van Vughts presentation (PDF) to get a better idea about the background of the project and have a look at Frans Kaiser’s presentation (PDF) for the technical aspects of such a multidimensional classification.

What the classification will look like exactly is not yet clear. If it will remain limited to the web tool and the resulting radar graphs, I expect the effects to be rather limited. The question is whether the various stakeholders related to the project will ultimately define real categories of institutions (like the old Carnegie classification did). This however might give the project a more political character. Even though the project-team stresses that they will not create a hierarchical classification, it is interesting to see whether some categories will be perceived as more prestigious than others.

Nevertheless, the classification project seems to be widely supported by institutions throughout Europe and their representative organisations. The feeling that Europe needs to create more transparency is widely shared and at the same time, many institutions are looking for benchmarking opportunities with like-minded institutions. After all, comparisons with Harvard, Oxford and Yale are not very useful for most higher education institutions in Europe…

Filed in: Europe, Higher Education, Politics, Ranking | Comment »

Tomorrow, the new 2008 Academic Ranking of World Universities will be officially published. Not surprisingly, it’s an almost all American affair. It’s rather interesting that the publication of the Shanhai Jiao Tong rankings almost goes by unnoticed, especially if you compare it to the publication of the Times Higher Education Supplement/QS World University Rankings (the THES-QS rankings 2008 will be published on 9 October).

This exactly is the strength of the SJT ranking. After all, universities are robust organisations and don’t change a lot in a years time. I guess it therefore corresponds with reality that the top 10 of 2008 is exactly the same as the one of 2007. Actually, not much has changed at all (although I of course did notice that the University of Sydney - my former employer - entered the top 100; the top 500 list is here).

2008(2007)		University
1 (1)		Harvard University
2 (2)		Stanford University
3 (3)		University California – Berkeley
4 (4)		University Cambridge
5 (5)		Massachusetts Inst Tech (MIT)
6 (6)		California Inst Tech
7 (7)		Columbia University
8 (8)		Princeton University
9 (9)		University of Chicago
10 (10)		University of Oxford

The main critique on the SJT rankings is that they only give an indication of a university’s research quality. They have only one proxy for teaching quality and that one isn’t exactly saying much about teaching quality at all. I have already pointed to some alternatives for these research biased rankings and league tables, for instance the new ranking being develop by CCAP (Center for College Affordability and Productivity).

This last one has now been published by Forbes Magazine. And yes…the criteria are very different than the ones we are used to:

1. Listing of Alumni in the 2008 Who’s Who in America (25%)
2. Student Evaluations of Professors from Ratemyprofessors.com (25%)
3. Four- Year Graduation Rates (16 2/3%)
4. Enrollment-adjusted numbers of students and faculty receiving nationally competitive awards (16 2/3%)
5. Average four year accumulated student debt of those borrowing money (16 2/3%)

And what’s the result?

2008	University
1	Princeton University
2	California Institute of Technology
3	Harvard University
4	Swarthmore College
5	Williams College
6	United States Military Academy
7	Amherst College
8	Wellesley College
9	Yale University
10	Columbia University

Compared with the SJT rankings, it are especially the liberal art colleges and the military colleges that are evident in the Forbes ranking. The high quality liberal arts colleges in the US (and elsewhere) are unfortunately lacking in nearly all international rankings. The reasons for this is of course again that these rankings are so research biased.

Another thing that I noticed after looking through the rest of the list is the relatively low standing of the public research universities. University of Virginia is the first one on 43, University of North Carolina at Chapel Hill at 66 and UC Berkeley at 73.This is probably due to another flaw in most rankings, that is that they measure the quality of the graduates without looking at the quality of the inputs. For more criticism on this ranking, see the comments on Vedder’s article in Inside HigherEd and the critical contribution of Patricia McGuire.

This challenge of actually measuring the added value provided by the university is taken up by the OECD’s AHELO project: assessing learning outcomes in higher education (sometimes referred to as the PISA for higher education). This exercise is still in it’s early stages and currently they are at the stage of studying the feasibility of such an exercise. And although the OECD explicitly does not want to promote it as a ranking, it might provide an alternative for the league tables.

Filed in: China, Ranking, USA, World | 3 Comments »

Just before going to a meeting on rankings I saw this. It is from the proposed new immigration policy: Blueprint for a modern migration policy (pdf; in Dutch). As in so many other immigration countries, it contains a chapter on skilled migration. Here is a translation of the passage that surprised me:

Anticipating the implementation of the new migration system, the government will at the latest in the first half of 2009 introduce a regulation for highly skilled immigrants. On the basis of the regulation, foreigners can stay in the country for a maximum of one year to find a job as a ‘knowledge migrant’ or to start an innovative company.

The objective of the regulation connects well to the ambition of the innovation platform to attract 1000 extra knowledge migrants. It is also in line with the advice on knowledge migrants of the Commission on Labour Participation in its report ‘towards a future that works‘.

The target group consists of foreigners that are relatively young and received their Bachelor, Master or PhD degree not longer than three years ago. Migrants are eligible if they received their degree from a university that is in the top 150 of two international league tables of universities. Because of the overlap, the lists consists of 189 universities…

And guess what the two league tables are. Yes, the Shanghai ranking and the Times Higher Education Ranking. Now…this will mean that firms like this have influence on who is eligible to come and work in the Netherlands. Something is not right here…

Filed in: Globalisation, Higher Education, Netherlands, Ranking | 2 Comments »

After the proliferation of accreditation bodies in the 1990s and 2000s, the sector witnessed the appearance of meta-accreditation. Do we - after the proliferation of rankings in the past 10 years or so - witness the first meta-ranking?

It looks like it, however I must admit it’s slightly different. It won’t be a meta-ranker, but more an accreditor of rankings. I’m talking about the establishment of the IREG–International Observatory on Academic Ranking and Excellence:

On April 18, 2008 an important decision was reached by the International Ranking Expert Group (IREG) to consolidate its partnership arrangement with the   creation of the IREG-International Observatory on Academic Ranking and Excellence.

Bob Morse, director of data research of one of the first and one of the most influential rankings - US NEWS & World Report - is one of the Executive Committee members and he writes on his blog Morse Code:

The International Observatory, headquartered in Warsaw, will conduct reviews of various ‘academic rankings’ and measures of ‘academic excellence’ to assess how well they serve higher education stakeholders and the general public. The observatory will use the recommendations formulated in the Berlin Principles on Ranking of Higher Education Institutions. Members of the body also will meet at the request of various ranking agencies to review their particular methodology criteria and standards. Ranking entities that receive observatory approval will be able to declare themselves ‘IREG Recognized’.

Especially that last item seems to point to an accreditor of rankings. But then, what gives IREG the authority to declare a ranking recognized or not? Well… at least they have some ‘recognized’ persons in their Executive Committee. Next to Bob Morse there are Gero Federkeil (CHE, Germany), Liu Nian Cai (Shanghai Jiao Tong University) and Alex Usher (Education Policy Institute, Toronto, Canada). The Committee is chaired by Jan Sadlak, the Director of UNESCO-CEPES in Romania. I wonder how this all will develop. And I wonder who will first get the ‘IREG-disapproved stamp’. Plenty of candidates…

Filed in: Higher Education, Ranking, World | Comment »

The Higher Education Funding Council published a report on the impact of rankings in the United Kingdom. It is probably one of the most extensive studies on ranking today. The study was conducted by the Centre for Higher Education Research and Information (CHERI) and Hobsons Research and is based on a survey of 91 higher education institutions in the UK and six institutional case studies.

The researchers looked at five rankings in particular, three national ones (Sunday Times Good University Guide, The Times Good University Guide, The Guardian University Guide) and two international rankings (Shanghai Rankings and the Times QS Ranking). The report itself and the background data are all available on HEFCE’s website.

Roughly, the study is divided into three parts. The first looks at rankings and their shortcomings in general. The second at the impact of rankings on universities in the UK. And the final part discusses alternative ranking methods such as the CHE ranking.

One of the most interesting questions posed in the first part is actually the same as the title of the report: counting what is measured or measuring what counts? In other words, are the criteria used in these league tables used because they are the most important determinants of quality or because those indicators are simply the ones that are (most easily) measurable? Not surprisingly, they find that:

The measures used by the compilers are largely determined by the data available rather than by clear and coherent concepts of, for example, ‘excellence’ or ‘a world class university’. Also the weightings applied do not always seem to have the desired effect on the overall scores for institutions. This brings into question the validity of the overall tables.

Several other points of critique - many of which have been discussed before, also in this blog - are confirmed in this part of the study. But the real value of the study is that it doesn’t stop here. It continues with an analyses of the survey and case studies to identify the ways in which these rankings actually shape policies. They find that institutions are indeed strongly influenced by league tables. One finding that I confirmed my expectations (see here and here) was about the link - and often contradiction - between league table criteria and other missions of the university:

League tables may conflict with other priorities. There is perceived tension between league table performance and institutional and governmental policies and concerns (e.g. on academic standards, widening participation, community engagement and the provision of socially-valued subjects). Institutions are having to manage such tensions with great care.

These are just a few quick observations. Read the full report! I will and probably post more about it at a later stage.

Filed in: Higher Education, Ranking, UK | Comment »

One of the main criticisms of international rankings is that they measure research quality rather than teaching quality. This is especially the case in for the Shanghai Jiao Tong Ranking. The THES Ranking uses proxies like employer surveys, student staff ratios and the number of international students in order to indicate education quality. The best known national university ranking is probably the one of the US News and World Report.  However, their proxies for educational quality (such as selectivity) can not be applied in a standardised global setting.

The most ambitious project to date to rank universities on education quality is the plan of the OECD to rank according to learning outcomes. Andreas Schleicher, the OECD’s head of education research explained this in the Economist in November last year:

“Rather than assuming that because a university spends more it must be better, or using other proxy measures for quality, we will look at learning outcomes”

Just as the OECD assesses primary and secondary education in their PISA assessment, it will sample university students to see what they have learned. Once enough universities are taking part, it may publish league tables showing where each country stands, just as it now does for compulsory education. This of course is a very ambitious project, if not over-ambitious. But at the same time, the OECD is probably one of the few international organisations that have the capacity and experience to assess educational outcomesat a (near) global level. Or not?

The Center for College Affordability and Productivity (CCAP) at the University of Ohio recently proposed an alternative ranking of US colleges and universities:

(more…)

Filed in: Ranking, USA, World | 3 Comments »

Ok…I seriously had the intention not to pay too much attention to the THES ranking this year. So this will be the last post about it (of course not the last post about rankings in general and their dynamics). I played around a bit with the data in Excel and had a look at it from a country perspective.

I gave a score of 200 for the number one university (Harvard) and 1 for the number 200 (RMIT; U of Cape Town) etc., and than aggregated these scores for every country. The graph below shows that the United States (with 57 universities in the top 200) and the United Kingdom (with 32 universities) are clearly superior to all other countries:

(more…)

Filed in: Higher Education, Ranking, World | 9 Comments »

I have probably written more than I should about rankings, and especially the Times Higher Education Supplement list and its flaws and shortcomings, but I just couldn’t resist… Here is a preview of this years results [last year between brackets]:

The full top 100 can be found here

For what it’s worth…

Update: Richard Holmes at the University Ranking Watch has plenty of coverage on the issue. See also University World News for a special issue on the THES ranking and rankings in general.

Related Posts:

International Rankings: a self-fulfilling nightmare?

Counting what is measured and measuring what counts

SJT World University Rankings 2008

Filed in: Higher Education, Ranking, World | 18 Comments »

Dutch newspaper the ‘Volkskrant’ has published its annual list of the 200 most influential Dutch people. The list portrays the Dutch ‘old boys network’ that rules the Netherlands through its interpersonal networks and interlocking boards and directorates. Clearly, the strength of weak ties at work; but how are these ties connected to the Dutch universities?

Topping the list this year was Alexander Rinnooy Kan, Chairman of the Social and Economic Council of the Netherlands (SER). His current profile lists chairmanships/memberships of a wide variety of boards and councils, ranging from multinationals to business schools and from museums to hospitals.

The list is assembled according to a network analysis by the Erasmus University in Rotterdam and uses a database of over 8,000 persons. The ranking is based on more than 5000 positions in 1098 bodies in 518 organisations in various sectors. The Queen, Dutch Ministers and Members of Parliament are excluded from the list. For a further explanation of the methodology, have a look here (in Dutch).

I decided to have a closer look at the top 100 and their connections to the Dutch higher education landscape. If we first have a look at the Alma Maters of these powerful individuals, the graph below shows that Leiden University (the oldest university in the Netherlands) has awarded the most degrees to the list. 19 out of 100 attended Leiden University. Interesting detail is that 15 of them studied Law in Leiden (making this club the low countries’ equivalent of the Skull & Bones??).

(more…)

Filed in: Higher Education, Netherlands, Politics, Ranking | 1 Comment »

In the latest issue of the American Journal of Sociology, Wendy Nelson Espeland (Northwestern University) and Michael Sauder (University of Iowa) present an impressive paper on Rankings and Reactivity: How Public Measures Recreate Social Worlds. The paper shows how the rise of public measures change social behaviour by looking at the law school rankings of the US News and World Report (USNWR). It struck me how many of their findings and arguments can be applied to international rankings as well. In some cases, their arguments might even be stronger for international rankings and expose additional complications.

Through processes of what the authors call ‘reactivity’, the independence between the measures and the social world they target are threatened. Rankings thus not just measure the current situation; they define it as well by changing behaviour. They identify two mechanisms of reactivity that are important in this respect: ’self-fulfilling prophecies’ and ‘commensuration’. Here, I will discuss the self-fulfilling prophecy mechanism and make an attempt to ‘translate’ this to the level of global higher education and global rankings.

(more…)

Filed in: Globalisation, Higher Education, Ranking | 6 Comments »

Here’s a typical example of how international university rankings directly influence university policies. This is a newspaper article about a national university, reported in a major national newspaper somewhere in the world:

“The university has started recruiting international undergraduate students in an effort to boost its image on a global scale. About 300 international students from various countries registered at the university here yesterday.

The Minister for Higher Education said recruiting international students was to improve its rank in world university rankings, such as the Times Higher Education Supplement’s (THES) World University Rankings.”

You could argue that it doesn’t make too much sense to focus on an indicator which only is 5% of the total score. But even more you could argue that a minister should base his objectives on national needs and national circumstances. Not base them on a widely disputed ranking which is characterised by flaws and errors.

Filed in: Higher Education, Ranking, World | Comment »

Robert Birnbaum, professor of higher education at the University of Maryland and author of some very interesting books on higher education (How Colleges Work; Management Fads in Higher Education) has written an interesting (and amusing) article in International Higher Education (the Quarterly of the Center for International Higher Education (CIHE) in Boston College).

Birnbaum is worried about the World Class University ranking crisis. Universities around the world are either proclaiming that they have attained or try to achieve this mythical status. But actually, we have no clue what it means. Philip Altbach, leader of CIHE, has written before on the cost and benefits of the race towards world class:

Everyone wants a world-class university. No country feels it can do without one. The problem is that no one knows what a world-class university is, and no one has figured out how to get one. Everyone, however, refers to the concept. We are in an age of academic hype in which universities of different kinds in diverse countries claim this exalted status-often with little justification.

Birnbaum gives some suggestion on some alternative ways to identify world class universities:

1. The Bentham System - this scheme, based on the 19th century English philosopher Jeremy Bentham’s principle of Utilitarianism, proposes that the best universities are those that bring the greatest happiness to the greatest number.
2. Olympic System - In the Olympic System, teams of university faculty would compete every four years in head to head competitions combining athletic and intellectual prowess to determine their world rankings
3. Borges System - our scholarly task is merely to identify the book of true ratings from among the infinite number of books that contain very similar, but false, ratings which also exist
4. Sausage System - Throw U.S. News, the Gourman Report, the Times Hiigher Education Supplement, and other rankings created by systems of all kinds into the same bowl, add and average out the results and voila! Just as we do not know how a sausage is made (or, more to the point, we don’t want to know) the Sausage System makes it difficult to understand just what has gone into any particular set of ratings.
5. Lake Wobegon System - in Lake Wobegon, you will remember, all the children are above average. This suggests the possibility of significantly expanding the number of institutions that can be ranked as world class merely by increasing the number of institutions in each category.

But he continues on a more serious note. Building on a metaphor by Daniel Dennett, he argues that such World Class Universities can only be built if they are firmly grounded in strong and indigenous educational and social foundations Trying to develop them by using imported rhetoric, imported models and large sums of money is destined to fail:

“Attempting to build World-Class Universities without attending first to the educational and social ground on which such institutions might stand is, as Ivan Illich once said, is “like trying to do urban renewal in New York City from the twelfth story up.” Rather than more World Class Universities, what we really need in countries everywhere are more world-class technical institutes, world-class community colleges, world class colleges of agriculture, world class teachers colleges, and world class regional state universities.”

A similar conclusion was drawn by Altbach. He concluded that as universities around the world seem to be orienting themselves to this single academic ideal, institutions and nations need to assess carefully their needs, resources, and long-term interests before launching into a campaign to build world-class institutions:

“Universities operate in both national and global contexts. The world-class idea falls into the global sphere. It assumes that the university is competing with the best academic institutions in the world and is aspiring to the pinnacle of excellence and recognition. National and even regional realities may differ. They relate to the need of the immediate society and economy and imply responsiveness to local communities. In these contexts, the nature of academic performance and roles may differ from what is expected at institutions competing in the global realm. To label one sphere world class while relegating the others to the nether regions of the academic hierarchy is perhaps inevitable, but nonetheless unfortunate.”

I wholeheartedly agree with both conclusions. Universities are one of the oldest institutions and are clearly embedded in a nations’ cultural, political and social context. This is not just the case for the Oxbridge-like universities, but also for the more recently established universities. However, this should not be a reason to avoid learning from each others experiences or models. Even copying models from other parts of the world does not necessarily lead to failure. It’s just a matter of adaptation. And it is this process of local adaptation that is made difficult by the pressures on universities to adhere to so-called global world class standards.

Filed in: Globalisation, Higher Education, Internationalisation, Ranking, World | Comment »

Serious critique on rankings and league tables. This time not about the flawed methodologies and inaccuracies in the data collection, but on the detrimental effects of rankings. They ‘erode academic vigour’ and they ‘threaten the country’s prosperity’.

In the Guardian, Geoffrey Alderman argues that the league tables lead to grade inflation and a lowering of standards in order for universities to appear high in the rankings:

“How has British higher education got itself into this mess? Part of the answer lies in the league-table culture that now permeates the sector. The more firsts and upper seconds a university awards, the higher its ranking is likely to be. So each university looks closely at the grading criteria used by its league-table near rivals, and if they are found to be using more lenient grading schemes, the argument is put about that “peer” institutions must do the same. The upholding of academic standards is thus replaced by a grotesque “bidding” game, in which standards are inevitably sacrificed on the alter of public image - as reflected in newspaper rankings.”

Filed in: Higher Education, Ranking, UK | 1 Comment »

The OECD has issued its latest Factbook. The OECD factbook 2007 contains a large amount of indicators on issues ranging from economics to the environment and from population to health. And of course on higher education. Some time ago I’ve been critical about the presentation of some of the OECD higher education statistics, but I must admit that they do a great job in collecting them. The OECD is without doubt the best source for cross-national statistics in the fields of higher education and science & innovation.

But of course you can do a lot with statistics and the media knows that. Just check out this article in the Higher Education section of the Australian:

“Australia’s spending on tertiary education per student went backwards in the eight years after the Coalition came to power, leaving the nation ranked alongside Portugal, Poland and the Slovak Republic.”

That sounds pretty bad… This definitely leaves the impression that funding per student levels in Australia are now behind Portugal, Poland and the Slovak Republic. I checked and… they are just behind the Netherlands, Norway and Denmark, clearly above the OECD average and ranking 8th in the OECD member countries. And in the mean time, it has one of the highest attainment rates for young people in the OECD (see graph; click to enlarge). Obviously this journalist only looked at the graphs that were presented on the OECD website and didn’t check any further.

But didn’t it decline then? Yes it did, and yes the commonwealth government should channel more resources to universities if it expects them to be Backing Australia’s Future. I’ve said before that I’m not exactly convinced that the current government is doing a good job in the field of higher education, but still I prefer to stay with the facts.

But what exactly the facts are is also not always clear. Some time ago, the news programme ‘the 7.30 Report‘ featured a debate between Minister Julie Bishop and Labor education spokesman Stephen Smit. I think that they spend half the time talking about the OECD statistics and how bad Australia scores in them:

Filed in: Australia, Higher Education, Politics, Ranking | 1 Comment »

It’s a fact! Rankings are getting more and more important. In a recent article in Inside Higher-ed I found out that they are even explicitly included in the performance criteria for some university presidents. In this case it was the Arizona State University president who would get an extra 60,000 US$ bonus if his university would improve its ranking in the U.S. News & World Report. But in the US, resistance against this ranking is gradually growing.

For the rest of the world it is probably the ranking of the Times Higher Education Supplement (THES) that has the biggest impact. I am sure that this ranking functions as a performance criterion for many university leaders in the world (although, much more implicitly). All the more reasons to be careful and accurate in composing the rankings. That’s what you would think…

This week’s Economist reports on an MBA ranking published by Fortune magazine. The top exists of the usual suspects like Wharton and Harvard. But one well respected business school was missing in the list: the Kenan-Flagler Business School of the University of North Carolina in Chapel Hill. The school for instance had recently placed eighth in a national ranking based on recruiter ratings published by the Wall Street Journal. So….what was the case?

It turned out that Fortune had used a European firm, Quacquarelli Symonds Ltd (QS), to collect data from the schools and create the list. When officials from Kenan-Flagler spoke with QS they were told that their school had been confused with North Carolina State’s College of Management. NC State rarely appears in any of the major rankings, but it placed 25th on Fortune’s list. Along with Kenan-Flagler, Boston University School of Management was also a victim of the flawed research.

After reading the first line, I thought: ‘again!?’ Yep… Quacquarelli Symonds Ltd (QS) did it again.

QS is the (UK based) organisation responsible for the THES rankings and they are making a real mess out of it. First time I noticed that was when the University of Malaya (UM) fell in the rankings from position 89 in 2004 to 169 in 2005. This caused quite some political turmoil in Malaysia. The result of the ranking was even discussed in Parliament. Even Prime Minister Abdullah Ahmad Badawi expressed his concerns. The Vice Chancellor at the time of the ‘rise and fall’ of UM did not continue for a next term as VC…..

What proved to be the case was that QS had counted all the Malaysian Chinese and Malaysian Indians as foreign students (one of the criteria in their rankings) in 2004. In 2005, they corrected their mistake with a steep drop in the rankings as a result. During last years publication of the THES ranking I was in Kuala Lumpur (at UM) and I could experience the impact of the THES rankings myself. I’ve never seen so many articles and letters about higher education in regular newspapers.

But…this was just the tip of the iceberg. It’s a good thing that there are people like Richard Holmes that are keeping a close watch on the methodology used in those rankings. He reports on many, many instances where QS messed up. Here are a few examples:

• On the flaws of peer review, and especially incorporating peer review as such an important criterion (40%), have a look here (on the geographical bias) , here (comparing the peer review with citations) and here (about the methodology of the peer review: a response rate of 0.8%!!!).
• On the student/faculty ratio. All indicators are indexed on the best performer on a that indicator. For faculty/student ratio in 2005 this was Duke University. It was clear that this figure was wrong (not Duke’s mistake but QS’s). However, it was not corrected for the 2006 rankings. Here he finds out what mistakes were made. Since the rest was indexed on this score, the other scores are wrong as well!
• There were also other universities were things went wrong, for instance here and here.
• And then there are simply a lot of factual mistakes reported here. No…that’s not all. There are many more.

Holmes informed THES in an open letter about all of the QS mistakes, but they are not exactly in a hurry to correct these misstakes.

But also from the research community there’s criticism. For instance this article in The Australian from Simon Marginson, a higher education researcher at the Center for the Study of Higher Education of the University of Melbourne. He also agrees that rankings are here to stay, but he does provide some valuable input for improving these rankings.

It’s rather disappointing that reputable publications like THES and Forbes use the services of companies like QS. QS clearly doesn’t have any clue about the global academic market and has no understanding of the impact that their rankings are having throughout the world. There has been a lot of critique about the indicators that they use, but at least we can see these indicators. It are the mistakes and the biases that are behind the indicators that make it unacceptable!

Fortune already took the results of the MBA ranking of their website. I wonder whether THES will do the same thing… Probably not.

For the THES/QS World University Rankings of 2007, look here

Filed in: Ranking, USA, World | 6 Comments »