Besselaar and Leydesdorff tested whether the grant decisions correlate with the past performances of the applicants in terms of publications and citations, and with the results of the peer review process organized by the Netherlands Research Council

In their paper they show that the Council is successful in distinguishing grant applicants with above-average performance from those with below-average performance, but within the former group no correlation could be found between past performance and receiving a grant. When comparing the best performing researchers who were denied funding with the group of researchers who received it, the rejected researchers significantly outperformed the funded ones. Within the top half of the distribution, neither the review outcomes nor past performance measures correlate positively with the decisions of the Council.

The authors conclude with some questions for further research. They suggest a network analysis of applicants, reviewers, committee members, and Council board members. This might provide an answer to the question whether funding is correlated to the visibility of the applicants within these networks. After all, in the social process of granting proposals many processes play a role, apart from scholarly quality: bias, old-boys’ networks and other types of social networks, bureaucratic competencies, dominant paradigms, etc., all play an important role in selection processes.

If my reading of the paper is correct, it might also point to a discrepancy between the grant decision makers and the international academic community. If we consider that metrics (past performance) and peer review very much emerge in international networks and the grant distributors make decisions contradicting the metrics and peer review, what does that tell about the Council members’ involvement in these international networks?

The paper will be published later this year in the journal Research Evaluation.

Research output in management is still concentrated: less than 3 per cent of the world’s universities produce more than 70 per cent of global output. Of these 214 universities, 126 are in the US, 13 in Canada, 57 in Europe and 18 in Asia and elsewhere. But comparative world positions have been changing quickly. My research** reveals how the world’s academic business research output has become more dispersed.

While Wharton and Harvard are still the best by a margin, Europe now accounts for 25 per cent of international research output. Its best schools – London Business School, Rotterdam School of Management (Erasmus), Insead and Tilburg are in the global top 30. Asian schools – in China and Singapore especially – are further behind but their stock is rising even faster.

While some of the best US schools admit privately to being worried, publicly they stress their continued dominance – at least, according to their data. But their measurements overemphasise past successes, ignore current trends and importantly use narrowly based research measures, looking only at material published in US journals and ignoring the fact that important new ideas are increasingly being published in highly regarded, peer-reviewed non-US publications.

The interesting part is his explanation for the rise of European and Asian management research. He claims they are more innovative in their approaches and engage more in cross border comparative work than their Colleagues in the US. Another factor is that European and Asian researchers seem to focus more on micro issues where US academics emphasise macrostatistical trends.

I have always admired the US management research and think they have produced some of the most interesting and sophisticated social science studies in the past decades. Not just in the field of economics but especially in sociology where many of the recent breakthroughs have come out of business schools. At the same I indeed found them to be very US centric. I think this is related to their emphasis on macrostatistical trends. If the priority is on the cleanliness of data sets and the complexity of the modeling, than comparative studies are just a nuisance. But of course, social sciences can not be just about data and models, it’s also about reality. And the reality is after all becoming less tidy, more global and less US centred…

The authors collected six year of student evaluations where students rate their programmes on a scale of 1-10  (as published annually in the Dutch weekly magazine Elsevier). Market share for each programme/university combination was calculated by dividing the number of students in programme X in university Y by all students in the Netherlands in programme X. When the evaluation of the programme is compared with the market share, we get the following graph:

On the basis of this finding (and the results of a simulation that they run), they present some interesting conclusions. I won’t go in details, but one of them is that there is no clear relation between evaluation and market share. Hence, students have other criteria than quality in the choice of where they will attend university (especially location, and in particular the distance to their place of residence).

In their discussion, the authors indicate that this shows that competition on the basis of quality is not really taking place in the Netherlands. One reaction could be that the government should enable universities to compete on the basis of price. In other words, universities should be able to set their own tuition fees. This is currently not allowed for Dutch and EU students and the government decided last week that this will not be possible in the near future.

I could pose an alternative hypothesis on the interpretation of the results. The results show that students don’t take quality of education into account in their decision-making process. This could indicate that students don’t purchase a service (a high quality education), but a product (a degree). Universities should therefore emphasise the quality of their degrees, not the quality of their education. If the value of the degree differs per university (reflected in the opportunities to enter the labour market in higher positions or on a higher remuneration), the students will take this into account. This then would mean that universities are better of improving their position in rankings, attracting more prestigious scholars (like Nobel laureates) and increase their budget for marketing. This will bring them a in a better competitive position. On the other hand… maybe economics doesn’t provide answers for everything…

Erjen van Nierop, Peter Verhoef, Philip Hans Franses. Studie Evaluaties en marktaandelen van universiteiten (subscription required). Economisch Statistische Berichten, 4 April 2008.

One of the major current issues in (higher) education policy – in my opinion – is the issue of excellence versus equity. Is it most important to focus on the masses and see to it that everyone gets the same high quality education? Or should the country’s prime talents be nurtured and given the opportunity to fully exploit their opportunities. Or, better yet, can you do both?

The BBC Radio Documentary series ‘The Changing World’ investigates the dilemma, and what better places to visit than Finland on the one hand and the UK and US on the other. Finland is generally seen as the success story in creating a high quality egalitarian knowledge society. Finland shows very good results on the global tests like PISA (Programme for International Student Assessment), TIMSS (Third International Mathematics and Science Study) and IALS (International Adult Literacy Survey).

In addition, they manage to do so without having schools that really fail. The range between the outcomes in the best and the worst schools is very small. An interesting observation in the interviews is that the consistent high performance of Finish schools is not only a consequence of their egalitarian education system, but also of the stability and equality in Finnish society as a whole. Major differences between suburbs or districts don’t exist and therefore major quality differences between schools don’t exist either. This of course sets limits to the transferability of the Finnish system.

Consistent high quality is not the case in the UK and the US. Their global tests scores are lagging behind many northern European countries and Asian countries (see for instance this post and this table). On the other hand, they consistently occupy the top positions in international university rankings. Westminster is the main purveyor to Oxford and Cambridge, while Harvard students see themselves as the future leaders of America and of the world (as one of the students in the documentary explains).

Of course it is a case of comparing apples and oranges. Singling out individual schools and comparing them with a whole system is impossible. It does however illustrate the dilemma. Clearly, there seems to be a worldwide shift towards ‘the excellence model’. In all the knowledge economy talk nowadays there appears to be a firm belief that it is the top layer that makes the difference for a country that wants to become an innovative knowledge economy.

Economic research to support this is usually based on a whole range of assumptions and has not really succeeded in fully convincing me. On the other hand, talent shouldn’t be wasted and is something that should be given the opportunity to flourish (not just for economic reasons). Have a listen and judge for yourself.

— Download the part on the egalitarian system in Finland (MP3) –

Finland has been judged to have the best educational system in the world. Here all schools get good results, teachers have extensive training — and children don’t start school until they are seven years old. It’s education for all. And each child matters.

— Download the part on the excellent institutions (MP3) –

Has education become a commodity? A visits to two of the world’s leading educational establishments: Harvard University and the Westminster School in England. They’re both elite, expensive schools with an impressive track record – the Westminster School, for instance, has been operational without interruption since the 14th Century.

Or subscribe to the podcast of The Changing World to listen to all their other outstanding radio documentaries.

Going through my daily news intake, my eyes fell on this alarming headline in Businessweek: Europe Falls Short in Higher Education. Going through the article, it seemed like the same old story (more of this and this). Nonetheless,  I decided to have a look at the source of the article. It was based on a policy brief (Why reform  Europe’s universities?) issued by Breugel, a European think tank devoted to international economics.

The report presented some interesting data and analysis of the determinants of research performance (interesting despite the fact that much of the data is based on the – ‘slightly’ controversial – Shanghai University Rankings). First of all they calculate the relative country performance based on the Shanghai rankings (indexed for US=100 and relative to country population).

This clearly shows that the US outperforms nearly all European countries in terms of ‘top-class’ universities, with only Switzerland and the UK coming close and Canada, the Netherlands, Japan and Sweden playing a small role in this league. France and Germany are not even in the top ten, but that is also related to the fact that much of the researchers in these countries are not in universities but in centres such as the Max Planck Institutes in Germany and CNRS in France. Taking the whole top 500 into account however, the US is outperformed by Europe, especially by the north west European countries. This shows that the diversity in quality is much higher in the US than in Europe.

If the research performance is related to university funding, they find a strong positive correlation between national research performance (measured by the top 500 index) and funding per student, as is shown in the graph below (click to enlarge). The US seems to be a bit of an outlier here, pointing to some inefficiency in the US spending on universities (my interpretation, not the authors’). This picture would of course look quite different if the top 50 was taken into account instead of the top 500.

The authors continue by analyzing the relation between expenditure per student and the autonomy of universities. The data on these variables come from a survey send to European universities. I do have some questions on the accuracy of the data because some of the terms used here are rather fuzzy in the European context (for instance the distinction between public and private). But leaving this aside, the most important conclusion is that more funding leads to better research performance, and that this positive effect is intensified by an increase of autonomy, especially budgetary autonomy. Analysis of American data shows a similar effect on the number of patents: the positive effect of funding increase on the number of patents is also here intensified when a university has a greater level of autonomy.

Interesting policy brief, but the question is not so much ‘why?’ European universities need to be reformed, but ‘how?’. And the answer based on this analysis is: spend more money on universities (either private or public money), and give them more freedom. I’m sure universities in Europe will welcome this conclusion. The million dollar questions then of course become how universities should spend this money and how the governments are going to keep them accountable…

The policy brief is available here and was written by Philippe Aghion, Senior Fellow at Bruegel and Professor of Economics at Harvard University; Mathias Dewatripont, Professor of Economics at Université Libre de Bruxelles; Caroline Hoxby, Professor of Economics at Harvard University; Andreu Mas-Colell, Professor of Economics at Universitat Pompeu Fabra; André Sapir, Senior Fellow at Bruegel and Professor of Economics at Université Libre de Bruxelles.

Cultural orientation toward the future differs between countries and strongly correlates with the level of competitiveness of the country. That was one of the findings of Mansour Javidan and his colleagues in the Project GLOBE (see this month’s issue of the Harvard Business Review). Since 1993, the project examines the inter-relationships between societal culture, organizational culture, and organizational leadership. Through a survey of over 17,000 middle managers in 61 societies, they found clear international differences in several areas, one of them being “future orientation”.

Their study finds that the Singaporeans have the most future oriented culture. Singapore is followed by Switzerland, the Netherlands and Malaysia. Russia, Poland, Hungary and Argentina were least future oriented. As the graph above shows (click to enlarge), there is a strong correlation between the level of competitiveness (based on the World Economic Forum rankings). In addition correlations were found for factors like GDP per capita, level of innovativeness, happiness and confidence.

The article does not say anything about the causal relationships between culture orientation and these factors…

In both the Netherlands and Australia the salaries of the top university leaders lead to controversy. The Australian reports that all but one of the leaders of Australia’s Group of 8 Universities earn more than 600,000 Australian Dollars (378,000 Euros). Top earner was John Hay of the University of Queensland with 655,000 Euros. But the Australian found even higher figures for La Trobe University where someone (probably the former VC) received over 930,000 Euros!

In the Netherlands, the salaries and bonuses in the public sector are a hot issue as well. Many claim that the Prime Minister’s salary should be the norm for others in the public sector. In the Netherlands that is a mere 171,000 Euros (John Howard’s salary was recently increased to 208,000 Euros). But most university leaders in the Netherlands make significantly more than that.

The new Dutch Minister for Education this week showed his discontent about the managerialism in education and the accompanying rise in salaries. He observes that most of them enjoyed enormous salary increases when they came into their current positions. And I am sure he is right about that (although that is not the case for all of them). One of the most visible cases has been the one in my own Alma Mater. Their top level managers were given a 31% salary increase, which sparked a reaction of the Minister claiming that this was ‘unbelievable’. This increase brought the salary of the Chairman of the Executive Board (more or less the CEO of the University) to 171,000 Euros. In comparison, the lowest earning VC in Australia, David Battersby of the University of Balarat (poor guy), earned over 200,000 Euros!

So how do the Dutch university CEOs compare with the Australian Vice-Chancellors? Basically, compared to Australia, the Dutch salaries are still very modest. Here is the list of the top 6 for both countries:

Netherlands (Source: Intermediair (pdf); in Euros)
1. Aalt Dijkhuizen	University of Wageningen	307,520
2. Sijbolt Noorda	University of Amsterdam	284,400
3. Rene Smit	Vrije Universiteit	245,900
4. Hands van Luijk	Delft Univ. of Technology	240,000
5. Yvonne van Rooy	University of Utrecht	233,000
6. Jos Elbers	Hogeschool Inholland	228,928
Australia (Source: The Australian; converted to Euros)
1. John Hay	University of Queensland	655,000
2. Gavin Brown	University of Sydney	454,000
3. Fred Hilmer	University of NSW	378,000 (+95,000)
4. Steven Schwartz	Macquarie University	378,000 (+63,000)
5. John Rickard	Central Queensland Univ.	425,000 – 434,500
6. Glyn Davis	Melbourne University	384,000

Don’t get me wrong! This is no justification for the Dutch salary hikes. More like a condemnation of the Australian salaries. The argument is usually that salaries have to be competitive. This argument is put forward just a bit more often when people talk about managers than when they talk about academics and professors. Australian professorial salaries average A$120,000 (75,600 Euros). I am not sure how much it is in the Netherlands but I think it will be slightly higher, or at least similar. Let’s just say that the huge gap between managerial and academic salaries in Australia better not be taken as an example for the Netherlands.

Of course I hadn’t expected India to have turned in to one big IT science park in just one or two decades (although some publications seem to give that picture). But I had expected India’s optimism, ambition and rupees to have trickled down to other sections of society…at least a little bit. I have not been in the booming cities of Bombay, Bangalore or Chennai, but judging from my experiences from Delhi and Rajasthan, there’s a lot of work to be done, in terms of public facilities, but especially in terms of equality.

Delhi’s airport was in many ways worse of than the smaller regional airports I had just seen while visiting Indonesia and Malaysia the two months before. The roads and other public works were definitely a lot worse. Steve Hamm of Businessweek fears that the lack of investment in public space might hurt India’s progress:

The infrastructure deficit is so critical that it could prevent India from achieving the prosperity that finally seems to be within its grasp. Without reliable power and water and a modern transportation network, the chasm between India’s moneyed elite and its 800 million poor will continue to widen, potentially destabilizing the country. Jagdish Bhagwati figures gross domestic product growth would run two percentage points higher if the country had decent roads, railways, and power. “We’re bursting at the seams,” says Kamal Nath, India’s Commerce & Industry Minister. Without better infrastructure, “we can’t continue with the growth rates we have had.”

“If you have to build a road in China, just a handful of people need to make a decision. If you want to build a road in India, it’ll take 10 years of discussion before you get a decision.”

Nearly all sectors of officialdom are riddled with graft, from neighbourhood cops to district bureaucrats to state ministers. Indian truckers pay about $5 billion a year in bribes, according to the watchdog group Transparency International. Corruption delays infrastructure projects and raises costs for those that move ahead.

Rather than having successfully produced a “new middle class,” as touted in media representations of India’s success, emphasis on knowledge for development and a knowledge economy in India has had the effect of producing an elite with formidable economic strength, as well as the cultural dominance to re-imagine and negotiate meanings of Indianness.

(…) So long as those engaged in the knowledge economy are blinded by the belief that their success reflects the progress of the nation as a whole, and that their class positions are not privileged, the possibility for sparking true social and economic change greatly diminishes.

German Minister of Research and Education Annette Schavan announced in October last year that the Ludwig-Maximilian University (Munchen) and the Technical University of Munchen and the University of Karlsruhe became Germany’s first ‘elite universities’. The three institutions are the biggest winners in Germany’s ‘excellence initiative’. This was established to improve the country’s chronically under-funded universities (and its decreasing reputation abroad), by encouraging high level research and competition. The three universities will receive around 120 million euros each in federal and state funds over the next five years.

This week, the finalists for the second round were announced. Being one of the winners is crucial considering that getting designated ‘elite’ will mean enjoying a piece of the 1.9 billion euros pie, made available from 2007 to 2011. This time the result seems less skewed towards technology, and less towards the southern part of Germany than the first round. The finalists include two institutes of higher education in Berlin, the Free University and the Humboldt University. The others are the RWTH Aachen and the universities of Bochum, Freiburg, Gottingen, Heidelberg and Constance.

The final decision on which of these eight will be designated ‘elite’ will be made in October.

Some interesting views from the German academic community on the excellence initiative can be heard in this radio interview (from NPR; 4:26 in english):

Listen to the interview in Realplayer

“The Director of a CAS research institute in Northeast China has said that while the people he attracts usually fall into the top 50% to 80% of overseas scholars, the top 20% still remain abroad. Rao Yi, a neurologist at Northwestern University in Illinois claims that, in terms of international reputation and prestige, few returning scholars are of comparable quality to those who stay abroad. He believes that there are between 800 to 1,000 scientists of Chinese origin running independent labs in the U.S., and that these people are unlikely to return.”

“In fact, surveys have shown that only a few scholars returning to China had to sacrifice high salaries or stable, tenured positions, and even fewer were returning with patents for innovative research. What’s more, getting the very talented to return is just the first step; getting them to stay is another matter altogether.”

It’ s a rather strange moment to introduce the RQF because the UK has plans to abolish the RAE and return to a system that looks more like the current Australian model. In the words of Snitch:

“So Britain is scrapping its research assessment exercise just as Australia prepares to introduce one. What’s more, Britain is returning to a metric system of measuring quality, just like the one Australia uses now.”

One very visible result of the RAE is the concentration of research. Obviously this gives less concerns to Australia’s leading universities (joined in the Group of 8) than to the other players. It’s expected that most of the research funding will be concentrated in these 8 research intensive universities.

Although the framework has not been implemented yet, some of the consequences are already visible in anticipation of the RQF:

“In a significant loss for RMIT University, a leading expert in biomedical sciences has left the campus, taking his entire staff of 15, his laboratory and research grants worth nearly $1 million a year to a research quality framework-free medical institute. As universities prepare for greater competition under the framework, global diabetes specialist Mark Febbraio has announced he will leave RMIT for the Baker Heart Research Institute in Melbourne, blaming the impending introduction of the RQF and its effect on universities outside the Group of Eight.”

Many of the universities outside the group of 8 complain that criteria for societal and economic relevance are missing in the framework, and this will even increase the diversion of funds away from the technological universities to the Group of 8. The concentration of research will likely lead to a ‘bidding war on stars‘. An Australian equivalent of a European Champions League, where the Barcelonas, the Milans, the Arsenals and the Inters will always be in the semi-finals because they can afford to buy the best players?

On the other hand, maybe not:

“The plan to introduce a national assessment system for research quality has stalled after federal Education Minister Julie Bishop announced yesterday she was setting up another advisory group to consider it.”

]]> http://blog.beerkens.info/index.php/2006/03/academic-champions-league/feed/ 0 http://blog.beerkens.info/index.php/2006/03/the-economics-of-selective-knowledge/ http://blog.beerkens.info/index.php/2006/03/the-economics-of-selective-knowledge/#comments Thu, 16 Mar 2006 05:23:00 +0000 Eric http://www.beerkens.info/weblog/http:/www.beerkens.info/weblog/the-economics-of-selective-knowledge/

1) Create and maintain a system of diverse, sustainable and high-quality educational institutions with the freedom to respond to demand and accountable for the outcomes they produce

2) Ensure that the growth and development of tertiary educational systems are managed to improve access, raise quality and enhance equity

3) Implement financing and student-support policies which mobilize public and private funding in ways that better reflect the social and private benefits of tertiary education

4) Encourage universities to evolve so that their leadership and strategic management capacity matches that of modern enterprises, with appropriate strategic, financial and human resource techniques to ensure long-term financial sustainability and accountability requirements, and

5) Ensure that universities are governed by bodies that reflect a much wider range of stakeholder interests than the academic community