At the EAIR conference in Copenhagen last month I attended an interesting presentation by Trudy Banta, a professor of higher education and vice chancellor for planning and institutional improvement at Indiana University-Purdue University. Her question was clear: Can institutions really be compared using standardised tests?

Policymakers seem determined to assess the quality of HEIs using standardised tests of student learning outcomes. Yet, Dr. Banta claims that such tests do not provide data for valid comparisons and on top of that, they measure other things than institutional performance:

Comparing test scores sounds easy, but are today’s standardised tests of generic skills capable of yielding data for valid comparisons? Twenty years of research conducted in the US using these tests indicates they are not.

It is however not the use of standardised tests as such that was criticized by Banta, but the use of such tests to compare institutions. Research in the US showed that the scores of such tests were highly correlated with the SAT scores (with correlations up to 0.9). It appeared that 81% of the variance between institutions could be explained by previous schooling. This means that the residual 19 percent is explained by a whole range of other factors (e.g. motivation, family situation, etc.), only one of them being institutional performance!

Bante therefore concludes that:

standardized tests of generic intellectual skills do not provide valid evidence of institutional differences in the quality of education provided to students.

Moreover, we see no virtue in attempting to compare institutions, since by design they are pursuing diverse missions and thus attracting students with different interests, abilities, levels of motivation, and career aspirations.

This provides food for thought for many national policy makers, but also for some international actors. I’ve written a few times about the OECD AHELO project. In this project, the OECD tries to differentiate between institutions on the basis of an assessment of the learning outcomes.

AHELO focuses on an assessment of students’ knowledge and skills towards the end of a three or four-year degree programme. The assessment will be based on a written test of the competencies of students, and will be computer delivered.

The feasibility study is expected to demonstrate the feasibility – or otherwise – of comparing HEIs’ performance from the perspective of student learning rather than relying upon research-based measures which are currently being used across the globe as overall proxies of institutional quality.

AHELOAHELO can thus partly be seen as a response to the research-biased rankings and league tables. They are presently working on a feasibility study. Whatever will be the result of this, it’s a sure thing that such a (near-)global assessment is going to be an enormously complex exercise. And therefore a very expensive one…

It’s reasonable to expect that results here also correlate strongly with prior learning, just as was the case in the US. Therefore PISA results might better explain AHELO results than institutional performance does. If the AHELO-assessment results only explains a few percentages of the variance between institutions, comparing higher education institutions will be impossible. And then all that money might better be spent otherwise. I would hope the OECD takes these American research findings into account in the feasibility study.

This article has 13 comments

  1. ???


  2. ???? ???

    [??] ???????????????..Mon Guerlain ?? @ ? Bonjour?Amanda ??????????? :: ??? :: [??] ???????????????..Mon Guerlain ?? @ ? Bonjour?Amanda ??????????? :: ??? ::

  3. tory burch

    I simply desired to thank you very much once again. I do not know what I would’ve created without the type of suggestions revealed by you over my concern. It has been the hard situation in my opinion, nevertheless considering a skilled form you handled it forced me to cry for happiness. I’m happier for the work and then expect you are aware of a powerful job that you are providing instructing most people via your blog. I am sure you’ve never got to know all of us.

  4. michael kors handbags outlet

    Thanks a lot for providing individuals with such a remarkable possiblity to read critical reviews from here. It is always very kind and full of a good time for me personally and my office acquaintances to visit your web site minimum three times a week to find out the latest stuff you have got. And of course, I’m usually satisfied for the wonderful inspiring ideas you give. Certain two tips in this post are ultimately the best I have ever had.

  5. jordan retro 6

    I as well as my friends have already been checking the excellent tips and hints found on the blog and the sudden got an awful feeling I never expressed respect to the blog owner for those techniques. The ladies appeared to be thrilled to learn all of them and have in effect in actuality been taking pleasure in them. I appreciate you for genuinely indeed considerate and then for deciding upon this sort of outstanding resources millions of individuals are really wanting to learn about. Our own sincere regret for not expressing gratitude to you sooner.

  6. yeezy boost 350 v2

    I and my guys ended up viewing the great pointers located on your site while immediately got an awful feeling I had not thanked the blog owner for them. All of the young boys are actually as a consequence glad to read through all of them and already have in actuality been having fun with them. Thanks for genuinely quite thoughtful and also for deciding on this form of exceptional areas millions of individuals are really desperate to know about. Our honest regret for not saying thanks to earlier.

  7. jordan 12

    I wanted to post a brief note so as to express gratitude to you for these wonderful concepts you are sharing at this website. My incredibly long internet research has finally been honored with incredibly good suggestions to exchange with my neighbours. I ‘d express that many of us site visitors are very blessed to live in a notable community with very many perfect professionals with valuable basics. I feel truly lucky to have used the web page and look forward to tons of more exciting minutes reading here. Thanks once again for a lot of things.

Leave a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.