Sympathetic ranking under fire
Van Rooij is one of five specialists who, on behalf of the Coimbra group (a network of 38 classic European universities), wrote a position paper stating that the problems surrounding the ranking are worthy of further discussion. The raw data is too difficult to verify and access, according to the writers of the paper, which has to be submitted to the European Commission. Independent quality assessments, such as visitation commissions, are not included. As a result, universities are still not being compared fairly and institutions are choosing not to participate.
‘U-Multirank is a sympathetic and brave endeavour by the universities’, says Van Rooij. ‘But there has been criticism from the beginning.’ They should take a step back rather than focusing on the rankings, according to the writers.
On the U-Multirank website, universities are not ranked in an overall list like they are in by Times Higher Education or the Shanghai ranking. Instead, they are compared by academic field or study. Such an approach should be fairer, given that all universities have different strengths and weaknesses. But unless the details can be verified and an held to an international standard, the system will inevitably fall short.
‘And now that the financing for the project is ending, they will have to come up with alternative funding’, Van Rooij says. ‘They are thinking about publishing a magazine which they can connect to a commercial benchmark tool, but then it would simply become some sort of international study guide.’
Project coordinator Gero Federkeil from U-Multirank denies that there is a lack of transparency. ‘Our methodology is completely transparent and is free for anyone to see on our website’, he says in a response in University World News. The website put up a official response. ‘It is not simple to find valid and reliable data for outcomes that have not been analysed before. After two successful U-Multirank publications and the near completion of the third, we are confident that “better rankings” can be produced.’
Van Rooij and his position paper co-authors propose that U-Multirank continue as a public database with objective and transparent information. Universities can then decide for themselves what to do with the details. ‘Then they can relaunch as something that we can actually benefit from.’