Research assessment

Richtig gemessen?

What makes research excellent? The answer determines, for instance, who is appointed professor at ETH Zurich or which research proposals receive funding from the Swiss National Science Foundation. But what is the best method for assessing the quality of research?

external page Peer review is the current standard procedure for research assessment. The reviewers are independent researchers working in the same field.

A tricky question to answer is how such evaluations can be quantified. On the one hand quantification is often desirable, for instance, in order to be able to rank research proposals or applications for a position. The prestige and reach of research journals may function as a practical proxy. In addition, there are various metrics such as the external page Impact Factor or the external page h-Index, which were designed to measure the impact of publications and researchers e.g. via their citation frequency. Such metrics can boil down performance to simple numbers.

On the other hand, the metrics that are currently in use do not perfectly map scientific quality. If they are used in excess, other aspects of quality may end up neglected. Some examples are innovaiton potential, societal use, or researchers' own perception of how much they contributed to their field. Metrics can also create strong pressure to publish as much as possible and to prefer specific journals. High demands to quantity may eventually wield negative influence on quality.

Since ETH Zurich places high value on the quality of research, it has signed the external page San Francisco Declaration on Research Assessment (DORA) and has implemented it as described below.

Advertisements for open faculty positions at ETH Zurich mention DORA. More importantly, the members of the responsible selection committee are required to judge the candidates and their research by qualitative criteria. The same applies to the Tenure Committee, a body advising ETH's President on the evaluation of tenured assistant professurs for permanent professorships.

Further, the processes for promoting associate to full professors, awarding the title of professor ("adjunct professors"), and permanently employing scientific staff are based on diverse criteria which are independent of quantitative metrics.

Advertisements for other scientific positions, too - especially for doctoral students and for postdocs - must comply with DORA. ETH Zurich's HR department controls all published ads.

 

ETH's internal funding schemes make use of peer reviews by independent international experts in order to determine which proposals should be funded. The evaluation process respects the DORA principles.  

The ETH Research Grants, ETH's most important internal funding instrument, negate the importance of publication-based metrics by design. Their evaluation focuses on the quality, originality, and feasibility of the proposed research but ignores researchers' publication record. This automatically excludes publication-based metrics from the assessment, too.

The external page Swiss National Science Foundation, which is the biggest provider of third-party funding for ETH researchers, has likewise external page signed DORA and is implementing it step by step. 

DORA

The San Francisco Declaration on Research Assessment (external page DORA) is directed against the unduly use of publication metrics in the assessment of scientific work. ETH Zurich has external page signed DORA.

JavaScript has been disabled in your browser