Background: Russian Academic Excellence Project 5top100

The Russian Academic Excellence Project 5top100 is a Russian excellence initiative in higher education that ran from 2013 to 2020. The aim of the initiative was to place five Russian universities in the top 100 of global university rankings. The goal was not achieved; in political and academic discourses, there are conflicting assessments of the results of the project. On the one hand, the main goal of the project was not achieved; no five project participants got into the top 100 of the most common global university rankings (here we mean ARWU, THE and QS rankings). On the other hand, the Accounts Chamber (official auditor of the state budget) noted that the implementation of the 5top100 project made it possible to form a group of leading universities in the country, as well as to integrate Russia into the global trends in the implementation of academic excellence programs and strengthen the scale and significance of Russian university science in the world. We tried to look at the problem from inside the university.

The Ural Federal University has been involved in the project from the very beginning. The main mechanism for bringing public funding to the final recipients was the creation and development of research groups. This scheme was typical for most of the participants of the 5top100 project. To the best of our knowledge, our work offers the first study of the results of the implementation of the excellence initiative not at the level of universities, but at the level of research units. This approach was made possible thanks to the availability of structured data for the period 2014-2020 provided by the Department of Strategic Development and Marketing.

Key results

In total, the analysis included performance indicators of 79 research groups. The number of articles indexed in Scopus and Web of Science was used as the main indicator. As an alternative indicator of quality, we used the number of articles in journals with an impact factor above two. Unfortunately, disciplinary features were not taken into account in the process of research assessment, so the threshold was the same for all groups. Correlation analysis revealed a moderate positive relationship between the dynamics of these two indicators (0.56); thus, we refuted the widely held assumption that stimulating quantitative growth of publications necessarily leads to a decrease in their quality. Our results indirectly support the existence of constant or increasing marginal returns for most research areas.

The amount of funding was set annually depending on the type of group and the performance for the previous period. The type of group (research group, research laboratory, center of excellence) was determined depending on the number and level of group members. Paradoxically, funding (current and previous) does not have a significant effect on the change in research group productivity. Moreover, in some cases there was an inverse relationship. We can assume that in this case, prospective funding plays a more significant role than current level of funding. For example, a decrease in funding forces a group to work harder in the hope of increasing funding in the future. It is important to note that this variable refers exclusively to government funding. In terms of R&D revenue, we found an inverse relationship: the more R&D revenue, the smaller the increase in the number of articles. Thus, we can state that publications and innovative activities (measured as R&D income in our case) in research groups are substituting, not complementary.

No less surprising is the fact that the experience of the research leader of the group also has an inverse relationship with the increase in performance. This does not mean that groups with an experienced leader have poor results, quite the contrary. However, the results of such groups tend to be stable rather than growing.

Our study showed that the main factor influencing research productivity is the size of the research group. The age of the project, defined as the period of time from the establishment of a group to the termination of its operation, is also important. Note that we performed separate analyses for social and humanities areas (15 of the 79 groups) and all others. Interestingly, the age of the group has the greatest significance for both models in the social sciences and humanities fields with either the number of articles or articles in the journals with IF > 2 as dependent variables. We attribute this to the fact that there are fewer journals in these areas (purely humanities journals do not have an impact factor in principle); respectively, it takes more time to accumulate publications in top journals. Moreover, it simply takes more time to publish anything in these fields compared to natural and technical sciences. Thus, we can conclude that social sciences and humanities projects require a longer planning horizon so that one can see the effects from funding.

In conclusion, our study found that:

• The main factors affecting the performance of research teams are the number of participants and the age of the project (the latter especially for social sciences and humanities)
• Funding (current and previous) has no impact on performance, although it certainly affects the ability to hire more team members
• Decrease in current funding can paradoxically increase productivity
• Experienced researchers are more likely to create stable rather than rapidly developing groups in terms of publication activity
• Publication and R&D activities are at least partly substituting towards each other

The study focused on one university, and the number of research groups is relatively small. Comparative analysis and expansion of the empirical base will provide more meaningful results in the future. For example, it will be interesting to compare the performance of research groups across different disciplines in a larger sample. Another limitation is that only two databases (Scopus and Web of Science) were used in the evaluation procedure and in this study as well. The results obtained are highly dependent on the sources of bibliometric data. This study mainly covered quantitative performance indicators but did not touch upon other project outcomes such as increased academic mobility and foreign recruitment.

Concluding reflections

Our study showed that the 5top100 project empowered a dramatic increase in the number of publications by Russian scientists in international databases. This statement is true for both a particular university and the country as a whole. Below is the statistics of publications for 2013-2020 for the Ural Federal University, participants of the 5top100 project, and Russia.

In the end, I would like to return to where I started this post, i.e., to the project 5top100. In my opinion, the Project failed to solve two major problems of Russian higher education. First, there remains a monstrous gap between the budgets of the world's leading universities and Russian universities. Universities should have a more balanced funding structure with more private participation, for example in the form of revenue from R&D. It is impossible to solve this problem solely at the expense of public funds, and the demand for university research products (first of all, innovations) from the leading Russian companies is still at a consistently low level. The reality is that most of the science-intensive products are purchased abroad, with the only exception for the military complex. Secondly, the 5 top100 project did not lead to institutional changes within the universities and in the national higher education system. All changes were implemented in the traditional rigid vertical with no real autonomy of universities (all project participants are state universities) and decentralization within universities. The quality of academic institutions, and first of all, academic freedom determines academic excellence, in my opinion. Therefore, it will be interesting to observe not only the research performance issues under the new Priority 2030 project, but also how the new program will cope with these two challenges.

This text is solely the personal position of the author and was prepared on the basis of materials in the public domain.

Photo by Marvin Meyer