Usos da avaliação da CAPES e seus fatores estimuladores e desestimuladores

Abstract

Evaluation use corresponds to the ways in which the information originated from an evaluation impacts the evaluated object. The trajectory of the evaluation field has led to the understanding that the ultimate goal of evaluation is to improve the entity being evaluated, and for this purpose, it is necessary for the evaluated entities to internalize and apply the evaluation recommendations, that is, for the evaluated entities to use the evaluation. The importance of this understanding has led to the emergence of a research area dedicated to investigating the uses of evaluation, in which this study is inserted. Although the evaluation use area has well accepted theoretical concepts and premises, there are many open questions, such as the capacity for the scope of its theoretical concepts, since they were basically based on studies of individual evaluations; which theory could better serve evaluative practice and the proper use of evaluation? Is the solution to generating more robust evaluation theories to create specific theories or to generate them from broader theories in the social sciences? The solutions provided by the literature to these questions converge to a single path, the need for more empirical studies that investigate how the usus of evaluation occur in ifferent evaluation contexts and formats. Therefore, the aim of this study was to investigate the uses and the stimulating and inhibiting factors of the use of the Capes evaluation system. 63 interviews were conducted with coordinators, professors, and Pro-Rectors associated with postgraduate programs in Administration and Accounting evaluated by Capes from 2013 to 2016, using a qualitative research approach and a strategy classified as a basic qualitative study. The main results were: a) identification of a new source of evaluation use called, in this study, use of guidelines; b) denial of the incompatibility between process use and evaluations carried out through evaluation systems; c) contestation of the understanding that evaluation systems are not capable of changing the status quo of the evaluated programs; d) ratification of the idea that uses generate other uses, but the uses generated from the process presented greater potential for this; e) existence of a diversity of stimulating and inhibiting factors for the uses of the Capes evaluation system, which were aligned with the categories of factors given by Alkin and King (2017); and f) evidence that the factors reported by the interviewees follow a hierarchy of influence, in this order: user factors, evaluation factors, context factors, and evaluator factors. It is concluded that evaluation systems can generate uses of the process, just like individual evaluations, as long as there is an element that creates the conditions for involvement, which in the case of the Capes evaluation system is the peer evaluation system. In addition, the incorporation of the new source of use - guidelines - opens up new avenues of research and raises questions about the existence of underreporting of uses in the literature, especially in studies carried out in a context with a low level of involvement.