Evaluation and impact research

The design and implementation of evaluations and impact analyses is part of the core portfolio of the Competence Center Policy and Society (CCP). The entire spectrum of evaluation types is used: In addition to estimates of the expected effects of a funding measure (ex ante) or the final evaluation at the end of a funding measure (summative / ex post), there are evaluations of the start phase of a measure, interim evaluations after a few years of program duration as well as accompanying evaluations with a formative character. Evaluations can include short, medium and long-term impact analyses. We also carry out portfolio evaluations of entire bundles of measures in a thematic area, a funding body or a region, although the latter are still relatively rare in Germany. CCP also has a prominent position in the evaluation and impact analysis of regulation and standards.

The subject of CCP's evaluations are funding measures in the field of research, technology and innovation policy and the research and innovation projects funded by them. We also evaluate networks (clusters, competence centers, research alliances, etc.) and, increasingly, funding bodies, research institutions and research infrastructures.

In addition, the CCP has recently stepped up its activities to establish impact-oriented monitoring systems, including the development of customized indicators for the strategic monitoring of funding activities.

Our evaluations are based on a mix of methods consisting of quantitative and qualitative, reactive and non-reactive approaches. Depending on the object of investigation, primary data collection such as online surveys or expert interviews are combined with secondary data analysis or interactive formats such as stakeholder workshops and focus groups. Text analytical methods and software-supported web research are also increasingly being used.

With the increasing focus on needs and impact orientation (not only) in German research and innovation policy, the objects of evaluation and reference points for assessments are changing. The mission orientation implies a stronger focus of research, technology and innovation policy measures on societal challenges, which, in combination with technological developments, lead to new topics and, in addition to the classic socio-economic effects, also require the measurement of a wide range of other effects (e.g. ecological, personal group-related, political effects). Furthermore, the boundaries between previous target groups of funding measures are disappearing, e.g. due to new initiatives for network-ing science, business and society, resulting in funding constellations with heterogeneous funding recipients.

The developments outlined above give rise to a number of challenges. Evaluations of research, technology and innovation policy must deal with objects of evaluation that have changed significantly in terms of content and structure, which requires an adapted integration of technical expertise and evaluation expertise. In particular, the evaluation of "mission fulfillment" requires an even stronger orientation towards multidimensional impact evaluation (social, economic, environmental, political). The CCP, in close cooperation with the other CCs of the Fraunhofer ISI, sees itself in a central role here and can already present various relevant reference projects in this regard (see below, FONA accompanying research and evaluation, accompanying research on the High-Tech Strategy).

CCP is an active part of the international evaluation community and is involved in numerous networks and expert forums. These include, in particular, membership of the DeGEval - Gesellschaft für Evaluation e.V. and the co-organization of the bi-annual REvaluation conferences.