How we process Collaborative and Knowledge-building Project applications

The calls describe the thematic areas for which research funding can be awarded, as well as the funds available for distribution under each topic. Applicants must tick the thematic area they wish to be assessed against.

All the applications we receive are directly assigned to the topic the applicant has selected in the application form. Examples of topics are ‘circular economy’ or ‘environmentally friendly energy’. If the applicant has selected several of the topics under the call in their application, we will take this into account in the further processing. Please note that calls for a Collaborative and Knowledge-building Project do not have an ‘open arena’ in addition to the topics given under each call.

Joint overarching requirements and guidelines apply to the calls, and all applicants must comply with these. In addition, each topic has special delimitations and strategic priorities that will be important to the assessment of applications.

The applications are assessed in a four-step process:

• First, the Research Council’s administration carries out a preliminary administrative review to check that the applications meet all formal requirements described in the call.
• The applications are then forwarded to the thematic panels for processing. Each panel generally comprises four or five peer reviewers who process five to ten applications in the panel, but the size of the panels and the number of applications they assess will vary somewhat from topic to topic.
• The Research Council’s administration will then consider the extent to which the application is relevant in relation to the call for proposals.
• Finally, the administration carries out a portfolio assessment and presents a recommendation to the relevant portfolio board, which will then make a funding decision.

Here you can read more information about how we process applications. Go straight to:

Reception, preliminary administrative review and rejection of applications  Recruitment of peer reviewers  Mapping of peer reviewers’ expertise and impartiality  The peer reviewers’ tasks before and during the panel meeting

Assessment of relevance  Overall mark Portfolio assessment and decision on allocation

Reception, preliminary administrative review and rejection of applications 

As soon as the Research Council has received the applications, it will conduct a preliminary administrative review. Applications that do not meet the formal requirements described in the call may be rejected.

The following formal requirements are checked:

• Is the applicant an approved research organisation?
• Are the application and all attachments written in English?
• Has the project description been attached, and has the correct template been used?
• Has the project manager’s CV been attached?
• Does the project involve at least two Norwegian partners?
• Does the project manager meet the competence requirements?
• Does the application meet the requirements relating to partners’ contribution or cash financing?
• Has a Letter of Intent from the partners been enclosed?

Recruitment of peer reviewers 

Well before the application deadline, we start the work of finding international peer reviewers to consider the applications. However, this work is only complete once the applications have been received and sorted under their respective topics.

The peer reviewers we use must meet a number of general requirements:

• Several peer reviewers in the panel must be active researchers who have produced a significant body of work in terms of both quantity and quality. In a Collaborative and Knowledge-building Project, panels can include members from outside academia with good industry know-how about the research challenges in question.
• Researchers should have professor qualifications. The minimum requirement is associate professor qualifications or similar. 
• The peer reviewers’ place of work should preferably be located abroad.

We also aim to avoid using the same panels several years in a row. Ideally, a panel should have some peer reviewers who have participated before while others have been replaced.

To find new experts, we use various databases and tools, such as Expert Lookup. We perform automated searches on the basis of project titles and summaries as well as manual searches. We supplement these with searches on other websites such as the Web of Science and Google Scholar, as well as foreign universities with high profiles in the various fields. We also consider the applicants’ own suggestion for suitable peer reviewers. The list of sources varies from field to field.

Case officers thoroughly assess each of the potential peer reviewers. Our priority is finding good peer reviewers, including someone with broad expertise who can be active in discussions and do a good job on giving feedback to applicants. 

We publish the names of all peer reviewers after the application assessment process has concluded.

Mapping of peer reviewers’ expertise and impartiality 

The panels normally comprises four to five peer reviewers, and never fewer than three. We endeavour to ensure a good gender balance in the panels. The panel’s overall expertise must cover the breadth of the applications as regards topics, disciplines and industry and sector knowledge, and the axis from basic to applied research. The panels are also put together with a view to the need to consider interdisciplinary applications.

Before the peer reviewers are given access to the titles and project summaries of the applications assigned to the panel, they must consent to a confidentiality statement. They then declare their level of expertise on the basis of this information.

The peer reviewers indicate one of the following levels of expertise for each application:

• Specialist: The proposal is within your primary area(s) of expertise. You are well qualified to evaluate the proposal.
• Generalist: You have a general knowledge of at least one of the main subjects of the proposal. You are qualified to evaluate the proposal.
• Minor: You have only minor relevant expertise on the main subjects of the proposal.

The Research Council's expertise requirement states that at least two of the peer reviewers must have generalist or specialist expertise in the application’s field of research. If the mapping of the panel’s expertise shows that some applications are not adequately covered, we will bring in additional external peer reviewers with special expertise. The peer reviewers we bring in to assess individual applications will do this on par with the ordinary peer reviewer panel members.

In addition to the mapping of expertise, the peer reviewers will be sent a list of all project participants mentioned in the applications the panel will assess. This also includes project participants affiliated to partners in the project. To be permitted to assess the application, the peer reviewers must declare their impartiality in relation to everyone participating in the project. The peer reviewers are asked to pay special attention to the points in the impartiality provisions that concern collaboration, friendships and conflicts.

See our regulations on impartiality and confidence.

The peer reviewers’ tasks before and during the panel meeting

All panel members read and assess all applications to be considered by the panel. The goal of the panel assessment is to arrive at a consensus-based assessment of the three main criteria Excellence, Impact and Implementation. The assessment consists of numeric marks, in addition to a comment explaining why the panel chose this mark.

An application consists of the application form, project description, the CVs of the project manager and key project participants, as well as Letters of Intent. If the applicant has appended any other information, the Research Council’s administration ensures that these documents are removed from the material sent to the peer reviewers to ensure that the applications are assessed on the same grounds.

Before the panel meeting, there is a preliminary meeting, the main purpose of which is to ensure good technical implementation and to give the panel members an opportunity to ask questions. The questions can concern the call itself, the topic or the understanding of or emphasis on the marks.

The panel members submit their individual assessment of each application prior to the panel meeting. The individual assessments will not be forwarded to the applicant, but will be shared between the panel members when everyone has submitted their assessments and form the basis for the discussions during the panel plenum meeting.

For each application, one of the panel members will be assigned special responsibility as principal assessor. The principal assessor usually has specialist expertise in the application’s thematic area, and has a special responsibility for the discussion in the panel meeting and for compiling the panel’s assessments.

The members attend the panel meeting with their different professional views and interpretations of the scale of marks and will also emphasise the various elements of the application differently. The members discuss each application and consider them from various perspectives, and the panel reach a consensus-based assessment and a joint understanding of the scale of marks.

Research Council employees with responsibility for the panels will not participate in the scientific discussion. Their role is to provide guidance and ensure progress, contribute to a joint understanding of the assessment criteria and scale of marks, and ensure that everyone gets to speak. They also handle impartiality according to the rules and stop discussions about matters that fall outside the panel's remit.

When peer reviewers are formally appointed to serve as a panel member, they receive information about the administrative procedure through the document Guidelines for peer reviewer panels Collaborative and Knowledge-building project (pdf).

Assessment of relevance 

One important objective for a Collaborative and Knowledge-building Project is to develop R&D competence in Norwegian R&D groups in relation to topics of importance to Norwegian societal actors and Norwegian industry. We also want to stimulate collaboration between the research groups and those representing the societal challenge for which funding is sought. The call and the topics therefore include requirements and guidelines to that end.

The Relevance criterion ensures that the requirements and guidelines described in the call are followed up. The Research Council’s administration carries out the assessment of relevance, which is intended to supplement the peer reviewers’ assessments.

In principle, all applications awarded a mark of 4 or better from the peer reviewers on all the criteria will be assessed for relevance. For some thematic areas, however, the application scope will be so large that the threshold for assessment of relevance will be higher.

In the assessment of relevance, we use a standardised assessment table with the same scale of marks as for the other criteria. A brief written comment will also be included in the feedback to applicants once the application assessment process has concluded.

The assessment table for the Relevance criterion consists of the following points:

• The extent to which the project satisfies the thematic approach the topic requires
  • The extent to which the project satisfies the thematic guidelines and delimitations

• The extent to which the project satisfies the requirements and characteristics set out in the call and that otherwise apply to the topic
  • The extent to which the project satisfies the requirements relating to project partners
  • The extent to which the project satisfies the purpose of knowledge-building in the research groups

Overall mark

The overall mark is the average of the marks awarded for the four main criteria, i.e. the peer reviewers’ three criteria and the administration’s assessment of relevance. The four main criteria are equally weighted.

Portfolio assessment and decision on allocation 

For each topic, our goal is to have a comprehensive project portfolio based on the information given to the applicants through the call and the relevant plans referred to therein. The administration therefore carries out an overall assessment of the portfolio. This assessment does not affect the mark awarded to the applications, but is a tool used to prioritise between applications whose ranking is relatively similar after the assessment of individual applications.

The portfolio assessment takes the following factors into account:

• The applications’ assigned marks based on the assessments
• A good distribution of projects in relation to priorities set out for the specific topic
• The relative volume and quality of grant applications under other calls within the same thematic area in the same application year
• Any changes in the financial or scientific framework set by the ministries
• Priority will be given to projects led by women project managers when the applications are otherwise considered to be on a par

Based on the peer reviewer panel’s assessment, the assessment of relevance and the portfolio assessment, the administration writes a recommendation for the portfolio board on whether to grant or reject the application. Applications targeting more than one topic can be included on several lists of recommendations given to different portfolio boards. The Research Council’s administration ensures that the information provided enables the portfolio board to quality assure that the processing of applications has complied with the stipulated procedures, and that all applications have been properly assessed.

The portfolio board will be presented with the recommendation documents for all applications, including those that have not been assessed for relevance and in principle are not eligible for funding. They are also sent overviews showing the marks awarded to the applications for the individual criteria.

The recommendations (one for each application) are intended to give the portfolio board good insight into the assessments made. The recommendations are also intended to make the portfolio board aware of applications where the assessment of relevance and/or portfolio assessment has had a major bearing on the application’s overall ranking. In this way, the portfolio board can quality assure the administration’s assessments. The portfolio board’s discussion may result in decisions that do not correspond with the recommendation the Research Council has submitted.

The Research Council has 11 portfolio boards with variation in the budgets at their disposal, the number of application types announced, the number of applications for processing, thematic breadth and the expected number of applications granted.