Skip to main content

EVALUATION.

Common Fund Evaluation and Assessment

Assessment and Evaluation Philosophy

The NIH Common Fund believes assessment and evaluation are essential to planning, improving, and maintaining the quality of our science programs. They can ensure accountability of programs and build a case for future science investments. These webpages provide access to formal evaluations, bibliometric analyses, and ongoing programmatic assessment and describe our assessment approach and methodologies.

To obtain a 360-degree view of complex systems, we use multiple sources of information, employ multi-platform, mixed-method designs. This includes triangulation of data from multiple sources of data gathered during program planning, implementation, and close-out. Data sources include input from science experts, NIH leadership and staff, tool and resource users, publication analysis, portfolio review, landscape analysis, site visits, program documents, and others.

Evaluation Report Library

We have included publicly available evaluations and assessments of Common Fund programs. The following list is ordered by the Common Fund program name or topic for cross-program evaluations.

Expand All to search page using Ctrl+F |
I. Common Fund Program Evaluations

Bioinformatics and Computational Biology Program (which supported the National Centers for Biomedical Computing) 

National Centers for Biomedical Computing Mid-Course Program Review Report (July 13, 2007)
An external panel assessed the status and progress of the National Centers for Biomedical Computing Initiative three years after the program started. This report presents the panel’s findings and provides guidance for the future course of the program.


Bridging Interventional Development Gaps (BrIDGs) (now housed in the National Center for Advancing Translational Sciences (NCATS)) 

NIH-RAID Pilot Mid-Course Review Meeting Summary (March 7, 2008) -part of Bridging Interventional Development Gaps (BrIDGs)
This review of the NIH Rapid Access to Interventional Development (RAID), an initiative of the BrIDGs program, was conducted by an external panel. The report provides the findings and recommendations that include how to improve the approach to this area of science and the management of the program.


High-Risk, High-Reward Research Program

(Four unique NIH Director’s awards are available for exceptionally creative scientists who propose highly innovative approaches with high-impact potential to major challenges in biomedical research.)

NIH Director’s New Innovator Award

Feasibility Study of an Outcome Evaluation of the National Institutes of Health’s New Innovator Award Program – Final Report (May 2011)
To determine whether an evaluation was warranted and feasible, staff interviews were conducted, literature on how innovative research has been defined and operationalized was reviewed, and a comparison group was identified. This resulted in evaluation questions, an evaluation design, a logic model, and collection of pilot data to guide a future outcomes evaluation.

Process Evaluation of the National Institutes of Health Director’s New Innovator Award Program: FY 2007–2009 – Final Report (May 6, 2011)
After reviewing the origins of the program and changes over the first three years, examining the characteristics and perceptions of applicants and external reviewers, and analyzing the scoring of the applications, the findings indicated that the program had been implemented without significant challenges. Six recommendations were made to improve the program.

National Institutes of Health Director’s New Innovator Award Outcome Evaluation Fiscal Years 2007-2009 (2016)
This outcomes evaluation examined the initial awardees of the 5-year New Innovator Award (NI) program. Two key questions were addressed: 1) Is NI research significantly more innovative, high risk, or impactful than traditionally funded NIH research; and 2) What are the impacts of NI awards on the careers of awardees compared to the career impacts of a comparable traditional NIH award?

NIH Director’s Pioneer Award

FY 2004 – 2006 NIH Director’s Pioneer Award Process Evaluation – Comprehensive Report – Final Report (January 2008)
As a pilot, the NIH Director's Pioneer Award (NDPA) continued to evolve and change structurally and conceptually. Using administrative data, interviews, and surveys, this report summarized NDPA’s design, implementation, and participation. The findings included recommendations to continue to clarify program criteria and operations; increase consistency, communication, and transparency of the selection process; and attract a diverse pool of investigators to apply to the program.
 
FY 2004 – 2008 NIH Director’s Pioneer Award Process Evaluation – Comprehensive Report Final Report (January 2010)
This report summarizes the process evaluations of the first 5 years of the program, highlighting changes in the program’s design and implementation and describing program participants’ perceptions. Also included are overall assessments of the program and key recommendations.
 
Outcome Evaluation of the National Institutes of Health (NIH) Director’s Pioneer Award (NDPA), FY2004 2005 – Final Report (July 22, 2011)
The 5-year NDPA awards represent a novel approach for supporting biomedical and behavioral research. This evaluation examined the outcomes of the first two cohorts of the NDPA. Two over-arching questions drove this evaluation: (1) Did the awardees conduct pioneering research with the funds? (2) What are the spillover effects of the program?

An Outcome Evaluation of the National Institutes of Health (NIH) Director’s Pioneer Award (NDPA) Program, FY 2004–2006 (August 2012)
Expanding on the previous outcome evaluation of the NDPA, this evaluation focused on scientific publications to answer questions: (1) To what extent does this research produce unusually high impact, (2) To what extent are the research approaches used highly innovative, (3) To what extent is this research interdisciplinary, and (4) To what extent are the Pioneers awardees collaborative? 

NIH Director’s Transformative Research Award

FY 2009-2010 NIH Director’s Transformative Research Award Process Evaluation (August 2010)
For this evaluation, Transformative Research Award applicants and application reviewers were surveyed. Based on the findings, recommendations about the application and review process were made.


Interdisciplinary Research Program

Facilitating and Experiencing Interdisciplinarity in Biomedical Research – Mid-course Evaluation of the Interdisciplinary Research Consortium Program: an NIH Common Fund Program (Sept. 2011)
This mid-course program review identified facilitators and inhibitors of interdisciplinary at the project level.


Molecular Libraries and Imaging Program

NIH Roadmap Molecular Libraries and Imaging Program Mid-Course Review Meeting, December 20–21, 2006 – Executive Summary
This progress review made the recommendations to (1) focus the program on difficult or unique problems as an organizing theme to drive innovation and differentiation from drug discovery screening efforts in industry, (2) manage the program as a diversified portfolio of initiatives, and (3) reassess the program and chart the overall direction at the 5-year point.

NIH Roadmap Molecular Libraries and Imaging Program Needs Assessment Report – Final Report (January 2010)
This needs assessment examined whether the program accomplished its goals during the initial pilot phase, and gathered feedback from network users and potential users on their level of satisfaction with Molecular Libraries Program services.


Nanomedicine Program

Evaluating the Selection Processes for the NIH Roadmap Nanomedicine Initiative Nanomedicine Development Centers (2005-2006) 
This evaluation examined the processes for selecting the Nanomedicine Development Centers. This included identifying which aspects of the selection process facilitated the solicitation and identification of applications best suited to meet the objectives of the program. Also, the use of Flexible Research Authority for selecting Centers was assessed.

Post-Award Management of the NIH Nanomedicine Development Centers (2008) 
As an extension of the 2005-2006 evaluation of the Nanomedicine Development Centers, this evaluation focused on the effectiveness and efficiency of management and operations at the program level.

Midcourse Review of the NIH Nanomedicine Roadmap Initiative – (2009) 
In 2009, an external panel of scientists was convened to examine questions related to the structure, management, and direction of the program. The panel identified successes and challenges of the NIH Nanomedicine Program and made recommendations to improve the program.


Patient-Reported Outcomes Measurement Information System (PROMIS) Program

The PROMIS Initiative and Mid-Course Review – 2007
This process evaluation was conducted by an external panel of scientists to examine if the goals were being achieved and if the program continued to be relevant and significant in relation to re-engineering the clinical research enterprise. The panel made recommendations to support the initiative’s continued relevance.

II. Common Fund Cross-Cutting Evaluations

Common Fund Evaluation

NIH Council of Councils Common Fund Evaluation (2014)

The NIH Common Fund (also known as the NIH Roadmap) was established in 2004. This 2014 process evaluation examined the planning and management of the Common Fund. Two major questions were answered: (1) Are planning processes optimal for identifying program areas that meet the Common Fund criteria? (2) Are management and oversight processes optimal for achieving program goals?


Common Fund Patent Analysis

Common Fund Patent Report (2017)

The Common Fund is a unique funding entity at NIH, functioning as a “venture capital” space, where high-risk and innovative research can be supported. This led to the question, does research supported by the Common Fund generate more patents compared to the entire NIH research portfolio. Read the full patent report to see patent numbers from Common Fund programs compared to the NIH and to learn about select Common Fund patents.

Presentations

This page includes posters, slide handouts, and videos about evaluations and assessments from the NIH Common Fund. This list is ordered by date.

BEST DATA

Broadening Experiences in Scientific Training (BEST) Cross-site Evaluation Data

Submissions for this form are closed.

BEST Data Files

The Strengthening Biomedical Research Workforce program issued five year awards in 2013 and 2014. These “Broadening Experiences in Scientific Training” awards (aka BEST awards) were designed to develop sustainable approaches to broaden graduate and postdoctoral training, aimed at creating training programs that reflect the range of career options that trainees may ultimately pursue. The awardee sites were part of a comprehensive cross-site evaluation designed to understand trainee agency, time to desired careers, and sustainability of BEST activities and resources at academic institutions to support BEST activities. The data available here are from this evaluation.

Registration

Prior to downloading any Cross-site data, users will be prompted below to register a name and valid e-mail address in order to download data and to accept their responsibility for using data in accordance with the Data Use Agreement.

Protection of Human Subjects

The primary concern in sharing data is the protection of human subjects. The rights and privacy of people who participate in NIH-sponsored research must be protected at all times. Thus, data on this site have been de-identified to prevent linkages to individual research participants. This includes removal of all Personally Identifiable Information (PII) and indirect identifiers that are not listed as PII but could lead to “deductive disclosure” such as comment fields and educational institution.

Cross-Site Evaluation Data Collection Instruments

For information on data collected by the cross-site evaluation, please see the Data Definitions and Data Collection tools, which include:

  • Entrance Survey for Graduate Students
  • Entrance Survey for Postdoctoral Scientists
  • Exit Survey for Graduate Students
  • Exit Survey for Postdoctoral Scientists
  • Data Form
  • Phone Interview Questions

Additional Information

The recommended citation for the data is:

  • National Institutes of Health Office of Strategic Coordination. (2020) Public Use Data from the Broadening Experiences in Scientific Training (BEST) Award Trainee Surveys and Awardee Data Forms.

This page last reviewed on September 1, 2023