2014 PRIM&R Poster

Turnaround Time Poster F11-6-14

Click here for a PDF version of the poster

The Impact of Submission Assistance on IRB Review Turnaround Time

Problem/Challenge:

Turnaround time is a common measure used when evaluating IRB operations. Organizations may even establish “target” turnaround times for their research community; however, these times often do not account for variability in the quality of IRB submissions. With this in mind, the Harvard Longwood Campus Quality Improvement Program (QIP) set out to improve IRB submissions to facilitate the IRB review and approval process.

Solution: 

QIP provides a variety of human research support services, including IRB Submission Assistance. This service provides investigators and their study staff general consultation as well as hands-on assistance preparing IRB applications, responding to the IRB’s queries, and drafting and editing study documents (e.g., protocol, recruitment materials, and consent documents). IRB Submission Assistance has been a popular service with our research community. In the past year, our office responded to more than 170 formal requests for Submission Assistance.

While qualitative data, including customer service surveys and informal feedback, have demonstrated a positive and enthusiastic response to this service, quantitative data also confirm that Submission Assistance facilitates the IRB review and approval process.  This is evidenced by shorter turnaround times for assisted submissions.

It is helpful to describe our entire Human Research Protection Program (HRPP) in order to provide some additional context for our turnaround times – both with and without Submission Assistance.

The Harvard Longwood Campus IRB research portfolio is diverse and includes both social behavioral and educational research (SBER) and biomedical research of which approximately 15% is conducted oversees in developing countries. Of the total sample analyzed for this project, 68% was reviewed on an expedited basis, 16% determined exempt; 14% determined to be not human subjects research, and 2% reviewed by the convened IRB.

The Harvard Longwood Campus Office of Human Research Administration (OHRA), which oversees human research conducted at the Harvard T. H Chan School of Public Health, Harvard Medical School and Harvard School of Dental Medicine, employs a one-stop-shop model for carrying out IRB reviews.  This model has been implemented at Harvard T. H. Chan School of Public Health (Harvard Chan School) since 2008, and Harvard Medical School and Harvard School of Dental Medicine since 2013. The impetus behind implementing such a model was initially to improve efficiency and strengthen IRB-investigator relationships. This model requires that IRB staff, i.e., “IRB Review Specialists,” be sufficiently qualified through experience (e.g., Masters-level education, minimum of 3 years direct IRB experience) and expertise (e.g., current CIP designation). Each IRB Review Specialist serves as an IRB member and is designated by the IRB Chair to perform the expedited review procedure (as well as ‘Not Human Subjects Research’ and exempt determinations). In our Office, IRB Review Specialists are assigned workload according to academic department (e.g., Jane Doe reviews all submissions from the Department of Epidemiology). Therefore, IRB Review Specialists follow the submissions for their assigned departments from initial application through study closure, regardless of the type of submission or type of review. In doing so, we have eliminated the need for additional knowledge transfer between submissions.  For those submissions requiring Full Board Review, the IRB Review Specialist (also an IRB member) prepares the submission for the Board ensuring that it is complete, attends the meeting to answer any questions, and then prepares and sends the notification letter to the Principal Investigator.  This method has greatly streamlined the review process by eliminating redundant “pre-reviews” and reducing submission turnaround times.  Additionally, this approach facilitates a strong customer service component by providing a direct point of contact for investigators for all IRB-related questions.  During our AAHRPP re-accreditation site visit in 2013, in addition to the site visitors having no findings, the one-stop-shop model was regarded as one of our Human Research Protection Program’s strengths.

flow-chart1
Flow Chart 1: Harvard Longwood Campus Office of Human Research Administration (OHRA) model for carrying out IRB Reviews.

Quality Improvement Program (QIP):

The Quality Improvement Program is a unit within the Office of Human Research Administration (OHRA), independent of the IRB. QIP’s mission is twofold: 1) to liaise between investigators and the IRB and 2) to provide institutional regulatory compliance.

QIP provides a variety of human research support services for the research community:

QIP also ensures institutional regulatory compliance by:

  • Conducting Directed (for-cause) Audits at the request of the HSPH, HMS/HSDM Institutional Officials. Directed Audits focus on areas of concern that may have been identified by a variety of entities, i.e., federal, state or institutional.
  • Recommending action to the IRB based on onsite observations during Directed (for-cause) Audits.
  • Investigating allegations and findings of non-compliance.
  • Reporting potential serious or continuing non-compliance to the IRB and/or Harvard Chan School, HMS/HSDM Institutional Officials and proposing corrective actions.
  • IRB File Audits
  • IRB minutes assessment
  • IRB membership composition evaluation

Methods/Approach:

Our analysis included a total of 2,096 IRB reviews and determinations made between September 1, 2013 and September 8, 2014. Only applications receiving Submission Assistance between September 1, 2013 and August 31, 2014 were eligible for inclusion in the analysis. The total period under consideration included more than a year to allow for the approval of as many applications as possible.  Study closures, “not engaged” determinations, and protocols ceded to another institution for IRB review were excluded from the analysis. Of the 2,096 reviews, a total of 170 (8.11%) received Submission Assistance prior to, or during, the IRB submission process. 1,926 protocols did not receive Submission Assistance.

Review Type Submission Assistance No Submission Assistance
Full Board 5 31
Expedited 65 1353
Exempt 82 255
Not Human Subjects Research 18 287
Total 170 1926
Application Type Submission Assistance No Submission Assistance
Initial Application 104 540
Modification 45 990
Continuing Review 21 396
Total 170 1926

Results:

Submission Assistance reduced IRB review times. As presented in the five charts below, overall, average approval times for initial applications with Submission Assistance were 14.33 calendar days sooner for Full Board reviews, 9.44 days sooner for Expedited reviews, 10.25 days sooner for Exempt determinations, and 4.34 days sooner for Not Human Research determinations. Expedited modifications/amendments did not show a similar improvement in turnaround time. In examining this group more closely, we found three outlier data points for applications that received Submission Assistance. All belonged to a single investigator whose response time to requests for additional information was on average 50 days. When we controlled for these outliers, Expedited modifications receiving Submission Assistance were approved 2.93 calendar days sooner.

chart 1
Chart 1: Average number of calendar days from submission to approval for all submission types.
chart 2
Chart 2: Average number of calendar days from submission to approval for Full Board Reviews.
chart 3
Chart 3: Average number of calendar days from submission to approval for Expedited Reviews.
chart 4
Chart 4: Average number of calendar days from submission to approval for Exempt determinations.
chart 5
Chart 5: Average number of calendar days from submission to approval for Not Human Research determinations.

The Association for the Accreditation of Human Research Protection Programs, Inc. (AAHRPP) provides aggregate annual performance metrics for all AAHRPP accredited organizations.  The data provided in their 2013 Metrics on Human Research Protection Program Performance report (updated August 1, 2014), the current data available at the time of our analyses, offers turnaround time calculations in median calendar days.  For comparison purposes, we also calculated median calendar days from submission to approval.  Chart 6 shows that, when evaluated against the 2013 AAHRPP performance metrics, approval times for protocols with Submission Assistance were 28 calendar days sooner, 26 days sooner for Expedited reviews, and 15 days sooner for Exempt determinations.  AAHRPP does not provide performance metrics for Not Human Research determinations.

chart 6
Chart 6: Median calendar days from submission to approval.

Conclusion:

Submission Assistance has a positive impact on IRB review turnaround time. In addition to reducing review time, this service promotes communication between investigators and the IRB office and fosters partnership with the research community. Organizations can replicate this model with little or no FTE support. An existing QA/QI program may accomplish this by offering Submission Assistance with a prescribed focus, e.g. consent form review and editing, preliminary review of the protocol prior to IRB submission, or assistance for specific review type (Full Board or Exempt).

Some limiting factors:

  • Investigator response time varies and cannot be controlled.
  • IRB Reviewers may prioritize incoming submissions differently (e.g., responding first to Full Board or continuing review applications rather than modifications or initial applications).
  • Review times for protocols going to the Full Board can be impacted by the meeting schedule, consequently increasing the total time from submission to approval, regardless of the quality of the submission. (The two Harvard Longwood Medical Campus IRBs each meet monthly.)

 

Contact Information:
Lisa Gabel, CIP
QA/QI Specialist
Phone: 617 432-5842
Email: lgabel@hsph.harvard.edu