70 - Blinded Protocol Exercises as a Structured Quality Improvement Tool in IRB Deliberation Review

Background:  To assess deliberation consistency and enhance quality in IRB review processes, our institution developed a structured internal quality assessment strategy using blinded protocol exercises, implemented annually with evolving themes.

Program

Description:  Each year, a mock protocol is developed to reflect timely regulatory or ethical issues (e.g., e-consent, exempt determinations). All identifiable information is removed, and the protocol is submitted to multiple IRB panels under blinded conditions. Committee members review the protocol independently, unaware that it is a simulated exercise. After deliberation, outcomes and comments are compared to assess variance in review focus, decision points, and interpretation of regulations. Themes are chosen annually based on current challenges or recurrent investigator inquiries. The blinded review tool has been integrated into broader institutional quality assurance efforts and used to inform continuing education, policy clarification, and the development of shared review standards across review bodies.

Program

Assessment:  Findings from the past three years demonstrate consistent approval outcomes across committees, with notable variation in the specific issues raised during review. For example, in a protocol addressing e-consent, some committees emphasized data privacy and identity verification, while others focused more on comprehension and voluntariness. In another year’s protocol on exempt research, there were differences in the application of exemption criteria and documentation requirements. These differences provided insight into interpretive tendencies among reviewers, often shaped by disciplinary background or prior training. Follow-up debriefings allowed committees to discuss diverging approaches, promoting mutual understanding and identifying areas where harmonization or further training might be needed. The exercise also served as a low-risk environment for new IRB members to observe and practice deliberation without regulatory pressure. Staff coordinators found the process helpful for aligning administrative practices and clarifying review expectations.

Limitations: As only one mock protocol is used each year, generalizability is limited. Additionally, knowing that the protocol is part of a quality exercise may influence how rigorously reviewers engage with the content. The process does not fully replicate the pressures of live deliberation, and not all committees may invest equal effort.

Discussion:  Blinded protocol review has proven to be an effective and sustainable tool for internal QA/QI in IRB operations. The annual thematic structure keeps reviewers engaged and reflects real-world ethical tensions. The model can be adapted by other institutions seeking to evaluate decision-making variability, support peer learning, and improve consistency. It also offers a platform for structured education of new IRB members and staff, and for identifying opportunities to clarify internal policies and guidance documents.