Clinical Trial Optimization

How are benchmarks on protocol design practices changing and what’s the impact of the exploding number data points in clinical trials? That’s one of the important issues discussed in Ken Getz’s  interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). Getz is the newly appointed director and a professor at the Tufts Center for the Study of Drug Development (CSDD), Tufts University School of Medicine. Getz discusses the results of a recently released study by the Tufts CSDD, which updated previously published benchmarks while also exploring new clinical trial developments, including the collection of data points. One key takeaway from the study, according to Getz, is that when you connect longer cycle times, higher costs, and poor retention, you must arrive at one inescapable conclusion; It’s becoming more difficult to predict how any given protocol will perform and whether it will be meet expected deadlines. Another key takeaway: In typical  Phase 3 clinical studies, there are now more than 3.5 million data points, which is about three times the number of data points collected ten years ago in similar studies. A final takeaway, Getz notes, is the challenge they encountered benchmarking the impact of protocol complexity on study timelines because many companies utilize “moving plan targets”. As far as post-COVID clinical trials are concerned, the space will continue to be complex and customized and better outcomes will be achieved only if we revise standards operating procedures, according to Getz.

How can you best measure whether vendors are meeting study milestone goals? That’s one of the important issues discussed in Keith Dorricott’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC).  Dorricott is a Director at Dorricott Metrics & Process Improvement Ltd in the UK and facilitates many MCC metric development and community of practice groups. Timeliness metrics — measurements that assess whether study milestones are achieved by their target dates —- are valuable but don’t always tell the entire story, according to Sullivan and Dorricott, such as whether the milestone dates are realistic and how on-time measurements influence human behavior. Then, when you move from measuring a single study to cross-studies or portfolio level, aggregation of data can get even more complicated. There are many ways to “cut” the data, they note, depending on the performance questions you want answered. With timeliness, Dorricott says, it always comes back to expectations — and whether those expectations are reasonable. In this respect, measuring how long it takes to achieve milestones is important and should be reviewed to determine if timeliness expectations are achievable. In terms of rolled-up data, it’s easy to aggregate results across studies but it’s important to determine which studies or process steps are not meeting expectations and need attention. When you aggregate results from a large number of studies, it can be challenging to see when you have a few studies that are not performing well. For this reason, the MCC Vendor Oversight Metric Toolkit includes several portfolio-level milestone timeliness metrics that answer difference performance questions, such as what the overall performance of the portfolio is in terms of milestones met across active studies, performance over the last 3 months and performance reported by study phase. Additional information about the MCC Vendor Oversight Community of Practice group is available on the MCC website (www.metrcischampion.org).

How can sponsors and CROs assess relationships through the use of streamlined surveys and careful review of results? That’s one of the important issues discussed in Keith Dorricott’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC).  Dorricott is a Lean Sigma Master Black Belt with extensive experience in process improvement at CROs and is a Director at Dorricott Metrics & Process Improvement Ltd in the UK. He leads the MCC Vendor Oversight Metric Work Group which examined the topic in detail and produced a comprehensive and useful guidance document on this issue. The guidance document contains tips and tricks, from what questions to include to how to convert responses into insightful metrics and communicate results. The starting point of surveys, Dorricott says, should not be the questions themselves but the purpose of the survey and what you’re trying to achieve by running it.  In the end, Dorricott notes, the group determined that there were a handful of strategic elements to measure. They considered these strategic areas and defined five relationship assessment questions. While additional questions can be asked, smaller surveys, he says, respect the time restrictions of those completing the survey and are likely to have a higher response rate – which is important for the validity of the survey.  The MCC website, metricchampion.org, contains a White Paper that explains the guidance document, https://metricschampion.org/mcc-white-papers-index/

How can clinical trial organizations achieve TMF excellence? That’s one of the important issues discussed in Sholeh Ehdaivand’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). Ehdaivand, President and CEO of LMK Clinical Research Consulting, a certified woman-owned TMF consulting company, notes that the TMF (Trial Master File) is the collection of documents and data associated with the clinical trial. Often referred to as “the story of the clinical trial,” it is designed to serve the needs of all stakeholders, including sponsors, sites, regulators, and auditors. The TMF serves as confirmation that participant rights have been protected and that the data are reliable, Ehdaivand says. While that may seem like a simple concept, she adds, not everyone understands what is actually included in the TMF. The TMF is not a “document dump,” but should be an active file containing all relevant and, especially in 2020, applicable data, according to Ehdaivand. Her company’s motto is “if it isn’t written down, it didn’t happen.” She also notes that the TMF should be compiled and organized in a timely manner and in compliance with regulations from governing authorities. She explains how the organization of the documents in the TMF is just as important as the documents themselves. Sullivan and Ehdaivand discuss LMK’s launch of TMF University in January 2021, the first and only internationally accredited TMF training program in the industry. The curriculum includes completion required assessments to ensure that students truly learned and fully understand the components of an inspection-ready TMF. The certification program is designed to elevate the importance of the TMF so that stakeholders will not just look at it as a place where “documents go to die.” There are three learning levels within the TMF University accreditation program: Discovery level, which covers the fundamental topics concerning the TMF; Immersion level, which covers essential topics that are critically important to TMF quality; and Mastery level, which covers advanced topics of TMF Management.  Completion of learning levels one and two (six courses) along with the satisfactory completion of the required assessments will earn students the internationally recognized status and credentials of Qualified TMF Associate, while the completion of all three learning levels (9 courses) and assessments will earn students the internationally recognized status and credentials of Qualified TMF Manager.

Can metrics play a crucial oversight role in establishing an effective and efficient relationship between sponsors and CROs? That’s one of the important issues discussed in Steve Crow’s and Keith Dorricott’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). Crow, Associate Director, Performance and Training, Clinical Operations at GW Pharmaceuticals (known as Greenwich Biosciences in the U.S.) explains that the growth of the company’s portfolio of studies led them to seek CRO services - which began their journey of establishing a core set of CRO metrics to assist with oversight. The organization joined MCC to learn about metrics and gain access to the metrics toolkits available to members. His advice to other sponsors in similar situations, “Don’t reinvent the wheel … MCC has the resources you need to get started”. He notes that it’s important to find a “common purpose” by focusing on the key questions you seek to answer rather than giving CROs a list of performance metrics. Opening the discussion with key performance questions that are important to your organization results in enthusiastic discussion, collaboration, and the removal of barriers, explains Crow. Keith Dorricott, a Lean Sigma Master Black Belt with extensive experience in process improvement at CROs, is now Director at Dorricott Metrics & Process Improvement Ltd in the UK, notes that metrics are only beneficial if they answer specific performance questions. Indeed, Sullivan says, that’s the basic approach MCC advocates by including key performance questions in the metric toolkits and implementation support tools available to MCC members. Finally, Dorricott and Sullivan point out, context is vital not only when selecting metrics but also paramount when reviewing results. Collaborators should remind themselves of why they’re looking at the data, what the key questions were, and what actions should be considered if performance does not meet expectations. The metrics should drive decisions and action. Otherwise, they note, they’re just looking at numbers on a dashboard.

What’s the best way to get rid of unproductive waste in clinical trials? That’s one of the important issues discussed in Keith Doricottt’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). Keith Dorricott, a Lean Six?? Sigma Master Black Belt with extensive experience in process improvement at CROs, is now Director at Dorricott Metrics & Process Improvement Ltd. in England.  This sentence makes no sense. Dorricott describes the origins of the Total Cost of Quality Model in the US and its early adoption in Japan. He also discusses the basic idea that there is a balance between good and poor cost of quality – there are costs to building quality in, but much greater costs when quality does not meet the required standard. The costs of quality failure  can include protocol amendments, low enrollment, extended timelines, rework, reputational damage and delay to market. Dorricott suggests- that there be greater accountability for those responsible for certain processes rather than a simple reliance on others to find issues later and then have the cost of fixing them. Dorricott says that there are now ways to estimate the costs of poor quality, and that indeed the biggest challenge for the clinical trials industry is how to become more efficient and ultimately bring more high-quality therapeutics to market sooner to impact patients’ lives.   

Should Risk-Based Quality Management (RBQM) begin with technology or process? That’s one of the important issues discussed in Gary Tyson’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). Tyson founded Pharma Initiatives eight years ago after the sale of Campbell Alliance to what is now Syneos, where he had built the clinical development and  medical affairs practice area. It’s not unusual for Tyson’s clients to engage his organization to help them select and implement RBQM software systems. But RBQM “is not a classic software implementation” project because RBQM implementation doesn’t start with technology selection. Technology is only one of the three elements of transformative change needed to implement successful RBQM programs – you should take a three-pronged approach that includes people, process, and technology, says Tyson. Organizations need to educate staff about RBQM fundamentals and establish their  RBQM process before considering RBQM software.  Since RBQM is not highly prescriptive in terms of mandates established by regulatory agencies, it is more dependent on the particular needs of the clinical trial stakeholder. And small sponsors should engage with their CROs to establish how they will work together to analyze risk and implement risk controls. In addition, says, Sullivan, your CAPA process should also be connected to RBQM. Tyson suggests to his clients that they begin RBQM with a pilot project, using affordable tools available on the market today.

How can sponsors, CROs, and sites improve the performance of clinical researchers? That’s one of the important issues discussed in SAM Sather’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). Sather, Cofounder and Vice President of Clinical Pathways (CP), LLC, a consulting firm in North Carolina, leads the quality management consulting services for the company. She talks about the critical importance of human performance improvement (HPI) for the clinical trials industry, a science that is particularly significant today given the integration of quality risk management and GCP. HPI is about structuring your organization in ways that enable your employees to be star performers and engage in critical thinking, Sather says. “It’s based on doing a business analysis and defining what matters … and is very linked to metrics and what is an enforceable agreement, and [involves] the concepts of gap analysis and root cause as well.” she explains. One HPI model successfully used by Sather is Gilbert’s Behavior Engineering Model (BEM), which considers factors on two levels: environmental and individual. All factors included in these levels are equally important and need to be present for individuals to perform well. For example, organizations implementing risk-based management should consider whether it truly provides team members with the information, resources, incentives, knowledge, capacity, and motivation they need to succeed. The pandemic has pushed the industry to be agile and some stakeholders have succeeded, and others haven’t, she concludes.

How can CROs and sponsors detect potential problems and issues in clinical trial conduct and data quality? That’s one of the critical issues discussed in Steve Young’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). Young, Chief Scientific Officer at CluePoint, discusses how risk-based quality management (RBQM) methodology, data analytics, and data surveillance are being utilized today to go beyond just ferreting out fraud.  Now, Young says, clinical trial stakeholders are deploying advanced statistical engines to detect quality and data breaches in investigational studies. According to Young, while there are still instances, to varying degrees, of major misconduct in clinical trials, CROs and sponsors are increasingly turning to online solutions to risk planning. It’s important, he adds, that when risks are discovered, the solutions are integrated into the workflow of clinical trials. In fact, Young asserts, one of the biggest challenges facing the industry today is to embrace change and accept the use of centralized monitoring data to aid in the discovery and resolution of risk.        

Interested in attending the MCC Clinical Trial Risk and Performance Management vSummit Sept. 8-10? Visit https://www.centerwatch.com/mcc-summit-2020. If you subscribe to this podcast, you can use the code POD15 to receive 15% off your registration fee.