JCO PO author Dr. Dean A. Regier at the Academy of Translational Medicine, University of British Columbia (UBC), and the School of Population and Public Health, BC Cancer Research Institute shares insights into his JCO PO article, “Clinical Effectiveness and Cost-Effectiveness of Multigene Panel Sequencing in Advanced Melanoma: A Population-Level Real-World Target Trial Emulation.”
Host Dr. Rafeh Naqash and Dr. Regier discuss the real-world clinical effectiveness and cost-effectiveness of multigene panels compared with single-gene BRAF testing to guide therapeutic decisions in advanced melanoma.
Transcript
Dr. Rafeh Naqash:Hello and welcome to JCO Precision Oncology Conversations, where we bring you engaging conversations with authors of clinically relevant and highly significant JCO PO articles. I'm your host, Dr. Rafeh Naqash, Podcast Editor for JCO Precision Oncology and Assistant Professor at the OU Health Stephenson Cancer Center in the University of Oklahoma.
Today, we are excited to be joined by Dr. Dean A. Regier, Director at the Academy of Translational Medicine, Associate Professor at the School of Population and Public Health, UBC Senior Scientist at the British Columbia Cancer Research Institute, and also the senior author of the JCO Precision Oncology article entitled "Clinical Effectiveness and Cost-Effectiveness of Multigene Panel Sequencing in Advanced Melanoma: A Population-Level Real-World Target Trial Emulation."
At the time of this recording, our guest's disclosures will be linked in the transcript. Dean, welcome to our podcast and thank you for joining us today.
Dr. Dean Regier:Thank you. I'm delighted to be here.
Dr. Rafeh Naqash:So, obviously, you are from Canada, and medicine, or approvals of drugs to some extent, and in fact approvals of gene testing to some extent is slightly different, which we'll come to learn about more today, compared to what we do in the US—and in fact, similarly, Europe versus North America to a large extent as well.
Most of the time, we end up talking about gene testing in lung cancer. There is a lot of data, a lot of papers around single-gene panel testing in non-small cell lung cancer versus multigene testing. In fact, a couple of those papers have been published in JCO PO, and it has shown significant cost-effectiveness and benefit and outcomes benefit in terms of multigene testing. So this is slightly, you know, on a similar approach, but in a different tumor type. So, could you tell us first why you wanted to investigate this question? What was the background to investigating this question? And given your expertise in health economics and policy, what are some of the aspects that one tends or should tend to understand in terms of cost-effectiveness before we go into the results for this very interesting manuscript?
Dr. Dean Regier:Yeah, of course, delighted to. So, one of the reasons why we're deeply interested in looking at comparative outcomes with respect to single- versus multigene testing— whether that's in a public payer system like Canada or an insurer system, a private system in the United States— is that the question around does multigene versus single-gene testing work, has not typically tested in randomized controlled trials. You don't have people randomized to multigene versus single-gene testing.
And what that does, it makes the resulting evidence base, whether it's efficacy, safety, or comparative cost-effectiveness, highly uncertain. So, the consequence of that has been uneven uptake around the world of next-generation sequencing panels. And so if we believe that next-gen sequencing panels are indeed effective for our patients, we really need to generate that comparative evidence around effectiveness and cost-effectiveness. So we can go to payers, whether it be single payer or a private insurer, to say, "Here are the comparative outcomes." And when I say that uptake has been uneven, uptake there's been actually plenty, as you know, publications around that uneven uptake, whether it be in Europe, in the United States, in Canada. And so we're really interested in trying to produce that evidence to create the type of deliberations that are needed to have these types of technologies accessible to patients. And part of those deliberations, of course, is the clinical, but also in some contexts, cost-effectiveness.
And so, we really start from the perspective of, can we use our healthcare system data, our learning healthcare system, to generate that evidence in a way that emulates a randomized controlled trial? We won't be able to do these randomized controlled trials for various, like really important and and reasons that make sense, quite frankly. So how can we mimic or emulate randomized controlled trials in a way that allows us to make inference around those outcomes? And for my research lab, we usually think through how do we do causal inference to address some of those biases that are inherent in observational data. So in terms of advanced melanoma, we were really interested in this question because first of all, there have been no randomized controlled trials around next-gen sequencing versus single-gene testing. And secondly, these products, these ICIs, immune checkpoint inhibitors, and BRAF and MEK inhibitors, they are quite expensive. And so the question really becomes: are they effective? And if so, to what extent are they cost-effective? Do they provide a good reason to have information around value for money?
Dr. Rafeh Naqash:So now going to the biology of melanoma, so we know that BRAF is one of the tumor-agnostic therapies, it has approvals for melanoma as well as several other tumor types. And in fact, I do trials with different RAF-RAS kinase inhibitors. Now, one of the things that I do know is, and I'm sure some of the listeners know, is the DREAMseq trial, which was a melanoma study that was an NCI Cooperative Group trial that was led by Dr. Mike Atkins from Georgetown a couple of years back, that did show survival benefit of first-line immunotherapy sequencing. It was a sequencing study of whether to do first-line BRAF in BRAF-mutant melanoma followed by checkpoint inhibitors, or vice versa. And the immune checkpoint inhibitors followed by BRAF was actually the one that showed benefit, and the trial had to stop early, was stopped early because of the significant benefit seen.
So in that context, before we approach the question of single-gene versus multigene testing in melanoma, one would imagine that it's already established that upfront nivolumab plus ipilimumab, for that matter, doublet checkpoint inhibitor therapy is better for BRAF-mutant melanoma. And then there's no significant other approvals for melanoma for NRAS or KIT, you know, mucosal melanomas tend to have KIT mutations, for example, or uveal melanomas, for that matter, have GNAQ, and there's no targeted therapies. So, what is the actual need of doing a broader testing versus just testing for BRAF? So just trying to understand when you started looking into this question, I'm sure you kind of thought about some of these concepts before you delved into that.
Dr. Dean Regier:I think that is an excellent question, and it is a question that we asked ourselves: did we really expect any differences in outcomes between the testing strategies? And what did the real-world implementation, physician-guided, physician-led implementation look like? And so, that was kind of one of the other reasons that we really were interested is, why would we go to expanded multigene panel sequencing at all? We didn't really expect or I didn't expect an overall survival a priori. But what we saw in our healthcare system, what happened in our healthcare system was the implementation in 2016 of this multigene panel. And this panel covered advanced melanoma, and this panel cost quite a bit more than what they were doing in terms of the single-gene BRAF testing. And so when you're a healthcare system, you have to ask yourself those questions of what is the additional value associated with that?
And indeed, I think in a healthcare system, we have to be really aware that we do not actually follow to the ideal extent randomized controlled trials or trial settings. And so that's the other thing that we have to keep in mind is when these, whether it's an ICI or a BRAF MEK inhibitor, when these are implemented, they do not look like randomized controlled trials. And so, we really wanted to emulate not just a randomized controlled trial, but a pragmatic randomized controlled trial to really answer those real-world questions around implementation that are so important to decision making.
Dr. Rafeh Naqash:Sure. And just to understand this a little better: for us in the United States, when we talk about multigene testing, we generally refer to, these days, whole-exome sequencing with whole-transcriptome sequencing, which is like the nuclear option of of the testings, which is not necessarily cheap. So, when you talk about multigene testing in your healthcare system, what does that look like? Is it a 16-gene panel? Is it a 52-gene panel? What is the actual makeup of that platform?
Dr. Dean Regier:Excellent question. Yeah, so at the time that this study is looking at, it was 2016, when we, as BC Cancer—so British Columbia is a population right now of 5.7 million people, and we have data on all those individuals. We are one healthcare system providing health care to 5.7 million people. In 2016, we had what I call our "home-brew" multigene panel, which was a 53-gene panel that was reimbursed as standard of care across advanced cancers, one of them being advanced melanoma. We have evolved since then. I believe in 2022, we are using one of the Illumina panels, the Focus panel. And so things have changed; it's an evolving landscape. But we're specifically focused on the 53-gene panel. It was called OncoPanel. And that was produced in British Columbia through the Genome Sciences Centre, and it was validated in a single-arm trial mostly around validity, etc.
Dr. Rafeh Naqash:Thank you for explaining that. So now, onto the actual meat and the science of this project. So, what are some of the metrics from a health economy standpoint that you did look at? And then, methodology-wise, I understand, in the United States, we have a fragmented healthcare system. I have data only from my institution, for that matter. So we have to reach out to outside collaborators and email them to get the data. And that is different for you where you have access to all the data under one umbrella. So could you speak to that a little bit and how that's an advantage for this kind of research especially?
Dr. Dean Regier:Yeah. In health economics, we look at the comparative incremental costs against the incremental effectiveness. And when we think about incremental costs, we think not just about systemic therapy or whether you see a physician, but also about hospitalizations, about all the healthcare interactions related to oncology or not that a patient might experience during their time or interactions with the healthcare system. You can imagine with oncology, there are multiple interactions over a prolonged time period depending on survival. And so what we try to do is we try to—and the benefit of the single-payer healthcare system is what we do is we link all those resource utilization patterns that each patient encounters, and we know the price of that encounter. And we compare those incremental costs of, in this case, it's the multigene panel versus the single-gene panel. So it's not just the cost of the panel, not just the cost of systemic therapy, but hospitalizations, physician encounters, etc.
And then similarly, we look at, in this case, we looked at overall survival - we can also look at progression-free survival - and ask the simple question, you know, what is the incremental cost per life-year gained? And in that way, we get a metric or an understanding of value for money. And how we evaluate that within a deliberative priority setting context is we look at safety and efficacy first. So a regulatory package that you might get from, in our case, Health Canada or the FDA, so we look at that package, and we deliberate on, okay, is it safe and is it effective? How many patients are affected, etc. And then separately, what is the cost-effectiveness? And at what price, if it's not cost-effective, at what price would it be cost-effective? Okay, so for example, we have this metric called the incremental cost-effectiveness ratio, which is incremental cost in the numerator, and in this case, life-years gained in the denominator. And if it is around $50,000 or $100,000 per life-year gained—so if it's in that range, this ratio—then we might say it's cost-effective. If it's above this range, which is common in oncology, especially when we talk about ICIs, etc., then you might want to negotiate a price. And indeed, when we negotiate that price, we use the economic evaluation, that incremental cost-effectiveness ratio, as a way to understand at what price should we negotiate to in order to get value for money for the healthcare system.
Dr. Rafeh Naqash:Thank you for explaining those very interesting terminologies. Now, one question I have in the context of what you just mentioned is, you know, like the drug development space, you talked about efficacy and safety, but then on the safety side, we talk about all-grade adverse events or treatment-related adverse events—two different terminologies. From a healthcare utilization perspective, how do you untangle if a patient on a BRAF therapy got admitted for a hypoxic respiratory failure due to COPD, resulting in a hospitalization from the cost, overall cost utilization, or does it not matter?
Dr. Dean Regier:We try to do as much digging into those questions as possible. And so, this is real-world data, right? Real-world data is not exactly as clean as you'd get from a well-conducted clinical trial. And so what we do is we look at potential adverse event, whether it's hospitalization, and the types of therapies around that hospitalization to try- and then engage with clinicians to try to understand or tease out the different grades of the adverse event. Whether it's successful or not, I think that is a real question that we grapple with in terms of are we accurate in delineating different levels of adverse events? But we try to take the data around the event to try to understand the context in which it happens.
Dr. Rafeh Naqash:Thank you for explaining that, Dean.
So, again to the results of this manuscript, could you go into the methodology briefly? Believe you had 147 patients, 147 patients in one arm, 147 in the other. How did you split that cohort, and what were some of the characteristics of this cohort?
Dr. Dean Regier:So, the idea, of course, is that we have selection criteria, study inclusion criteria, which included in our case 364 patients. And these were patients who had advanced melanoma within our study time period. So that was 2016 to 2018. And we had one additional year follow. So we had three total years. And what we did is that we linked our data, our healthcare system data. During this time, because the policy change was in 2016, we had patients both go on the multigene panel and on the single-gene BRAF testing. So, the idea was to emulate a pragmatic randomized controlled trial where we looked at contemporaneous patients who had multigene panel testing versus single-gene BRAF testing.
And then we did a matching procedure—we call it genetic matching. And that is a type of matching that allows us to balance covariates across the patient groups, across the multigene versus BRAF testing cohorts. The idea again is, as you get in a randomized controlled trial, you have these baseline characteristics that look the same. And then the hope is that you address any source selection or confounding biases that prohibit you to have a clean answer to the question: Is it effective or cost-effective? So you address all those biases that may prohibit you to find a signal if indeed a signal is there.
And so, what we did is we created—we did this genetic matching to balance covariates across the two cohorts, and we matched them one-to-one. And so what we were able to do is we were able to find, of those 364 patients in our pool, 147 in the multigene versus 147 in the single-gene BRAF testing that were very, very similar. In fact, we created what's called a directed acyclic graph or a DAG, together with clinicians to say, “Hey, what biases would you expect to have in these two cohorts that might limit our ability to find a signal of effectiveness?” And so we worked with clinicians, with health economists, with epidemiologists to really understand those different biases at play. And the genetic matching was able to match the cohorts on the covariates of interest.
Dr. Rafeh Naqash:And then could you speak on some of the highlights from the results? I know you did survival analysis, cost-effectiveness, could you explain that in terms of what you found?
Dr. Dean Regier:We did two analyses. The intention-to-treat analysis is meant to emulate the pragmatic randomized controlled trial. And what that does is it answers the question, for all those eligible for multigene or single-gene testing: What is the cost-effectiveness in terms of incremental life-years gained and incremental cost per life-years gained? And the second one was around a protocol analysis, which really answered the question of: For those patients who were actually treated, what was the incremental effectiveness and cost-effectiveness? Now, they're different in two very important ways. For the intention-to-treat, it's around population questions. If we gave single-gene or multigene to the entire population of advanced melanoma patients, what is the cost-effectiveness? The per-protocol is really around that clinical question of those who actually received treatment, what was the incremental cost and effectiveness? So very different questions in terms of population versus clinical cost and effectiveness.
So, for the intention-to-treat, what we found is that in terms of life-years gained is around 0.22, which is around 2.5 months of additional life that is afforded to patients who went through the multigene panel testing versus the single-gene testing. That was non-statistically significant from zero at the 5% level. But on average, you would expect this additional 2.5 months of life. The incremental costs were again non-statistically significant, but they're around $20,000. And so when we look at incremental cost-effectiveness, we can also look at the uncertainty around that question, meaning what percentage of incremental cost-effectiveness estimates are likely to be cost-effective at different willingness-to-pay thresholds? Okay? So if you are willing to pay $100,000 to get one gain of life-years, around 52.8% of our estimates, in terms of when we looked at the entire uncertainty, would be cost-effective. So actually that meets the threshold of implementation in our healthcare system. So it's quite uncertain, just over 50%. But what we see is that decision-makers actually have a high tolerance for uncertainty around cost-effectiveness. And so, while it is uncertain, we would say that, well, the cost-effectiveness is finely balanced.
Now, when we looked at the population, the per-protocol population, those folks who just got treatment, we actually have a different story. We have all of a sudden around 4.5 or just under 5 months of life gained that is statistically significantly different from zero, meaning that this is a strong signal of benefit in terms of life-years gained. In terms of the changes in costs or the incremental costs, they are larger again, but statistically insignificant. So the question now is, to what extent is it cost-effective? What is the probability of it being cost-effective? And at the $100,000 per life-year gained willingness-to-pay, there was a 73% chance that multigene panel testing versus single-gene testing is cost-effective.
Dr. Rafeh Naqash:So one of the questions I have here, this is a clarification both for myself and maybe the listeners also. So protocol treatment is basically if you had gene testing and you have a BRAF in the multigene panel, then the patient went on a BRAF treatment. Is that correct?
Dr. Dean Regier:It's still physician choice. And I think that's important to say that. So typically what we saw in both in our pre- and post-matching data is that we saw around 50% of patients, irrespective of BRAF status, get an ICI, which is appropriate, right? And so the idea here is that you get physician-guided care, but if the patient no longer performs on the ICI, then it gives them a little bit more information on what to do next. Even during that time when we thought it wasn't going to be common to do an ICI, but it was actually quite common.
Dr. Rafeh Naqash:Now, did you have any patients in this study who had the multigene testing done and had an NRAS or a KIT mutation and then went on to those therapies, which were not captured obviously in the single-gene testing, which would have just tried to look at BRAF?
Dr. Dean Regier:So I did look at the data this morning because I thought that might come up in terms of my own questions that I had. I couldn't find it, but what we did see is that some patients went on to clinical trials. So, meaning that this multigene panel testing allowed, as you would hope in a learning healthcare system, patients to move on to clinical trials to have a better chance at more appropriate care if a target therapy was available.
Dr. Rafeh Naqash:And the other question in that context, which is not necessarily related to the gene platform, but more on the variant allele frequency, so if you had a multigene panel that captured something that was present at a high VAF, with suspicion that this could be germline, did you have any of those patients? I'm guessing if you did, probably very low number, but I'm just thinking from a cost-effective standpoint, if you identify somebody with germline, their, you know, first-degree relative gets tested, that ends up, you know, prevention, etc. rather than somebody actually developing cancer subsequently. That's a lot of financial gains to the system if you capture something early. So did you look at that or maybe you're planning to look at that?
Dr. Dean Regier:We did not look at that, but that is a really important question that typically goes unanswered in economic evaluations. And so, the short answer is yes, that result, if there was a germline finding, would be returned to the patient, and then the family would be able to be eligible for screening in the appropriate context. What we have found in economic evaluations, and we've recently published this research, is that that scope of analysis is rarely incorporated into the economic evaluation. So those downstream costs and those downstream benefits are ignored. And when you- especially also when you think about things like secondary or incidental findings, right? So it could be a germline finding for cancer, but what about all those other findings that we might have if you go with an exome or if you go with a genome, which by the way, we do have in British Columbia—we do whole-genome and transcriptome sequencing through something called the Personalized OncoGenomics program. That scope of evaluation, because it's very hard to get the right types of data, because it requires a decision model over the lifetime of both the patients and potentially their family, it becomes very complicated or complex to model over patients' and families' lifetime. That doesn't mean that we should not do it, however.
Dr. Rafeh Naqash:So, in summary Dean, could you summarize some of the known and unknowns of what you learned and what you're planning in subsequent steps to this project?
Dr. Dean Regier:Our North Star, if you will, is to really understand the entire system effect of next-generation sequencing panels, exome sequencing, whole genomes, or whole genomes and transcriptome analysis, which we think should be the future of precision oncology. The next steps in our research is to provide a nice base around multigene panels in terms of multigene versus single-gene testing, whether that be colorectal cancer, lung cancer, melanoma, etc., and to map out the entire system implications of implementing next-generation sequencing panels.
And then we want to answer the questions around, “Well, what if we do exomes for all patients? What if we do whole genomes and transcriptomes for all patients? What are the comparative outcomes for a true tumor-agnostic precision oncology approach, accounting for, as you say, things like return of results with respect to hereditary cancers?” I think the challenge that's going to be encountered is really around the persistent high costs of something like a whole-genome and transcriptome sequencing approach. Although we do see the technology prices going down—the "$1,000 genome" or “$6,000 genome" on whatever Illumina machine you might have—that bioinformatics is continuing to be expensive.
And so, there are pipelines that are automated, of course, and you can create a targeted gene report really rapidly within a reasonable turnaround time. But of course, for secondary or what I call level two analysis, that bioinformatics is going to continue to be expensive. And so, we're just continually asking that question is: In our healthcare system and in other healthcare systems, if you want to take a precision oncology approach, how do you create the pipelines? And what types of technologies really lend themselves to benefits over and above next-generation sequencing or multigene panels, allowing for access to off-label therapies? What does that look like? Does that actually improve patients? I think some of the challenges, of course, is because of heterogeneity, small benefiting populations, finding a signal if a signal is indeed there is really challenging. And so, what we are thinking through is, with respect to real-world evidence methods and emulating randomized controlled trials, what types of evidence methods actually allow us to find those signals if indeed those signals are there in the context of small benefiting populations?
Dr. Rafeh Naqash:Thank you so much, Dean. Sounds like a very exciting field, especially in the current day and age where cost-effectiveness, financial toxicity is an important aspect of how we improve upon what is existing in oncology. And then lots more to be explored, as you mentioned.
The last minute and a half I want to ask about you as an individual, as a researcher. There's very few people who have expertise in oncology, biomarkers, and health economics. So could you tell us for the sake of our trainees and early career physicians who might be listening, what was your trajectory briefly? How did you end up doing what you're doing? And maybe some advice for people who are interested in the cost of care, the cost of oncology drugs - what would your advice be for them very briefly?
Dr. Dean Regier:Sure. So I'm an economist by training, and indeed I knew very little about the healthcare system and how it works. But I was recruited at one point to BC Cancer, to British Columbia, to really try to understand some of those questions around costs, and then I learned also around cost-effectiveness. And so, I did training in Scotland to understand patient preferences and patient values around quality of care, not just quantity of life, but also their quality of life and how that care was provided to them. And then after that, I was at Oxford University at the Nuffield Department of Population Health to understand how that can be incorporated into randomized control trials in children. And so, I did a little bit of learning about RCTs. Of course, during the way I picked up some epidemiology with deep understanding of what I call econometrics, what others might call biostatistics or just statistics.
And from there, it was about working with clinicians, working with epidemiologists, working with clinical trialists, working with economists to understand the different approaches or ways of thinking of how to estimate efficacy, effectiveness, safety, and cost-effectiveness. I think this is really important to think through is that we have clinical trialists, we have people with deep understanding of biostatistics, we have genome scientists, we have clinicians, and then you add economists into the mix. What I've really benefited from is that interdisciplinary experience, meaning that when I talk to some of the world's leading genome scientists, I understand where they're coming from, what their hope and vision is. And they start to understand where I'm coming from and some of the tools that I use to understand comparative effectiveness and cost-effectiveness. And then we work together to actually change our methods in order to answer those questions that we're passionate about and curious about better for the benefit of patients.
So, the short answer is it's been actually quite a trajectory between Canada, the UK. I spent some time at the University of Washington looking at the Fred Hutch Cancer Research Center, looking at precision oncology. And along the way, it's been an experience about interdisciplinary research approaches to evaluating comparative outcomes. And also really thinking through not just at one point in time on-off decisions—is this effective? Is it safe? Is it cost-effective?—not those on-off decisions, but those decisions across the lifecycle of a health product. What do those look like at each point in time? Because we gain new evidence, new information at each point in time as patients have more and more experience around it. And so what really is kind of driving our research is really thinking about interdisciplinary approaches to lifecycle evaluation of promising new drugs with the goal of having these promising technologies to patients sooner in a way that is sustainable for the healthcare system.
Dr. Rafeh Naqash:Awesome. Thank you so much for those insights and also giving us a sneak peek of your very successful career.
Thank you for listening to JCO Precision Oncology Conversations. Don't forget to give us a rating or review, and be sure to subscribe so you never miss an episode. You can find all ASCO shows at asco.org/podcast. Thank you.
The purpose of this podcast is to educate and to inform. This is not a substitute for professional medical care and is not intended for use in the diagnosis or treatment of individual conditions.
Guests on this podcast express their own opinions, experience, and conclusions. Guest statements on the podcast do not express the opinions of ASCO. The mention of any product, service, organization, activity, or therapy should not be construed as an ASCO endorsement.
