A Novel Approach for Finding Predictive Biomarkers

Julia Geronimi

Transcript

A novel approach for finding predictive biomarkers

[00:00:00] Alexander: You are listening to the Effective Statistician podcast. The weekly podcast with Alexander Schacht and Ben Piske designed to help you reach your potential lead great science and serve patients while having a great [00:00:15] work life balance.

[00:00:22] Alexander: In addition to our premium courses on the Effective Statistician Academy, we [00:00:30] also have. Lots of free resources for you across all kind of different topics within that academy. Head over to the effective statistician.com and find the [00:00:45] Academy and much more for you to become an effective statistician. I’m producing this podcast in association with PSI community dedicated to leading and promoting use of statistics within the healthcare industry.

[00:00:59] Alexander: [00:01:00] For the benefit of patients, join PSI today to further develop your statistical capabilities with access to the ever-growing video on demand content library free registration to all PSI webinars and much, much more. [00:01:15] Head over to the PSI website at www.psiweb.org to learn more about PSI activities and become a PSI member to pick.

[00:01:29] Alexander: Welcome to another [00:01:30] episode of The Effective Statistician. Today I’m super happy to have Julia and Pavel with me to talk about, a very important topic. But before we dive into that, I have first want to authors to introduce [00:01:45] themselves. Julia, how about you getting first? 

[00:01:48] Julia: Yes, for sure. Thank you Alexander.

[00:01:50] Julia: Really happy to be here today. So I think that I should start saying that the view that I’m sharing today is my own, and I’m not talking on behalf of my employer. That being said, I’m a principal, [00:02:00] transnational statistician working at Sege, located in France near Paris. I’m part of the quantitative pharmacology group within the transnational medicine.

[00:02:08] Julia: I’m responsible for biomarker analysis from a statistical point of view. For sure. I’m working through the drug development, so [00:02:15] I’m involved in projects from targeted identification to late phases depending on projects. Today I’m more focused on oncology, but I’ve gained experiences in many therapeutic area in the past.

[00:02:25] Julia: So I’ve been involved in cardiovascular disease, metabolic disorder, [00:02:30] immuno-oncology, and immuno inflammation in the past. I’ve been working with Biomarker for the past 10 years and even more because I started more than 10 years ago with a PhD as working around viable selection in a really high dimensional context.

[00:02:43] Julia: The objective at that time was to [00:02:45] integrate missing data and repeated measurement in lasso type and methodology. At that time, my objective was about biomarker identification from omic data, but today I’m more focused on biomarker already identified with a strong biological evidence, and [00:03:00] how can we move forward in integrating.

[00:03:02] Julia: That type of biomarker in clinical trial. So either with innovative methodology for biomarker evaluation, define some gonna go rules based on the biomarker and why not biomarker based design. 

[00:03:14] Pavel: Thank you very much for [00:03:15] inviting us. So I’m Pavel. I’m a group leader at the MRC by Statistics Unit at the University of Cambridge.

[00:03:22] Pavel: And I’m also a consultant for the Statistical Methodology company, Sarga based in France. The topic that prompted [00:03:30] today’s discussion is a three-way collaboration between SARGA and the MRC by Statistics unit. So this is kind of in what capacity I was involved in this work. So what I’m primarily interested [00:03:45] in is innovative clinical trials and design and analysis of clinical trials.

[00:03:50] Pavel: Generally, I develop statistical methodology and find ways to apply it in both public and privately funded trials. And this is exactly the capacity [00:04:00] in which I was involved here, though it was, as Julia said, very interesting applied problem. And we were thinking about do we already have solutions for that problem or not?

[00:04:10] Pavel: And then, you know, no surprise they were not that we were so happy with. [00:04:15] So in, hence I was involved to develop and evaluate and work on that together. 

[00:04:20] Alexander: Very good. So let’s dive into predictive and prognostic biomarkers. Let’s first clarify what that actually means before we get [00:04:30] into the content. Julia, what are, what’s, what is the difference between prognostic and predictive biomarker and why is that so important?

[00:04:39] Alexander: I. 

[00:04:40] Julia: Yes. Maybe I should start with the classical physical definition of a [00:04:45] biomarker. The best glossary. There is a definition saying that a biomarker is a defined characteristic that is measured as an indicator that may include molecular, histologic, hydrographic, or physiologic characteristics. So that mean at the end, that biomarker could be a lot of thing.

[00:04:59] Julia: [00:05:00] And most of the time what I’ve been involved, the clinical study I’ve been involved, we’re looking at mutational status, gene expression, or expression. When we talk about prognostic and predictive biomarker, what we’re looking at our actually biomarker that could allow for the identification of patients [00:05:15] most likely to benefit from a therapeutic strategy.

[00:05:18] Julia: So that mean that this type of biomarker can really increase the chance that the therapeutic benefit will be demonstrated in our clinical trial. So. Those type of biomarker are really super great tool for the [00:05:30] drug development, and we definitely need to optimize our clinical trial and the way we evaluate biomarker for, let’s say, better transformative precision medicine.

[00:05:39] Julia: So now if we look at the clear definition, there is some definition in the best glossary. For the prognostic part, [00:05:45] it said that a biomarker, it’s a biomarker, used to identify likelihood of a clinical event, disease recurrence, or progression in patient who have the disease. So if we take the easiest example mutation, it could be either you have the mutation and you present an [00:06:00] aggressive form of the disease.

[00:06:01] Julia: So for example, the probability of being a long-term survival will be quite low, or it’s either you don’t have the mutation and so you present less aggressive form of the disease and the probability of being a long-term survival will be quite high. 

[00:06:13] Pavel: Mm-hmm. 

[00:06:14] Julia: So it could [00:06:15] drive the trap of treatment you will give to your patient, but it could also drive the inclusion in clinical trial because this way you will have a lot of events or improve the number of events in your clinical trial and improve the power or shorten your clinical trial.

[00:06:29] Julia: The [00:06:30] prognostic biomarker could be a really great tool in clinical development. And then if you go to the predictive one. The definition is a little bit more complex saying it’s a biomarker used to identify individuals who are more likely than similar individuals without the [00:06:45] biomarker to experience a favorable or unfavorable effects from exposure to medical product.

[00:06:50] Julia: So if we go back to the easiest definition with a mutation, when you’re looking at specific drug or specific mechanism of action, you have the mutation and the probability of being a [00:07:00] responder is really high, or you don’t have the mutation and the probability of being a responder will be really low. So it’s a super tool to really select the patient that will benefit from your treatment, and that means really that the predictive biomarker really help identify the right medicine for the right patient, [00:07:15] entering more focused trial and timely access treatments.

[00:07:18] Alexander: In essence, prognostic tells you across all patients, is it independent of the treatment, is a patient going to have a worse or better outcome? [00:07:30] But for predictive, it shows you. Differential effects for the different treatments. In linear model terms, you would see an interaction between treatment and a biomarker.

[00:07:42] Alexander: Very good. Now, why is it so [00:07:45] difficult to assess these biomarkers and find predictive biomarkers? 

[00:07:51] Julia: So it’s really difficult for me to answer that question because there are so many things that makes it really difficult. If we look specifically for prognostic and predictive [00:08:00] biomarker, I would say that first point for me is it’s really difficult to distinguish between prognostic and predictive biomarker if you only have experimental drug in your clinical trial.

[00:08:09] Julia: So first thing that you will need at some point to have a control arm in our clinical trial, and we do see a lot of [00:08:15] publication. Poster or, or science presentation in scientific Congress claiming for new predictive biomarker while having only experimental arm in the design. So it’s really complicated because then the clinician wants to test that in clinical trial.

[00:08:29] Julia: [00:08:30] Uh, and if you don’t have any idea of what the behavior of the patient in the control arm, then is difficult to conclude on the predictiveness of the biomarker actually. Then I would say that sometime you have some biomarker that may be prognostic, predictive, and it’s not really clear [00:08:45] today what is the best statistical strategy you need to put in place to really distinguish between birth and if the prognostic effect actually diluted your effect.

[00:08:53] Julia: And so you need to really be able to distinguish, or is it actually optimizing your effect in the biomarker positive [00:09:00] population. So in your design and power. It’s still difficult to put in place a good statistical strategy. Being able to take into account prognostic and predictive biomarker, both at the same time.

[00:09:12] Julia: Another point is that in the definition of the best [00:09:15] glossary for the best, um, predictive biomarker, they’re talking about with and without the biomarker, meaning it’s implied that the biomarker is already dichotomized. You are always talking with binary one, and I did it myself. I take the mutation example [00:09:30] because it’s the simplest example you will find, but we do have a lot of continuous biomarker and what should we do if we are facing continuous biomarker?

[00:09:37] Julia: Should we ize it? We know we’ll lose some power. If we have sample size, that could be really an issue. Should we use PEs tires [00:09:45] or the true value? I feel that there is still some room for renovation in the fields of continuous biomarker and the way we evaluate the predictiveness here. Another thing that could imply the loss of poor that I.

[00:09:57] Julia: When we talk about early phase and early clinical [00:10:00] trial, I do see a lot of proof of concept study, meaning that we’ve designed clinical trial to evaluate transformative efficacy signal at the end. We have some trial with really small sample size, but at the end of the trial when we are playing with really a small number of samples.

[00:10:13] Julia: We do have a lot of analysis [00:10:15] to perform, a lot of different question around the biomarker parts, so it’s really difficult and I think we need innovation in the field of small sample size to be able to produce robust biomarker evaluation and allow for decision making in this really complex context. [00:10:30] 

[00:10:30] Pavel: Yep.

[00:10:31] Julia: That was for the prognostic predictive parts. But I think there is really two points that are general for all the type of biomarker you’re looking at that I have the feeling that most of the time biomarker are seen as exploratory. [00:10:45] So that mean that you have brief or no statistical description in the protocol brief or no statistical description in the statistical analysis plan.

[00:10:53] Julia: And sometimes we do not have any clear strategy for prognostic and predictive evaluation. At the end of the game, the [00:11:00] biomarker are decisional because at the time analysis or at the end of the trial, if you are in the reason zone for the efficacy signal, you want to go back to the biomarker information and look at the efficacy signal in the biomarker positive population.

[00:11:11] Julia: If you didn’t define a clear strategy, it could be an issue. [00:11:15] And last thing that is still valuable for all type of biomarker is the technically and analytically validated one, because sometimes we don’t know exactly what type of it is. Is my ASAP precise enough? Is it really validated? What about the batch [00:11:30] effects, the local measurements, stretch time, fresh versus hyper altitude, that kind of stuff.

[00:11:34] Julia: It could really impact the value at the end that we have in our database and. Last important point is that one thing that I’ve been facing in a clinical trial is, is my biomarker [00:11:45] specific to the population included in the clinical trial? Imagine an example where you’re looking at protein expressed as a predictive biomarker, but this protein is actually really present, a high association with a mutation, and this specific mutation is used as an inclusion [00:12:00] criteria in the clinical trial.

[00:12:01] Julia: So it feels like you are already including patient that may be biomarker positive, but not based on the predictive biomarker you want to evaluate at the end of your clinical trial. So it’s really important for me to have a look at the inclusion criteria [00:12:15] and the association you may have with your potential predictive biomarker.

[00:12:20] Alexander: Yeah, because then you don’t have any variability in your biomarker, and then you can’t find any kind of prediction. Yeah. So if I understand it correctly, we [00:12:30] have. Not just one problem. We have actually many problems that we need to solve at once, which includes variable selection, digitalization, multiplicity, topics, all around pre specification, having a [00:12:45] strategy.

[00:12:45] Alexander: And then the essay itself might not be super sensitive, and so we can have problems with that as well. So you have basically potentially ERO in the covariates, and so it always kind of decreases the chance of you finding [00:13:00] something. This is not a new problem. What have others done before you publish to establish these kind of continuous biomarkers?

[00:13:11] Pavel: So, as you rightfully said, it’s multiple problems. That’s [00:13:15] why there were multiple, uh, approaches to these multiple questions. Shall we have a step back and see? What are the approaches? What are the domains and is there natural order in which we should tackle them? Should we try to [00:13:30] answer them straight away?

[00:13:30] Pavel: Or is there a natural order in which we should try to approach? In this sense, I think it’s important to distinguish. As Julie already said, identifying the biomarker subgroup, this question of predictiveness in the [00:13:45] first place. So we call it predictiveness versus cutoff identification. If this cutoff, we mean the biomarker negative group where we don’t expect treatment effect to be beneficial for the treatment and a [00:14:00] positive way, expect treatment effect.

[00:14:02] Pavel: So when we talk about predictiveness, we talk about is there a general trend that. Higher value or lower values of biomarker associated with better or worse treatment [00:14:15] effect. And we thought that in the very small sample sizes that we are primarily interested in in this project. So in early phase drug development, we would like to approach this two problems differently.

[00:14:28] Pavel: So before [00:14:30] trying to identify their biomarker negative and positive population, we should be first sure. Or. Sufficiently confident that we are looking for something, that there is something to look for. So [00:14:45] biomarker values can predict treatment effect. And if this is true, then we will go into the domain of cutoff identification.

[00:14:52] Pavel: Our recent work and what I think we would like to focus, at least for the purposes of today’s discussion, is this predictiveness [00:15:00] question. Is there a signal that the biomarker can predict treatment effect? Does it make sense? 

[00:15:06] Alexander: Yep, yep, yep. So you basically disentangle all these problems and do one by first kind of not think about what [00:15:15] kind of cutoff do I need, or multiple capitals, but first for many variables that you might have.

[00:15:21] Alexander: So all could be potential biomarkers. You wanna understand which of these actually is worth looking at. 

[00:15:29] Pavel: [00:15:30] Yes, absolutely. And I guess it goes without saying, you know about pre specification. So for us statisticians getting used to the protocols of clinical trials, we would like to approach the biomarker studies with the same rigor.

[00:15:42] Pavel: We approach the design and analysis of the [00:15:45] primary efficacy endpoint. So that’s step zero for us, goes, you know, whatever we are working on here and developing, we want it to be pre-specified and put into the protocol of SAP and relevant sections. So when we come to the [00:16:00] question of predictiveness, you’re absolutely right.

[00:16:01] Pavel: There was many, many approaches proposed. I guess one tool to talk about is based on the, um, rock curves and essentially the, the way it works is that [00:16:15] the, the curves show us the, the usual work of specificity versus sensitivity, and we. Do these rock curves for two groups. We do it for the experimental group for which we give the active treatment that we are [00:16:30] interested in, and another for the control group.

[00:16:35] Pavel: Then we find the area under the curves and we are asking ourselves, are these two rock curves different? And if they’re different, we’re saying that there is. Difference [00:16:45] between treatment group with respect to biomarker and control group. So we claim predictive. This is something is called, that was proposed I think in 1980s and it’s called the Longs test.

[00:16:56] Pavel: And so this is kind of one type of approaches and [00:17:00] another type of approaches is based on the outcome versus biomarker relationship, the most natural way to approach it. And I’m was really glad that Alexander, that you already said that. Is, you know, you fit a [00:17:15] model with interaction, you know, as simple as that.

[00:17:18] Pavel: So you have, you can imagine you have a model and you have variants. Your treatment, you have biomarker and biomarker by treatment interaction, and essentially [00:17:30] predictiveness question is equivalent is their significant interaction between treatment and biomarkers. So essentially. One of the most natural approaches to predictiveness is to test the significance of [00:17:45] this interaction term.

[00:17:47] Pavel: There were some other approaches proposed in in the literature. I guess one of those would be you try to ize your biomarker, but instead of dehumanizing it [00:18:00] once, you are trying a range of values. For all of your biomarkers, you feed many linear regressions with your biomarker being binaries below and above this value, and then you have a range of P values [00:18:15] and you compare the smallest of these P values against some critical value.

[00:18:18] Pavel: If it’s below some critical value, you claim that you have predictiveness or not. It’s another discussion of how you choose this critical value, but you don’t de itemize it once, but many, many times, and to some extent the [00:18:30] best fit for the interaction you claim whether the biomark is predictive or not.

[00:18:36] Pavel: And so these are all the quantitative, but there is also a couple of qualitative approaches to the predictiveness identification. Instead of just [00:18:45] fitting the coefficient of the interaction and checking predictiveness, a qualitative approach says. Should we just check the sign of the treatment effect if the biomarker group changes the sign of the [00:19:00] treatment effect?

[00:19:00] Pavel: So in one group it’s negative. In one group it’s positive, so we have qualitatively different evidence, and for this we kind of reject the hypothesis of predictive. Does that make sense? 

[00:19:12] Alexander: That’s a very interesting question. The [00:19:15] sign basically means that when one group biomark a negative group. Treatment A is better than treatment B, and in the other group, treatment B is better than treatment A I have often [00:19:30] call qualitative biomarker or qualitative prediction effect.

[00:19:35] Alexander: If just the size of the treatment effect first, then it’s more of a quantitative. 

[00:19:42] Pavel: Yeah, so this kind of test where you check the sign is [00:19:45] qualitative. That’s why it’s important to think about what. What is the working hypothesis? How do you think the biomarker works? Because in many instances you might assume that biomarker negative group does not mean [00:20:00] that the experimental treatment is worse than control.

[00:20:03] Pavel: It might mean they are the same. It’s not necessarily worse, but for the biomarker positive, you expect a better treatment. And then the question to be asked, you know, is this qualitative? Interaction test. Is it the right approach? If [00:20:15] you’re not expecting the sign to change, but you expect it to be zero or positive?

[00:20:20] Pavel: All important considerations to take into account. 

[00:20:22] Alexander: Very good. So what is your new approach now? 

[00:20:26] Pavel: We took a quantitative [00:20:30] approach to the predictiveness evaluation, and we also took an approach based on building the biomarker. Treatment biomarker relationship. So to some extent, you know, it has flavors [00:20:45] or the interaction test.

[00:20:46] Pavel: This is one of the approaches we can compare our new proposal to. So the idea is quite simple actually. So what we would like to do in the first place is [00:21:00] to understand what is the average effect of the biomarker. So what I mean by average effect. By how much the differences in the biomarkers can explain the differences in the treatment effect.[00:21:15] 

[00:21:15] Pavel: In practical terms, we feed our biomarker response relationship and derive the relationship of the treatment effect as function of the biomarker, and then what we would expect to [00:21:30] see. So our A axis is the treatment effect. Our x axis is biomarker. What we would expect to see for the predictive biomarker that this curve that we plot has either upward [00:21:45] strength, if we expect that the more treatment benefit for high values of biomarker or downward trend.

[00:21:50] Pavel: If the higher values of biomarker means lower treatment effect, but this is essentially the curves that we’re working in. And it doesn’t have to be linear. It [00:22:00] can be any kind of relationship that we think is plausible, and we think that’s one of the beauties of the thing that we’re proposing here. So this is our main object, and then we do the following to that.

[00:22:13] Pavel: So we take two [00:22:15] values. Audit values of the biomarker. For the purposes of my example, let me assume. Our working hypothesis is that the higher value of the biomarker the better the treatment effect. We take two values of the biomarker audit, [00:22:30] lower value and higher value, which randomly sample those and find the treatment effect for our lowest value of the biomarker and the treatment effect for our highest value of the biomarker.

[00:22:40] Pavel: So essentially for the difference in biomarkers, we are finding [00:22:45] their difference in treatment effects. We cannot modify the biomarker in a patient, but if we have two equivalent patients, differences, this biomarker would be equivalent to this difference in the treatment effects. So essentially, we have [00:23:00] estimated for these two values of biomarker, what is the expected increase in the treatment effect we would expect from this change in biomarker.

[00:23:09] Pavel: Then all along our range of the possible values, we sample pairs of the [00:23:15] lower value and higher values, and we do it many, many times, thousand of times. And we expected increase in the treatment effect for pairs of biomarkers. And then we are saying, because it could be non-linear relationship, for example, we’re saying what is our expectation on [00:23:30] average?

[00:23:30] Pavel: So we take the mean of this 1000 difference in treatment effect. This is the quantity that we are interested in. This is our average increase in the treatment effect from the biomarker increase. [00:23:45] Let us base our decision making on those and if we expect, if we’re fairly confident that the difference in the treatment effect is positive in my example, then we can confidently say that there is a [00:24:00] signal of predictiveness.

[00:24:01] Pavel: So we would like to claim predictiveness if we’re not confident enough. We’re saying we don’t, 

[00:24:07] Alexander: that’s a nice kind of visual explanation of thinking of it at putting the treatment effect on the horizontal [00:24:15] axis and the biomarker on the vertical axis. 

[00:24:19] Pavel: No, I think other way around. So the biomarker, horizontal Y axis.

[00:24:24] Alexander: Yeah. So how much of the treatment effect can be explained by the biomarker? How do you [00:24:30] estimate the treatment effect 

[00:24:31] Pavel: to use your favorite model? 

[00:24:32] Alexander: Okay. And use any model. 

[00:24:34] Pavel: So again, that’s one of the things that we also want to accommodate. We thought, is it plausible that there might be non-linear effect?

[00:24:42] Pavel: And this was something also that we want to accommodate in [00:24:45] our methodology. One of the examples in the paper is we have binary response. So we fit a logistic regression with interaction, treatment biomarker, and treatment by biomarker interaction. From that model fitting, we feed the treatment effect [00:25:00] versus biomarker as a function and work with that as an object.

[00:25:03] Pavel: And again, another example is generalized addictive models that we can also work with them. If you’re a big fan of splines, you’re more than welcome not to focus on one interaction terminal, your spline, [00:25:15] but this whole of object. 

[00:25:18] Alexander: That’s nice. And if you have multiple biomarkers, would you then look into some kind of.

[00:25:25] Alexander: Play. So three dimensional thing is, [00:25:30] or do you do it 

[00:25:30] Pavel: by biomarker? That’s a great question, and I don’t think we have explored that extensively yet. I guess it also comes back to the question of pre specification. I. We’re not [00:25:45] fishing for biomarker. We have a solid scientific evidence that these biomarkers should be predictive of this treatment.

[00:25:51] Pavel: If you would like to control your type one error with respect to claiming this predictiveness or this biomarker, this is the procedure that you can [00:26:00] use. I mean, it’s not to say that you can use it for multiple biomarkers, but obviously then you have a multiplicity problem that should take into account not only that you.

[00:26:11] Pavel: Testing multiple biomarkers, but it’s multiple [00:26:15] biomarkers in the same patients, so I think that becomes a whole new problem of biomarker selection. We did some exploratory work on that. It’s not in the paper, but I can tell you that you can use it [00:26:30] for the purposes of multiple biomarkers, but the multiplicity adjustment can be tricky because of the correlations.

[00:26:36] Alexander: And of course, one important thing is only takes into account the values that you have actually in your biomarkers. So [00:26:45] you can’t extrapolate from it because you sample over your range of biomarkers. It really depends on that range. 

[00:26:53] Pavel: Yes and no. I think it’s not exactly what I meant when I said the range of biomarkers.

[00:26:57] Pavel: I was thinking about not the actual. [00:27:00] Failures of the biomarker that you saw in the trial. Mm-hmm. But just also scientifically possible. Wow. Range. So if you trust your model, then you can extrapolate. Absolutely. This is essentially what [00:27:15] we did in our approach. We’re saying we assume that the biomarkers will can be from A to B, even if in our trial we only saw the biomarker values.

[00:27:27] Pavel: Inside of this interval, but never very close to A to [00:27:30] B. When we’re sampling in the procedures that I described, we would still have samples from A and B, you should trust your model and that this is the right and sensible values of biomarker. We also exploit model specification, and we can confidently [00:27:45] say that it’s quite robust to the mis specified model.

[00:27:48] Pavel: And if you know the model perfectly, you are safe and you can do better, but it’s also robust to the miss specification. 

[00:27:53] Alexander: So from what you described, I think. It is not too complicated to implement this approach. [00:28:00] Hopefully you have done something to make it easier to implement it. No, 

[00:28:04] Pavel: you’re absolutely right.

[00:28:05] Pavel: It’s as simple as it sounds. Maybe that’s why nobody thought about that before. Indeed, you can just implement it using the GLM in your favorite software and you just [00:28:15] extract all the, you know, because all you need is to estimate the relationship and then estimate the standard there are around this relationship.

[00:28:21] Pavel: That’s all you need to construct these intervals. So we have not yet. Developed kind of an art package for that, but we [00:28:30] do have our codes available with the paper, and they’re available in the GitHub, so they’re openly available for everybody to use. We have an example of the hypothetical dataset, just how you would approach an analysis of actual [00:28:45] biomarker study with the example that I already said with linear and a gem model.

[00:28:52] Pavel: And then we also have the code that, how you can evaluate the operating characteristics of such an approach, although not yet in our [00:29:00] package, but hopefully nicely looking and ti it up code is available with, with the work. 

[00:29:05] Alexander: Awesome, that’s great. Um, a couple of interesting further perspectives that we wanna dig into.

[00:29:12] Alexander: And so first one is [00:29:15] in early clinical trials, if you have small sample sizes. One very hot topic is enriching it with external evidence and basically borrow from external evidence. How will that work? [00:29:30] Is it the same thing just with the models that you have, including the external borrowing? Or do I need to have some kind of additional consideration there?

[00:29:40] Pavel: So, no, that’s indeed a great point, and this is along the lines of what [00:29:45] we are thinking now and I guess. One of the reasons why you do want external borrowing here is because if you want to show that the biomarker is predictive, you need to have patients in [00:30:00] experimental and control. But also to have confidence in that relationship, you should have patients in biomarker negative and biomarker positive groups.

[00:30:10] Pavel: So essentially you have four groups to worry about, and all of them should [00:30:15] have decent data points to feed the model confidently. That’s why. My thinking about enriching it with external data, for example, if you want to have more control, patients can make a huge difference for the purposes of predictiveness.

[00:30:28] Pavel: We saw in [00:30:30] our evaluations that if we. Can get hands on. Some external information can greatly increase our power of such a procedure. The main challenge here is data sharing and pre specification, because in [00:30:45] many, uh, historical information borrowing approaches, you can get away with the summary statistics.

[00:30:49] Pavel: For the pioneer outcome, you go look in the paper number of patients, number of responses, and you can use it for your information. Borrowing here, you. Actually [00:31:00] need the biomarker value of the patients. You need individual patient data that you might not have, and obviously another challenge here. You should have it for the biomarker, how you define it.

[00:31:11] Pavel: Yeah. Because you know the biomarkers, [00:31:15] even if they’re called the same in the literature, you should always go to the last page and see how they define it. The famous PDL one, it was a shock for me to learn that. There are 100 ways to define it. Statistical methodology can be similar, but there are [00:31:30] additional challenges when you look at the biomarker.

[00:31:32] Alexander: Mm-hmm. Let’s go into another point now, if you have found some kind of predictiveness of a biomarker. You still need to do kind of a decision [00:31:45] of within which group sooner or later. Julia, what can you tell us more about the cutoff evaluation? 

[00:31:52] Julia: Actually, I do not have a lot of innovation, but a lot of question around the cutoff evaluation because at the end of your trial, you’re able to [00:32:00] conclude or to claim if the biomarker is predictive or not.

[00:32:03] Julia: The next question for sure is how will I be able to define what is the best cutoff? And this is really, truly. Important question, but what actually does it mean? I mean, you can use several methodology to define the cutoff, but [00:32:15] each methodology will give you a different cutoff. So it should use a statistical methodology based on the best discrimination between responder and non-responder patient.

[00:32:24] Julia: Should we include the probability of having a market difference in the biomarker positive group between the [00:32:30] experimental arm and the control arm? Um. Should we include a difference in treatment in the biomarker negative? What about the difference with what is happening in the overall population and how to optimize all that is a really complex question.

[00:32:43] Julia: This question could have a [00:32:45] huge impact on next trial because depending of the robust robustness of the KE and what you able to estimate on such a small sample size, could we. Help you define the next design. Are you confident enough to go to an enrichment design? Do you want to have biomarker stratified design or an [00:33:00] adaptive design?

[00:33:00] Julia: The cutoff evaluation for me is really a hot topic and there is still some role for innovation, uh, in this, in such sample, small sample size. 

[00:33:09] Alexander: Yeah, so it really depends on what you will do with it. Whether you exclude all [00:33:15] patients that are biomarker negative or stratify for it or do whatever you wanna do, 

[00:33:20] Julia: refine it, what.

[00:33:22] Alexander: What is different if we have a survival endpoint and the Cox regression model, is that basically all the same, or do we then have [00:33:30] some other difficulties? 

[00:33:32] Julia: I would say that we have difficulty because as PA mentioned, in the Y axis, it’s a clinical endpoint, and so with binary endpoint, it’s actually the probability of NS and the objective response rates.

[00:33:41] Julia: But when you go to a survival endpoint, it will be more complex than [00:33:45] that. So we can, if needed, like in my example, with a long-term survival or that type, you can still optimize the survival endpoint, but I think that we could improve the methodology to replace and add something more related to the continuous information of the survival endpoint.

[00:33:58] Julia: For sure. So [00:34:00] still some place for innovation here. Also, 

[00:34:03] Alexander: thanks so much for the discussion o of your recent paper, and congratulations for all the work. I absolutely love this episode because it does into some of the [00:34:15] basics around biomarker research. Predictiveness, prognostic. We touched on a couple of different.

[00:34:21] Alexander: Fundamental problems. We also talked about what is already available in there, and this, we have a very, very [00:34:30] nice overview in your paper regarding this and how you can define predictiveness. I really likes this visual approach of the treatment effect and the biomarker, and they relate to each other.

[00:34:44] Alexander: Thanks so [00:34:45] much for the discussion. Pavel Julia will put links to the paper, your LinkedIn accounts, and to the companies that have been involved, as well as to the academic institution in Cambridge. Thanks so much. 

[00:34:59] Pavel: Thank you. [00:35:00] Thank you very much.

[00:35:04] Alexander: This show was created. In association with PSI, thanks to Reine and her team at VVS work with the show in the background and thank you for listening. Reach your [00:35:15] potential lead great science and serve patients. Just be an effective [00:35:30] statistician.