Framework for estimating policy estimands

Framework for Estimating Policy Estimands

[00:00:00] Alexander: Welcome to another episode of The Effective Statistician, and today I’m really happy to talk with Alberto about one of the most hot topics, the Estimands, and especially here about something that is Yeah, one of the main problems is treatment policy approaches and what happens when, we don’t have the data and what can we do there? And he and his Coors just published a really nice paper about a framework that can help us accomplish lots of these different things in a very alergent way. But before we dive into the the details welcome Alberto on the show. For those who don’t know you, maybe you can introduce yourself, shortly.

[00:00:55] Alberto: Hello, Alexander. This is really nice. I’m really happy to be here in the episode. Thank you for in the invitation. So I’m Alberto Garcia. I’m from Madrid, from Spain. I’ve been in the industry about 26. I started working 1996 in a CRO, so in not 30 years, but still just only three years to have this 30 years anniversary.

My experience is a pharma statistician experience. Initially in CROs, one a small CRO in Spain, and later in at P D in the uk. I lived in Cambridge, then I moved to pharma. I work in a biotech and in Spain. I worked in Astellas Pharma in the Netherlands. And at the moment, I’m working from home in back in Madrid. I’m a consultant working for different clients mainly doing what I do to lead the statistics in clinical trials. Phase two, phase three. That is my background.

[00:01:51] Alexander: Just before we started with the recording, you mentioned you did a little bit of a break maybe also because of covid and restructurings and things like this, and went on to do your PhD.

[00:02:05] Alberto: How? Yes. Basically when I worked in Astellas in the Netherlands starting about 10 years ago I started doing a bit of research. A few papers I published in different topics, risk benefit. Topic mainly and when Astellas decided to basically move r and d to the US and Japan, it was a moment when I thought, okay, let’s use that moment where after almost about 20 years working to have a bit of a break to do a bit of research. And at that moment, I decided to do that the research within the scope of a PhD research, eh, so I’ve been working now for about five years with professor per and pardon Madrid and also Professor Brisolus in Rotterdam, in Erasmus doing our research that is about missing data in presence of intercurrent events. So it’s fully focused on the estimands topic.

[00:03:02] Alexander: Yeah. Cool. Very good. I love that you were pursuing this. I think there’s a lot of people that think about it, talk about it, but never do it. And I think it’s a great, an inspiration for maybe some of the people listening at the moment. That’s a should I do a PhD later in my career? Am I too old or whatsoever? Here you have someone that yeah, at the moment finalizing it and and went for it. Yeah. Congratulations on that move.

[00:03:35] Alberto: Thank you. I’m now 49, so it’s true. It is never too late. And doing this at this age is maybe a bit hard because you have to combine it with family and work. But on the other hand, I think you can really do a research that you already, you will find useful in your work and more about what you are going to do the research on it. When you do the research when you are 20, maybe it’s just basically the professor at the university they choose the topic for you and maybe at the end you might not even use it. I really enjoyed it was very interesting and I learned a lot and I finish was interesting.

[00:04:08] Alexander: Bad is cool. Yeah. And missing data or estimates is also today our discussion. When we think about the treatment policy approach, so an approach where you wanna understand the efficacy based on the decisions that you make at the start of the treatment. Yeah. So you wanna understand, okay, if you now start with treatment X as compared to treatment Y, what will be the outcome of it? In 12 weeks, 24 weeks, two years whatsoever. Yeah. That is the treatment policy approach. And now the more, more advanced part in this is of course how do you exactly define treatment X and treatment Y? So can you tell us a little bit more about what are main challenges, there that you face faced with the treatment policy approach?

[00:05:13] Alberto: Okay, so this it is very good point and good question and it’s really interesting that if you look at the definition of treatment policy strategy at the very first site is the easiest strategy because by definition you have an intercurrent event that it depends on your setting, but maybe treatment discontinuation or maybe the use of rescue therapy by definition, using a treatment policy strategy for that intercurrent event means that you just have to ignore it. You shouldn’t exclude data because, the data have been collected after that interconnect event. So initially it’s super simple, you just ignore it and you just analyze your point at the final visit. For example, visit 52.

Okay? That is initially simple. However the problem is that in some indications, in some trials in the way they have been designed the data you can have a situation, two different situations or three different situations. The optimal situation, the data have been collected equally after that INTERCURRENT event. So even for the subjects with treatment discontinuation, for example let’s focus on treatment discontinuation as intercurrent event but the same applies to other intercurrent events. So the option one, this, the data have been collected equally after tremend discontinuation equally means that the missing is there, but the likelihood to be missing is the same.

It’s basically you keep collecting the data in the same way. You might lose some patience. But the inter current event is not changing totally. The amount of data you have that you can use, in that case, it’s super simple because you just ignore that event. You do, and you apply a step forward standard model.

If you have a continuous, you use period, but then sometimes you have other situations. Let’s go to a second an extreme situation. Let’s switch totally to, the extreme situation in some indications and in some protocols once the treatment is discontinued, the subjects are not followed up anymore. Or at least the primary endpoint is not collected. Sometimes it’s because maybe it requires an invasive test because it’s not the same if the primary endpoint is observed easily. Yeah. And you can go to the patient after the subject discontinued treatment is not happy with the trial, but you can still collect that information. But other times..

[00:07:39] Alexander: Biopsy or something like that.

[00:07:41] Alberto: Yeah. And the subject discontinued treatment is not happy with the trial and they will so if you have an extreme situation, Where you don’t have data after three minute discontinuation or let’s go, you don’t have data. Or maybe very rarely we have a data point, but in essence, you don’t have the data. You are in the situation that is the focus of the paper, eh, that we will discuss today in that situation. The treatment policy strategy is not simple and basically, You will have to speculate and it’s a speculation what has happened after the treatment discontinuation. Yeah. And there is a middle ground that is also problematic that you still have data, but the amount of messiness after three discontinuation is harder than the messiness before that is also problematic. In that case, you might want to try to model the data after treatment discontinuation, but, and a standard model will not capture that. Okay, differential mis and for this type of middle ground situations I have seen papers using this dropout retrieval approach where basically you try to model the data after treatment, discontinuation differently, somehow, differently, of course, in that middle ground situation.

The big question is you have enough data to do that modeling effort because if at the end you have only a few data points, you may have an issue. So you have these three possible situations. The paper we will discuss later is focused on the second option where when you have no data or hardly data, and you will have to speculate and normally one way to speculate what has happened is using the reference by ation, delta, Jasmine. So basically all of them are just ways to speculate.

[00:09:30] Alexander: There’s another case in which you can easily come up with this, missing this. Yeah. Imagine you have a treatment discontinuation and you then start with another treatment. To further continue to collect the data. And now the stakeholders that you work with is not interested on this. You know what happens with this kind of strategy? If you first start with that and gender discontinue, you go over to the second treatment that you used in your study. But they say in our country, We switched to a different treatment and said, you haven’t observed.

Yeah. So you have data, but you have under the wrong treatment. Yeah. Whereas in your study, you have start with X and if that doesn’t work, go to Y. This stakeholder says, start with X, and if that doesn’t work, go to z. So although you have for treatment X and then go to Y, you have the data, you don’t have it for X go to Z. So the data situation is completely the same just because you have your treatment policy. Yeah. So the treatment policy that you’re really interested in is not for that you have missing data, even though you have data, it’s basically missing. Yeah. So that is Yes. Another situation where else that can easily happen.

And that’s just very often the case in HTA analysis. Yeah. Where, for example, for you have a, an control arm. And once they are, people discontinue from the control arm, they go over to the experimental arm. In real life, that will never happen. Yeah. So in real life, these HTA person will be, oh, if that work doesn’t work on the control arm, then it should go to another control, not to your new drug. Yeah, so you have, you are missing sales c kind of still sales data.

[00:11:36] Alberto: So that is very interesting situation and formally speaking on the framework we have seen that after this guideline in Jud, for example some C H M P points to consider guidelines on different. Diabetes Alzheimer, others have been updated and they use the update to include one paragraph on estimands and a common trend is the following, for treatment discontinuation, eh, use statement policy. So basically you ignore the fact that the subject is not on treatment and you use the data after.

Okay, but also for rescue therapy and this type of therapies, you mentioned the recommendation is often to use hypothetical. Hypothetical means don’t use the data after that event because you want to estimate hypothetical where the, that other medication is, let’s call it rescue medication, not existing, has not been, so at the end, you end up in the same situation. Covered that by this paper because the data you do have after treatment discontinuation is often not usable because has been collected after those medications is very good point. Very good point. Yeah. So the strategy you use for that, for those others for those other medications, it’s going to be important because we will, you can end up in the same issue the same lack of data. Good point.

[00:13:08] Alexander: Okay there are a couple of existing current approaches for treatment policy. What are your main critique points around this?

[00:13:18] Alberto: Okay. So could you repeat that?

[00:13:20] Alexander: I think you make a really nice summary in your paper where you speak about all the benefits of your new approach compared to all the other existing approaches.

[00:13:33] Alberto: Ah, okay. Yeah.

[00:13:34] Alexander: Yeah. And you say your new approach can do all kind of different things that previous. I think previous approaches can do. So there’s the section 4.3 in your papers that I’m already Yeah.

[00:13:49] Alberto: Let explain this because it’s not it’s interesting, but I’m not sure if I we are on the same page on the tools we already have in the literature, right? Even before that paper. We already mentioned here the focus is, to use treatment policy, a strategy for treatment discontinuation. But we don’t have the data or as you mentioned, the data that we have might not be data we can use. End is the same. And we will have to speculate. We will have to be honest, we will have to invent or to, and normally, of course, that is speculation. Should be conservative, eh what we have what options we have, eh to use reference based imputation, that basically you assume that for the experimental group after team discontinuation, in average, the subjects will behave like the control group, the reference group average in average.

[00:14:39] Alexander: Okay.

[00:14:39] Alberto: Another option. Go ahead.

[00:14:41] Alexander: And so that would be after treatment and discontinuation to basically assume they directly, behave like the patients under reference?

[00:14:50] Alberto: Yeah. Basically the treatment effect after treatment discontinuation is zero. So there is a treatment effect only on treatment and after administration is zero. There is another option is the copy increment from reference when you assume that they already achieved it, manufac at that time point. Is maintained, or you can maybe apply sort of delta adjustment. But basically you add or take a delta value a bit of effect size and you can increase the effect size for placebo or decreases at the end you have to fix those delta values.