Defining Clinical Trial Estimands: A Practical Guide for Study Teams with Examples including Regulatory Buy-in

Transcript

Defining Clinical Trial Estimands: A Practical Guide for Study Teams with Examples including Regulatory buy-in

Defining Clinical Trial Estimands: A Practical Guide for Study Teams with Examples including Regulatory buy-in 

[00:00:00] Alexander: Welcome to another episode of the effective statistician, and I’m really, really happy about the topics that we are talking about today, and I’m really happy to have Elena on the call to talk about this important topic. Hi, Elena. How are you doing? 

[00:00:19] Elena: Hi, Alexander. I’m doing great. Thank you so much for inviting me.

[00:00:23] Alexander: Very good. Before we go into the topic itself because those who don’t know you, maybe you can quickly introduce yourself. 

[00:00:31] Elena: Sure, my pleasure. Again, thank you for inviting me to be part of this episode. So my name is Elena Polverejan, and I work at Janssen R&D. These are the pharmaceutical companies of Johnson and Johnson. And I’m in my 20th year at J&J. I am currently a Scientific Director in the Statistical Modeling and Methodology group. That’s a group in Statistics and Decision Sciences. And I am based in New Jersey, USA. With regard to my research work, I am mostly focused on estimands and missing data methodology and their implementation of clinical trials.

I am the lead of our internal working group on Estimands and Missing Data. And I’m also the co-chair of a working group that we will be talking about. And that’s the Estimands and Missing Data working group of the International Society for CNS clinical trials and Methodology, and CNS stands for Central Nervous System.

[00:01:34] Alexander: Awesome. Yeah. That area is actually very, very close to my heart. I’ve been working in CNS for, for many years on pain, on schizophrenia, on ADHD, on anxiety, and also on depression. And that’s the topic that we’re talking about today. And I remember when I was working on this, it was quite before the time of estimands.

There was a lot of debate about should we use LOCF or should we use MMRM or should we use observed cases and all these kinds of different things. And everybody had a different kind of favorite topic and lots of arguments about that. And that was, you know, in hindsight, with the Estimands Framework, of course, a pretty stupid discussion to have whether one is better or the other.

So it’s really great that we have this episode today. So, before we go into your case study, let’s talk about your perception about where we are with the adoption of the estimand Framework. Given that, you know, got published a couple of years ago. You are, you know, leading these efforts within J&J, one of the biggest pharma companies.

And of course, you also have, with your long experience, a good oversight of what’s happening across the industry, I guess. So I really welcome your thoughts on where we are with that. 

[00:03:15] Elena: Yeah, thank you. So first of all, the estimand framework, it’s a great framework, and I have not seen a single case study in which it was not useful when implemented. However, the widespread implementation of the estimand framework, you know, when defining Estimands or reporting Estimands is still in progress across clinical trials. And also, we can consider that the engagement of the non-statistical functions is also in progress. We are always on the lookout for case studies. They are really well sought after, especially case studies with documented clinical and regulatory feedback. And also, what will be very useful and what we are trying to achieve, it’s a well-defined process to follow when implementing it starting from A going to Z, knowing and having very clear steps to follow when implementing, kind of almost like following a template.

I think it would also be helpful when implementing this process. But overall, it’s still in progress. It’s happening across trials across therapeutic areas, but it’s still in progress. 

[00:04:26] Alexander: Yep. Completely agree. We have, as a statistics community, quite a lot to do to drive this change. And as you said, case studies learning from others, how they implemented it, knowing what are all the different steps that you need to go through, and what are the different pitfalls is really, really important.

And so this is exactly what we will talk about today in this episode. You already mentioned that case studies are really helpful because then you can, you know, if you work, for example, in depression, you can more or less kind of directly take it or if you work in, you know, related areas that have similar needs, let’s say anxiety or other affective disorders or probably lots of different psychiatric disorders that have similar designs, similar kinds of studies, similar problems in terms of, you know, missing data, all these kinds of different things. You can probably learn and adopt from it quite a lot. When you started, how did you actually get started in terms of discussing this framework about depression? 

[00:05:44] Elena: Yes. Well, first we had to decide that it was depression, right? And the main goal of the Estimands and Missing Data Working Group of ISCTM was indeed to provide examples, but examples that are relevant across many disease areas.

We didn’t think that our main goal was to study depression and the depression setup. We just used it as a placeholder for having meaningful discussions about the estimand framework in general. Why did we choose depression? I want to maybe go there or maybe not yet. Yes, we have chosen because it’s a disease area from which we can learn lots of things and that, as you have mentioned before, can be really expanded and it can be related to many other disease areas.

Definitely it’s prevalent and it’s well studied. It has some widely accepted endpoints. So definitely we don’t have to have discussions about what’s acceptable and what’s not. However, it’s a very complex indication to pursue. Has many challenges. One, for example, being high treatment dropout rates, right? So we have to deal with this intercurrent event. Also in the MDD (Major Depressive Disorder) setup, there are many types of settings and trials. You need to show short-term efficacy, but at the same time you also have to show maintenance effect, like long-term effect. And also many of the intercurrent events that you see here: as I mentioned, treatment discontinuations, also relevant to many other types of disease areas, and like, for example, starting other types of treatments for MDD, right? It’s a relevant intercurrent event in many other disease areas, starting other types of treatment for that indication, right? Also, MDD allows for many types of endpoints and discussions. Short-term depression trials use continuous endpoints, severity scale for depression. Long-term ones use time to relapse, so a time to event.

Responder analyses are also important, so some type of composite, binary endpoints also appear on some of the Estimands. So, overall, it was really a good area, and also many of the members of the working group, who are more like half statisticians and half clinicians, and I think that was one of the strengths of our working group, had expertise in this area.

So we thought that’s a great area to start discussing and to make it as broadly as possible and applicable as possible to many disease areas. 

[00:08:32] Alexander: For those who are not so familiar with depression studies, short term studies are usually kind of acute studies where patients have a depressive episode, then get on to placebo or the active compound.

Placebo usually has a pretty high effect. In this area, and then you need to show that you’re superior to placebo. The other areas are kind of relapse prevention studies where usually that all the patients are put on active treatment and then only those that respond and reach a certain threshold are re-randomized to receive either placebo or continue on the same medication. So you have both these kind of randomized withdrawal studies, as well as these typical kinds of two, three cohort studies at the beginning. 

[00:09:25] Elena: Indeed. Thank you. Yeah. And also you can have monotherapy studies or adjunctive therapy studies in which you will study the treatment plus another. Yeah, you know, widely available treatment. 

[00:09:37] Alexander: Yeah. Very good. You also have these, sometimes these kind of steps through trials where you first go for one medication and then for the next medication and, and these kinds of things. So, yeah, lots of, lots of different areas. You started to have this discussion not just within your company but with this association, how did that initiate it? 

[00:10:05] Elena: The discussion on having this working group? All right. Yes. This is a working group that has started under the ISCTM umbrella. Actually, I co-founded this working group together with my colleague Pilar Lim in 2017. This is when at that time, the draft addendum appeared.

We were interested in the estimand framework. And in 2016, we just had a session on missing data at one of the fall conferences of ISCTM. At that time, we recognized the need for multidisciplinary collaboration. While it is a statistical guidance, the ICHE9(R1) addendum, we cannot implement it without collaborations with other functions, but specifically clinical colleagues, but not only, regulatory and we can go to other functions, too. So we decided to start having these conversations. So the whole thing started in 2017. And actually, I like to definitely recognize the contributions of my colleague, Pilar Lim, who unfortunately passed away early this year.

And we had wonderful collaborations, she had a wonderful insight into what this working group can do. And I think we have learned and we have really come far in having meaningful conversations within this working group and with other colleagues from other pharmaceutical companies, from FDA, from EMA. So I just want to make sure that we also recognize Pilar’s contributions and her insight into this. 

[00:11:47] Alexander: Very good. Yeah. I really love that you took initiative and went to organizations outside of your company to drive that change. I think this is very often much easier to get consensus across the industry and then work together with the specialists, medical organizations as well as clinical trial methodology organizations to move forward these kinds of discussions. And as you said, multidisciplinary approaches are really key. So that is for sure. One of the areas that helped you get along. What were the biggest barriers that you see? 

[00:12:37] Elena: Yes, truthfully, at the beginning, when we started, we didn’t know where to start from. So it took a while to align our thoughts and put together a process. We didn’t have much, we had the addendum, which we have discussed at length, but the addendum provides a framework and not necessarily the implementation process. Also, we had the estimand jargon that we had to handle. We didn’t necessarily have a common language of discussing with clinical colleagues.

So naturally, we had thorough discussions to come towards that. We didn’t have a template of how to implement this process up, so we naturally thought that we should come up with a template to follow. So I think these were the barriers, the lack of a structured process of implementing. You know, what are the steps that we have to follow? What do we start from? Like we can discuss further, we have decided that the natural step to start from is recognizing the stakeholder or stakeholders for a trial and then moving naturally from there, step by step. So I think these were in general the initial, you know, barriers when discussing the estimand framework with our clinical counterparts.

[00:13:58] Alexander: How did you get the regulators on board with and get feedback there? 

[00:14:04] Elena: Regulators are involved in this society. So we are very happy that the ISCTM Society allows us to interact with FDA, with EMA, and not only, with whomever is interested in. And actually they expressed interest in being involved in this discussion.

We had medical reviewers involved in this discussion in 2020. We also had in the ISCTM Spring Meeting, a session on Estimands in which we involved EMA and FDA medical reviewers in the discussion, and they have always been part of the discussion from the very beginning. Again, we like to emphasize that through our discussions we promote a thinking process, a process in general, and that the examples that we have provided in the paper that has been published are exactly the mirror examples.

So we don’t necessarily, you know, promote a certain way of, of applying or promoting an estimand, but we promote the thinking process that needs to be applied in this area. 

[00:15:12] Alexander: By the way, as you’re listening to this episode, we’ll of course put the link to the paper that Elena mentioned into the show notes.

So just head over to theeffectivestatistician.com and you’ll find everything there. In terms of this kind of, how did you set this kind of goal for this working group? I think that it is probably very difficult to describe you know, if people don’t really know what are Estimands, what is that kind of the framework, how do you get to agree on a common goal for this working group so that everybody knows what is to achieve.

[00:16:00] Elena: Yeah. Everybody agreed that the estimand framework is a useful framework, but at the same time we have all agreed that we’d like to have more guidance. So over time we have agreed on our main objective: to develop such an implementation process with very clear steps, to provide examples in this process and to use depression as the area that will exemplify this process, but not necessarily as the ultimate goal. This is not a guidance for depression. This is a process. It’s a thinking process. It’s really aligned to the ICH E9(R1) expert working group thinking process that has been included in the training material. And depression was the disease area that has been used to provide examples from, for the reasons we have discussed.

So I think that was our main goal: to provide a practical guide for study teams, for multidisciplinary study teams. And I think this has been made clear in the title of our paper, you’ll see there’s a practical guide for study teams, and in our abstract and in how our main objectives have been presented.

[00:17:24] Alexander: Yeah. What I think is one of the key takeaways if people do not work in CNS is that you can have a look into and use this pretty much like a blueprint to come up to a similar thing in other areas. 

[00:17:43] Elena: Exactly. 

[00:17:43] Alexander: Maybe you work in dermatology or maybe you work in respiratory or maybe you work in all other fields. Yeah, you can use these kinds of steps to work together with a similar organization and also engage the regulators in a similar way. And so if you’re really interested in driving this forward, I highly recommend having a look into this paper. If you think about the most important thing that any listener should take away from this episode, what would that be?

[00:18:24] Elena: I think for statisticians, I will say, that they should take seriously the implementation of the estimand framework. And because it can lead to really very meaningful discussions not only with clinical colleagues, but also with other functions. They should understand the implementation process and they should take a lead in this process.

However, they have to understand that they need partners. And to be able to have these partners, we need to be able to communicate with them using concise, natural, non-technical language. So that’s also a must. So we hope that this paper provides many examples of how trial objectives can be formulated, and how questions of interest can be formulated in a non-technical, natural way, right?

Using a language that everybody can relate to and understand. So I urge not only the statisticians, but also their partners, to work towards understanding this process and also if they think they need any help to engage in the discussion, I believe this is a very active area. There are many, not only papers, but presentations, webinars right across many institutions that do wonderful work in this area. It’s a wonderful area to collaborate with, many partners that we can have. So that’s why I think we should definitely focus on, you know, future collaborations and endeavors in this area. 

[00:20:17] Alexander: Completely agree. So if your company is working in a specific therapeutic area where such a paper from Elena doesn’t exist, I think it’s time to get going so that you create these kind of case studies, you engage with all the different stakeholders because you want to do that as early as possible, so that your study designs are driven by all these kinds of different factors. And you don’t, you know, come to the submission and the discussions with regulators finding out that, ah, we should have collected something more in order to actually estimate these Estimands, because if you haven’t collected the right data in the right format, you will have problems with certain… 

[00:21:11] Elena: if you have not written the right questions, right? This is the most important thing, making sure that you ask the right questions. And these questions are aligned and help the stakeholders of the trials that are being designed. 

[00:21:26] Alexander: Completely agree. Thanks so much, Elena. That was an awesome discussion. And I really urge everybody to have a look into this paper. I can highly recommend it and yeah, thanks so much again. 

[00:21:41] Elena: Thank you, Alexander. Again, it was a pleasure to chat with you.