How do we define estimates effectively? What challenges arise in estimating treatment policy, particularly in the context of missing data and patient discontinuation? How can statistical approaches be communicated more transparently to both experts and the broader audience?

In this episode, Oliver Keene and I delve into the realm of estimating treatment effects with continuous outcomes over time. We also delve into the article by Naitee Ting, published in 2023, advocating for a reevaluation of estimation techniques. Ting’s emphasis on the last observation analysis sparks a robust discussion as we dissect its applicability in the context of treatment policy and its potential limitations.

We also discuss the following key points:

  • Discussion on Oliver’s Career
  • Estimates for Repeated Continuous Outcomes
  • Response to Ting’s Article
  • Defining Estimates
  • Discussion on Treatment Policy
  • Treatment Policy and Intent-to-Treat Analysis
  • Hypothetical Strategy
  • Communication Challenges
  • One Patient, One Vote Concept
  • Last Observation Analysis
  • Communication of Statistical Approaches

We invite statisticians to reflect on the evolving landscape of estimands, encouraging thoughtful consideration of estimation techniques and a deeper exploration of causal inference in clinical trials. The journey through this nuanced statistical terrain unfolds, offering valuable insights for both seasoned professionals and newcomers to the field.

Wright et al 2023 Response to Ting

Role play reference:

  1. Keene ON, Ruberg S, Schacht A, Akacha M, Lawrance R, Berglind A, Wright D. What matters most? Different stakeholder perspectives on estimands for an invented case study in COPD. Pharmaceutical Statistics. 2020 Jul;19(4):370-87.

Other references:

  1. Naitee Ting (2023) Emerging insights and commentaries – MMRM vs LOCF, Journal of Biopharmaceutical Statistics, 33:2, 253-255, DOI: 10.1080/10543406.2023.2184828
  3. Wright D, Bratton DJ, Drury T, Keene ON, Rehal S, White IR. Response to Comment on” Emerging insights and commentaries–MMRM vs LOCF by Naitee Ting”. Journal of Biopharmaceutical Statistics. 2023 Sep 23:1-3.
  1. Keene ON. Adherence, per-protocol effects, and the estimands framework. Pharmaceutical Statistics. 2023;1‐4. doi:10.1002/pst.232
  2. Keene ON. Intent-to-treat analysis in the presence of off-treatment or missing data. Pharmaceutical Statistics 2011, 10:191–195, doi: 10.1002/pst.421.
  3. Keene ON, Wright D, Phillips A, Wright M. Why ITT analysis is not always the answer for estimating treatment effects in clinical trials. Contemporary Clinical Trials. 2021 Sep 1;108:106494.
  4. Keene ON, Lynggaard H, Englert S, Lanius V, Wright D. Why estimands are needed to define treatment effects in clinical trials. BMC Medicine. 2023 Jul 27;21(1):276.

Never miss an episode!

Join thousends of your peers and subscribe to get our latest updates by email!

Get the shownotes of our podcast episodes plus tips and tricks to increase your impact at work to boost your career!

We won’t send you spam. Unsubscribe at any time. Powered by ConvertKit

Learn on demand

Click on the button to see our Teachble Inc. cources.

Load content

Oliver Keene

Statistical Consultant at KeeneONStatistics

Oliver is currently a statistical consultant. He previously worked at GSK for over 25 years in all stages of new medicine development, from the pre-clinical area through to support for reimbursement. He has particular experience of medicines for infectious diseases (influenza, herpes, hepatitis) and respiratory diseases.


Estimands for Repeated Continuous Outcomes

[00:00:00] Alexander: Welcome to another episode of the effective statistician and today I’m super happy to have Oliver Keene on this show, and I can’t believe. He hasn’t been on it before. A couple of years ago we had a really, really nice discussion about estimands as part of a PSI conference where we did some kind of Role play and he was one of the key person involved in preparing all of these and we had these different viewpoints from the sponsors, the regulators, the HTA guy and, and all these kinds of different viewpoints.

[00:00:44] And I remember at the time I was kind of defending the. HDA position on S demands at that time and was strongly favoring pull treatment policy. And Steve Ruberg was the patient voice and he [00:01:00] was favoring kind of this on treatment approach because all the patients that he talked to said, well, if I take this medication to the end, how will it work?

[00:01:11] And so it was Really, really insightful discussion. And I think if you scroll back quite a lot of episodes, there should be some discussion with Steve Ruberg about this. And of course, you can check out the PSI video on demand part. I’m pretty sure it’s on there as well as part of a conference. Who was that?

[00:01:36] Hi, Oliver, how are you doing?

[00:01:39] Oliver: Hi, Alexander. Well, first of all, thank you very much for inviting me on your podcast. And congratulations on having 300 episodes, I think it is.

[00:01:49] Alexander: Yeah, over 300 episodes. Yeah, yeah.

[00:01:53] Oliver: I think it’s a fantastic way of getting statistics discussed more widely among [00:02:00] people. And you’ve covered such a, such a lot of ground, had some very eminent speakers on, so I feel privileged to be part of it.

[00:02:06] Alexander: Yeah, I had, honestly, I had speakers from all kinds of different areas and also experience levels, you know, from people that are New grads up to people that, you know, had decades of careers, professors’ senior leaders from companies, but also kind of, you know, projects that stations, you know, that work in the trenches.

[00:02:33] So, and that, that I think makes me really, really happy. You have, have a long career already. Tell, tell us a little bit about that. Yes. Yeah.

[00:02:45] Oliver: Well, I, well, I worked for over. 30 years in in the palm seed industry. Working for GSK as a, as a statistician. I’m now an independent consultant. I mean, it was, it was, it was a long, [00:03:00] a long, a very varied career.

[00:03:02] But I always, I always enjoyed it. It’s always been challenging. The industry has never, never stayed the same over that period. It was, it was always evolving. I always felt I was learning new things. So yeah, and I continue to enjoy working, working in the industry.

[00:03:17] Alexander: Yeah, so if you want to work with a really experienced consultant, Oliver is definitely a person to go to and of course I’ll connect his LinkedIn profile into the podcast show notes. , Today, we’ll talk about estimates. So, and for sure, this hasn’t been, you know, it’s the first time that we have talked about estimates. And we’ll specifically talk about estimates in the situation where you have continuous endpoints collected over time. Yeah, like and then psychiatry, for example, you have a lot of these different [00:04:00] questionnaires, but could be any kind of continuous endpoint.

[00:04:04] And I remember quite some time ago Craig Mallincrotch did a lot of research about different ways to. Estimate things like MMRM, LOCF that was, you know, kind of about 20 years ago, a big, big topic. And at the time he’s, he was strongly favoring MMRM over LOCF and made this. Estimation technique, a very, very prevalent thing, and it’s nowadays also very easy to implement both in R and SAS, of course, and so that has helped a lot to make it widely used.

[00:04:50] Now, recently actually this year in 2023 Ting published an article and advocated [00:05:00] Yeah, for our discussed what could be kind of different estimates around this. And as a response to this Oliver together with a couple of other authors David Wright, Daniel Breton, Thomas Drury, Sunita Rihal, and Ian White, so a couple of really, really famous names, responded to this, and this is what we want to talk about today.

[00:05:25] So Oliver, from your point of view, what are, what’s the first most important point you have about the paper from Ting? Okay,

[00:05:38] Oliver: well, let me first of all, you know, start, you said this was a collaborative work, and I just wanted to say how much I enjoyed working with my co-authors Sunita, Dan, Tom, David, and Ian, and it was really great to be part of a project with, you know, people who are such experts on the subject of S Demands.

[00:05:57] I mean, estimates have always [00:06:00] been something that I’ve, I’ve been really interested in. Right back from when the, the US NIH published, published the theirs, their tracks on. Missing data. I mean, it’s always interested me. You know what treatment effect you’re actually estimating it in an in a clinical trial.

[00:06:18] And this comes back to, you know, even before estimands, the idea of intent to treat analysis and what exactly did that mean? So I really welcome the, you know, the introduction of the estimates framework. I think that’s, that’s been really important. And obviously we saw, the group saw this article by NaiteeTing and we came together, you know, this is Ian White works in academia and all of us work for different, well, we work for different companies.

[00:06:46] We felt Quite strongly, the article had some misconceptions about the ICHE 9 estimates framework and we felt uncomfortable with the arguments that were, that were being put forward for a, for a broad [00:07:00] adoption of last observation

[00:07:01] Alexander: analysis. Yeah, that is the first point. What actually is last observation analysis?

[00:07:08] I know last observation carried forward analysis, but what is the difference to last observation analysis?

[00:07:15] Oliver: Well, in practice, I think the points Natty the article is that essentially they’re the same thing. I mean, conventionally, traditionally we’ve always thought of last observation carried forwards.

[00:07:25] You know, that’s the idea that if somebody discontinues the treatment. You take the last observation on the treatment and use that as, you know, for future time points right through to the, to the final, final time point of the study. I think the distinction he was making was he didn’t see any kind of carrying forward that he sees the last observation that is the.

[00:07:46] the kind of the outcome for the patient is the last observation. So he’s trying to make that that kind of kind of distinction. But that really then brings into question is that is that really a treatment, an appropriate estimate for a treatment [00:08:00] policy approach.

[00:08:02] Alexander: Okay, so let’s untangle this a little bit.

[00:08:06] So the first point I think is that estimation is one thing and defining the estimate is a completely different thing. What should drive what?

[00:08:22] Oliver: Yes, I mean I think current views is you define, you define your estimate first and then consider what’s an appropriate estimation strategy for your, for your estimate.

[00:08:34] Within the paper, some of this gets a little confused, I think, between, you know, starting with an estimation and then kind of justifying it with a, with the estimand and the strategies. I think, as you say, I think it’s important to define your estimate first and then consider, you know, what are the appropriate estimation strategies.

[00:08:56] Alexander: Yep. Yeah. I completely agree [00:09:00] with that. You always first need to start with your intent mind. Yeah. And then it’s about kind of, okay. And how do I know? Approach that. Of course, if you then come up with something that is not estimable, then well, probably you need to go back. But,

[00:09:20] Oliver: yeah, yeah, that’s, that’s always the, you know, there’s always this kind of iteration between, you know, what you would ideally like and what you can, what you can, what you can estimate.

[00:09:30] But I don’t think, I mean, people seem to think that, for example, that treatment policy is easy to estimate. Well, actually it’s not. When you’ve got missing data, it’s actually one of the more, it is quite difficult because you have to make assumptions about the missing data. So. Okay. Yeah. Sometimes there’s misconceptions about what’s easy to estimate as well.

[00:09:50] Alexander: So let’s dive a little bit into Treatment policy here first. Yeah, sure. Treatment policy and correct me if [00:10:00] I’m wrong, basically speaks about you decide at the start of your treatment, how exactly you will treat patients. Yeah. And then you analyze accordingly. So for me, is that what basically mean? Okay, let’s estimate.

[00:10:21] Yeah what is the outcome of a decision we make at baseline? and this decision is, okay, we start with treatment A versus we start with treatment B. Or we start with treatment A and in case something happens, we’ll do this. Or versus B and in case something happens, we will do this. Yeah, so for example, it could also mean we start with treatment A, and if it doesn’t work, we increase the dose, yeah, and then we observe the outcome, or we have treatment B, and if it doesn’t work, we [00:11:00] increase the dose, yeah.

[00:11:01] But the, the, the policy needs to be clear, isn’t it?

[00:11:07] Oliver: Well, I mean, I think it’s, it’s interesting the way your kind of described it but I think, I worry about the way, the way you’re thinking treating policy. Because that assumes that that would be true if everybody followed the protocol, if everybody did what was said in it.

[00:11:25] And the whole point of the esterhans strategy is you have people who discontinue medication, who take medications that in theory are not allowed by the protocol, who once they discontinue the trial can do what, discontinue medication, they can go on to whatever medications they want. So I think, I think that’s one of the difficulties, you know, with I’m thinking of treating policy in that sense in that way, in the way that you’ve described.

[00:11:50] I mean, I would, I would worry about that. I think treating policy more, it’s better to think of it as an effective assignment. So all you’re thinking of is, you know, if we originally assign [00:12:00] a patient to these different medications. What is their ultimate outcome? You can try and control as much as you as you can in the protocol in terms of, well, if this doesn’t happen, but in real life, as we know, trials, people discontinue their medication.

[00:12:16] They don’t follow what’s in the protocol. They don’t you know, if a fails, they don’t necessarily go on to, you know, That’s true. So I think that’s one of the difficulties, the misunderstandings about treatment policy. It’s you’re going to end up, you know, once people have discontinued the treatment, they can do, you know, it’s, it’s the discretion of the investigator what, what they get.

[00:12:36] Alexander: Yeah, of course it is, but of course you could have treatment defined as something that is based on some kind of patient post baseline variables. You have said all the time, yeah, so that you, trig to target or things like this. Yeah. Or you have certain kind of ways to manage [00:13:00] communication or things like this.

[00:13:02] Yeah. So you could include that in your treatment decision and your treatment. Well, not in your treatment decision and how you define treatment. Yeah. So I think that is the. And when you think about the estimate framework, one of the things you need to be really clear about is about what is treatment a.

[00:13:23] Yeah, what is that really? And it could be, well, starting with this molecule, but it could also be a certain strategy, yeah, that you, of course, if then patients or physicians Deviate from the strategy, you could get missing data. Yeah, that is completely, completely see that. Yeah, but I think that and then managing the missing data is yet another point.

[00:13:54] First, you need to define what is really a treatment strategy.

[00:13:58] Oliver: Yeah, but you also, I mean, it just. [00:14:00] So you will also have data. You may not have missing data. You’ll have data, but you’ll have data from people who are following different treatment policies. You know, they’ve, they’ve taken it. And, and, you know, treatment policy, if you, if, if, if your strategy is treatment policy, you have to, that’s, that is the data.

[00:14:17] That’s the data you’ve got to use regardless of what, what, you know, what’s led them to those particular, what policy, what set of treatments they’ve used.

[00:14:25] Alexander: Well, I’m not sure. I’m not sure. Because, so, so, if, that, if you say your treatment policy is treatment A, you start with that, and then whatever the, the physician describes or does is your treatment policy, yeah, then you have, yeah, then you never have any missing data, of course.

[00:14:50] Yeah, because you will always have complete data, and then you can analyze as you randomize. Very, very easy. Yeah. The drawback, of [00:15:00] course, is that it’s very, very hard then to later on describe what was, what is the treatment. Yeah, especially if things change in terms of, you know, background and all these kind of other things.

[00:15:15] Yeah. and you don’t know at the start, of course, how many patients will deviate. Yeah. Oh, well, they don’t really deviate because you, you include any deviation in your treatment definition. Yeah. So then you basically move the, the problem from the intercurrent events into the, into the treatment definition.

[00:15:41] Like what you sometimes do with the confidence at the endpoint, you move it into the endpoint definition. Yeah. So these are of course, both, both possible scenarios.

[00:15:53] Oliver: Yeah, I mean, I think it’d be interesting, you know, perhaps we can get on to talk about composite and, you know, other strategies I think obviously, you know, going [00:16:00] back to the Ting paper is, is very dismissive of any, any strategy other than treatment policy or, or interestingly, while on treatment, um, and particularly the paper is very dismissive of the hypothetical strategy.

[00:16:14] And I think sometimes that word, Hypothetical is kind of problematic to people because it sorts of sounds like something that didn’t happen count. You know, well, I’m only interested in the facts. But I think, yeah,

[00:16:26] Alexander: the word is not really helpful.

[00:16:29] Oliver: No. You know, people, people react against it. You know, I’m a scientist, I’m interested in facts.

[00:16:33] I’m not interested in what might have happened. But I think what I think a better way to think of a hypothetical strategy, you know how I’ve been trying to think more, more recently. I mean, the important thing is it can address causal questions of what, what is the treatment actually doing? You know, these, I think these are important scientific questions.

[00:16:52] They’re important clinical questions. I think, you know, because we think so much in regulatory terms and HTA terms and, you [00:17:00] know, what is the policy, you know, what was the effect of this policy? And you know, but to an individual patient, I think, and for science as well, I think it’s important to try and address questions of what does it, what does that treatment itself, what is the cause and effect of that treatment?

[00:17:17] And if you think of it that way, then the hypothetical strategy doesn’t sound so, you know unattractive because you’re actually trying to get out, what does that treatment actually do?

[00:17:28] Alexander: Yeah. So hypothetical strategy could be something like, if I tolerate If the treatment and everything works fine, and I don’t discontinue what will be my outcome after eight weeks of treatment.

[00:17:47] Yeah. And I think this is really kind of a question about. Yeah, you can make this assumption. Why not? Yeah. Why, why is that a, [00:18:00] a, a better assumption than, than any other assumption that you’re making? Yeah,

[00:18:06] Oliver: yeah. We wouldn’t call it an assumption as such. I mean, I think assumption might, might, you know, that’s the word I, well, a condition worry about.

[00:18:12] Yeah. I think it’s, I think it’s a treatment effect you would be interested in. I, I would be interested in, as a patient, you know, if, if I’m, if I’m looking at a, you know, a drug label. I, you know, I, I worry a little bit, you know, I do take medications. I look at the labels and it says, you know, treatment A will have this effect.

[00:18:30] You know, if I take it but I know that that’s a mixture of people who’ve taken the medication and people haven’t. I don’t expect to, you know, the treatment to work if I don’t take it. And it’s, it’s, it’s that kind of average. It’s not telling me, you know, what is the, if, if I met, if I, if I take this treatment for the length of the period.

[00:18:47] You know, what can I expect in terms of benefit? It doesn’t tell me that. It tells me a policy type you know outcome, which we’re a mix of people. Some people have taken it the whole time. Some people have gone on to other treatments. That’s, that’s, that’s the [00:19:00] estimate that corresponds to a treatment policy approach.

[00:19:02] And I think there are really interesting questions around, you know, what, what, what effect is this treatment actually causing itself? Yeah,

[00:19:11] Alexander: I think this is also especially interesting if you also think about those response things, if you think about do you wanna better understand method of action, all these kind of different things.

[00:19:25] Yeah. This, you know, not everything that we do is a phase three study. And so therefore I think there’s a lot of area for, for having this as well. And as you said, even for phase three study, you know, from patient point of view that could still be interesting. Yeah. My, my main point very often is We shouldn’t tell people what they should be interested in.

[00:19:55] I think we should explain people what the different [00:20:00] things mean so they can make an informed decision about what they are really interested in. I think it’s kind of, it’s not very patient centric, physician centric, decision maker centric that we are so arrogant to basically be saying, I know better what you want to hear.

[00:20:24] Yeah. No, I don’t. Yeah. And so I’m absolutely in favor of, you know, Describing, training making clear that people can make an informed choice, yeah? And so just from that point of view, yeah? I would never write a sentence like, for longitudinal data analysis, while on treatment strategy and treatment policy strategy are more appropriate than hypothetical strategy.

[00:20:55] Why? Yeah? I think they are all, you know, [00:21:00] equally important from the get go, and it’s a personal decision what’s most appropriate for you.

[00:21:09] Oliver: Yeah, and I think they address different questions. Exactly. It goes back to what we were saying, you’ve got to define. What you know, what treatment effects you’re interested in, and in some cases that will be treatment policy, in some cases that may be hypothetical, it may be composite, you know, there are situations where all of these provide treatment effects that are of interest.

[00:21:27] Alexander: yeah. So there’s another point about kind of this treatment policy, and then kind of this last observation. is actually a valid way to look into treatment policy? What are your thoughts about that?

[00:21:47] Oliver: Well, yeah, I mean, I think that’s clearly misguided. I mean, treatment policy approach requires complete, requires follow up off treatments.

[00:21:56] You can’t just use the last observation as your method [00:22:00] of estimation. That’s I think that’s generally agreed. You know, a treatment, a treatment policy, you need, you need to get the outcomes of the patient. And if they’re missing. You’ve got to do some imputation or make some assumptions about what would have happened to that patient.

[00:22:15] You know, using some kind of multiple imputation strategy to try and get to get a valid estimate for a treatment policy strategy. I think what, what, what, what’s being described when you talk about last observation, actually. corresponds. It’s an estimation method more for a composite type strategy, I would say, if anything.

[00:22:34] I’m not necessarily saying it’s a good way of estimating a composite strategy, but in some senses, you’re, you know, if you’re using the last observation, you’re penalizing people who, you know, if you say they discontinued treatment and will use their last observation. If that’s a bad observation, your kind of somehow saying, well, this is a composite and, you know, we’re trying to, trying to say discontinuation is a bad thing.

[00:22:57] And for those people who discontinue, we’ll, [00:23:00] we’ll plug it in their last observation.

[00:23:02] Alexander: Well, that assumes that the patients improve. Yeah,

[00:23:07] Oliver: well, we’ll get all this reflects the discontinuation part is reflected in the last observation. I mean, that’s an argument. You might. I’m not saying I agree with it, but it’s an argument.

[00:23:17] You could make that last observation is a type of estimation strategy for a composite. You know, you do have to be very clear then. we’re saying discontinuation is, you know, pulling, making that part of the end point and as part of the end point, you know, how we’re going to estimate it is to penalize it.

[00:23:34] And we’re going to penalize it by using the last observation. I mean, that’s, that’s the kind of argument you’d have to make, but you can’t, I don’t think you can say that last observation analysis is an appropriate analysis for a for treatment policy. I don’t think that’s.

[00:23:47] Alexander: I think my point is this, you know, if we look into continuous data over time, we can have two different scenarios.

[00:23:57] One is and that is [00:24:00] what lots of people thinking about is kind of, we have patients that have very bad symptoms at baseline, and then we give some treatment and the symptoms Yeah, and then usually the patients that drop out early have least improvement. Now there could be another scenario where patients have some kind of stable symptoms and they actually worsen over time.

[00:24:31] And the treatment delays the worsening or decreases the worsening. And then, of course, any patients that drop out early have better outcomes. Yeah. Yeah. So, so I think in terms of what you penalize depends a little bit on these two different settings.

[00:24:52] Oliver: Yeah. Oh, absolutely. Absolutely. Yeah. I, I, I entirely agree.

[00:24:56] You know, and that was always one of the, one of the problems, you know, if we go back [00:25:00] in time when people were using last observation, carry forward in these kinds of areas where diseases get worse. That was, it’s not a, I mean, it’s, it’s not a, it’s not a good strategy in that sense. As you say, it wouldn’t penalize people.

[00:25:11] It would actually benefit people who discontinue early. If, if you’re in the situation where people are getting worse, you know as, as time progresses. Yes. I mean, then clearly it’s not appropriate. You’d have to think of some other way of penalizing.

[00:25:23] Alexander: Yeah. Yeah. And so that is That also clearly shows that you need to look into the clinical aspects here.

[00:25:33] Yeah. Yes. Is it, is it a treat, what is usually the course of treatment? Yeah. Do you have, you know, a very fast response and then, you know, a decline thereafter? Or do you have a do you look into reducing the, you know, worsening of symptoms? All these kind of different things. You need to take into account and you can’t just say, well, that is better than this. [00:26:00]

[00:26:00] It really depends on the, on the clinical setup. There’s this one other concept that thing speaks about this one patient, one vote. What is your viewpoint on this and what does it actually mean?

[00:26:16] Oliver: Yes, I mean, I think it always sounds attractive, one patient, one vote. Everybody likes to be treated equally.

[00:26:22] And so, I mean, I think we make clear in the article. That we think one patient, one vote should be applied to the S demand itself. So that doesn’t mean to say that when you do, when you do an estimation, you have to wait everybody equally in the analysis. It’s the S demand where it’s one patient, one vote as such.

[00:26:42] The point here is that you might be able to estimate somebody’s. outcome better by using other patient’s data. So when your estimation method, you can actually weight other people because, because their data will give you more information about, about the patient who’s got the missing data, for example.

[00:26:59] So [00:27:00] I think again, that’s an, and that’s an important point. I mean, I think, you know, we’re saying one patient, one vote for the estimate, not for the estimation.

[00:27:09] Alexander: Yes, that is a very, very important thing. And honestly, we are not here. That’s not a democratic vote. We are interested really in understanding what’s going on.

[00:27:22] What do you want to

[00:27:22] Oliver: find the best way of estimation? You know, I mean, the other point, I think with the paper, it’s very critical of, of MMRM as a method of estimation. and I think actually it can be, it can be useful, particularly for treatment policy estimates. Actually you know, recent work that’s been going on within the Estimands Implementation Working Group, it’s coming up with some very good, good ways of actually using MMRM for estimating treatment policy estimates, where you’re dividing the data into on and off treatment periods.

[00:27:54] I think, I think it, it, it’s. It, it worries me that, or it worries us that [00:28:00] MMRM was so heavily dismissed in this, in this paper. Whereas when I think it’s actually can be a very useful method of estimation. I think,

[00:28:08] Alexander: That speaks very much to the point of how we communicate these more complex approaches.

[00:28:15] Yeah. Of course, last observation is very, very easy to communicate yet. That doesn’t, shouldn’t make us you always use and go the easy route. Yeah, that shouldn’t be kind of just because it’s easy, we do it. Yeah, if it’s not appropriate. Nevertheless, I think as a community, we can improve a lot on how we communicate to more complex approaches.

[00:28:47] I have seen so many publications that speak about, Oh, the estimate is a multiple imputation estimate. I was thinking, like, [00:29:00] what the hell have you imputed here, yeah? Under what assumptions? Multiple imputation is just a very general framework. It doesn’t tell you how you impute. Under which assumptions, when do you impute and so on.

[00:29:18] And so, we should go back from just, you know, just stating we have used statistical technique XYZ to what does it actually tell you? Yeah. And I think that is where we as statisticians need to get from our technical into more kind of plain language, yeah. And explain it this way. Yes.

[00:29:43] Oliver: I mean, you know, we agree, obviously agree, you know, communications are key.

[00:29:48] And I think, you know, that goes back to what I was saying about the hypothetical thing. I think there’s a lot of misunderstandings about what that’s actually trying to estimate, what treatment effect you’re targeting with a hypothetical [00:30:00] strategy.

[00:30:00] Alexander: Okay, very good. So as a final summary, what are kind of, what is your key takeaway for from this whole discussion about continuous outcomes and estimates?

[00:30:16] Oliver: Difficult question. I think it shows there’s still a lot of debate, even within the statistical community about these issues, and that there are still some, some parts of the community that are still in favor of things like last observation, last observation carried forward. I think, I think the estimates framework itself is a, is a.

[00:30:39] Massive step forward in trying to think about how we estimate treatment effects in clinical trials. I think the next stage is really around. For me, it’s thinking about cause, you know, causal inference as well, which is this sort of not really covered that well, not covered in detail in the ICHE 9 framework.

[00:30:59] But I think [00:31:00] that is, that’s very important to me, you know, to think about causal questions as well as questions of treatment policy. And I think, you know, going forward, that’s going to be a major area where we, you know, as statisticians, we need, we need to get involved. Completely agree.

[00:31:16] Alexander: There is still a lot of education needed within the stats community and even more outside of the statistics community.

[00:31:27] And so invest in answering and questioning what is really the problem here in the Course about estimates that I’ve recorded together with Kasper Rufibach. We go a lot into this and he always says if, if people kind of write him a quick email about, Oh, what should we do? It usually ends up in a, in a, in a meeting where he asks lots of, lots of different questions and say first, I think we first need to really think about [00:32:00] this a little bit more.

[00:32:01] And so it, this thought process. Is the biggest benefit, I think, you know because then we really get clear on what we really want, rather than kind of directly going to the estimation problem and with that completely confusing the overall problem solving technique and. All the discussions that we had today really clarifies this.

[00:32:29] So thanks Oliver for that. And we will put the link to the references, a link to your LinkedIn page and all the other things into the show notes. So check out the show notes on Thanks so much, Oliver.

[00:32:48] Oliver: Well, thank you very much, Alexander, for the opportunity.

Join The Effective Statistician LinkedIn group

I want to help the community of statisticians, data scientists, programmers and other quantitative scientists to be more influential, innovative, and effective. I believe that as a community we can help our research, our regulatory and payer systems, and ultimately physicians and patients take better decisions based on better evidence.

I work to achieve a future in which everyone can access the right evidence in the right format at the right time to make sound decisions.

When my kids are sick, I want to have good evidence to discuss with the physician about the different therapy choices.

When my mother is sick, I want her to understand the evidence and being able to understand it.

When I get sick, I want to find evidence that I can trust and that helps me to have meaningful discussions with my healthcare professionals.

I want to live in a world, where the media reports correctly about medical evidence and in which society distinguishes between fake evidence and real evidence.

Let’s work together to achieve this.