Driving Statistical Innovation – Barriers And Strategies Part 1

Transcript

Driving Statistical Innovation – Barriers And Strategies

PART 1

[00:00:00] Alexander: Welcome to another episode of The Effective Statistician, and today I have some pretty prominent guests here Mouna and Kaspar. Great to have you. Mouna, how are you doing?

[00:00:15] Mouna: Good morning. Very well, thank you. Yes, sir.

[00:00:18] Alexander: Very good. Kaspar, you have been on the podcast before a couple of times, so great to have you again.

[00:00:25] Kaspar: Yeah, thanks Alexander for the opportunity to share again a few things.

[00:00:30] Alexander: Yeah. Today we are talking about statistical innovation and there are hardly any two persons that would be better to speak about this than Mouna and Kaspar, given that they are both leaders of statistical innovation groups. in two big pharma companies. And they’re currently both sitting together in Basel because they work for Roche and Novartis. Starting with Mouna, can you give us a little bit of an introduction to yourself or those people who don’t know you yet?

[00:01:03] Mouna: Of course. As you said already, my name is Mouna I am working for Novartis based here in Basel in Switzerland. I’ve been with the company for around 12 years, have always been working in a group called Statistical Methodology within the analytics organization and drug development at Novartis. And five years ago approximately, I took on the leadership of that team and very much related to this topic, one of our key areas of Scope, let’s say, is around statistical innovation, driving advocacy for, yeah, innovative solutions in drug development. So looking forward to have this discussion today.

[00:01:46] Alexander: Very good. Kaspar for those who missed the earlier episodes with you.

[00:01:51] Kaspar: Yeah, so my name is Kaspar Rufibach I work for Roche in Basel as a biostatistician. I joined Roche 11 years ago and initially worked as a trial statistician for a couple of years, and then joined the methods collaboration and outreach group at Roche. And within that capacity, I do a lot of consulting for clinical trial teams. I give a lot of courses for statisticians in and externally. I do research jointly across industry with colleagues in other companies and with academia. And I’m interested in trying to improve clinical trial designs.

I do a lot of work in survival analysis and with estimands and and postal influence and. We develop a lot of methods and bringing this innovation into the company turns out to be a not too easy task. So I’m very happy we have this discussion today and can share some learnings and maybe also get some inspiration from the audience later on.

[00:02:44] Alexander: Yeah, that’s a good influence to our discussion actually. You create a lot of innovation. You bring a lot of innovation, what that’s happening outside of your company into your different companies. Why are you so concerned about senseis Innovation is not picked up, and maybe a little bit before that, what does really innovation mean to you?

[00:03:13] Mouna: So maybe I take a start and then Kaspar can jump in. So I think both Kaspar and I have talked about this topic before, and I guess we would come to a very similar definition of what we mean by innovation. So I think very often innovation is created with invention. So something new or novel that can improve maybe the ways of working or accelerate, let’s say, drug development in our context and make it more efficient.

So that’s one way of thinking about it. However, it, I think last year I saw someone explaining innovation as being more often combination of invention as well as, Commercialization where commercialization is essentially a key driver to create value out of an in invention, right? So just having an invention alone is not sufficient if we are not able to implement it to create uptake and positive impact.

Likewise, just being very good and commercializing something, but not having a great idea that actually improves something is not going to lead anywhere. So it’s really like the interplay between invention as well as commercialization that then ultimately leads to innovation. And I think that’s, for me at least very helpful to think about it this way because the skills needed for invention may not be the same that are needed for commercialization. You may not need the same team for both, but rather have collaboration. Yeah, that’s the way I understand innovation. Think about it, let’s put it that way.

[00:04:50] Kaspar: Yeah, maybe I can add to that. Just to Mouna’s last point. I think traditionally statisticians by our training and by how we see us, we focus a lot on the invention piece.

We invent a lot of great stuff, new methods, and try to improve trial designs. And over time I sense there is some frustration that because we, through methodology, through simulations, we show this new method is actually better than what’s currently out there and still people are not using it. And that generates potentially some frustration.

And that I think let us think about how can we do this change? And of course, not every new statistical method or invention needs to be broadly applied, but I think there are things that we should apply more broadly in pharma industry in terms of statistical methods because drug development. Becomes ever more competitive, 20 years ago you had all these new molecules that had huge effects in broad populations, and then clinical trial design is not so difficult. You have the impression but meanwhile there many therapeutic areas are so competitive that you need to tease out effects and you need to optimize trial design.

So invention and innovation out of these inventions becomes ever more important. And add to that, that we now rely on many more data sources. So it’s RCTs the traditional backbone of drug development. It’s just one thing. We have now real world data. We have imaging data. We have many different kinds of data we need to integrate and that also needs inventions and ideally also innovations in order to make best use of all these new data types.

[00:06:32] Alexander: Yeah, completely agree. Another area where I see a lot of need for innovative trial design is, for example, rare diseases. Whereas there’s a lot of concern about can we get enough patients and do we make, good use of these patients. And as you mentioned external controls are a big part there as well.

Is there an example where you are specifically frustrated? Something that was invented a long time ago and still hasn’t been broadly implemented within pharma companies?

[00:07:10] Mouna: I think maybe just to kick it off and then Kaspar can of course add, so just because you mentioned rare diseases, and while Beijing statistics is by no means my area of expertise, but of course I know that there are several people in industry and also Novartis who are work on this extensively. And I think that would be one of those examples where the invention itself has already been out there for ages, you could say.

There are many approaches that help us to leverage trial external data, combine it and synthesize data in a very efficient way. And Beijing statistics can play a very important role there yet. The uptake is missing a bit. And I think that’s where that equation, again, helps quite a bit where we could say the invention is really already there maybe. And of course you could explore more and add more inventions, but maybe that’s not where the main effort should lie. And maybe the main effort should lie in the commercialization piece. So how do we create acceptance uptake for these methods? How do we drive trainings? How do we get key opinion leaders as well as regulators on board to see the value And maybe using some of these approaches while being transparent about some of the risks also.

So I think that’s probably an area where some colleagues are frustrated for good reasons. And we are, it’s, in my opinion, not necessarily so much about the invention piece. It’s more about the commercialization piece.

[00:08:35] Alexander: That’s a very good area if I’m just thinking about invention and commercialization. Basically it’s it’s a product. Yeah. And you have one factor is invention and the other factor is commercialization. And the product of both is the values that you create. And if you focus only on one. Yeah. But commercialization is zero. Cancels out too. Yeah. That’s a very nice way of looking into it.

Kaspar. Any kind of areas that you are most frustrated about?

[00:09:11] Kaspar: Yeah, I can share a few. I think. And maybe I can make a plot for an earlier podcast I was on with Young Byman where we talked about the Savvy project. Yeah. And I guess many people have heard me talking about this if we think about how we report safety information in clinical trials, we just count the number of events divided by the number of patients. That’s what we call the incidence proportion. We know this is only a valid or an unbiased estimate of the true adverse event probability under very restrictive assumptions that are never fulfilled. And the methods, how we can deal with this, actually they exist for 50 years. It’s just basic survival analysis methods.

And for me, this is very frustrating. I think we just give biased estimands of probabilities. And why is that? I think we later we will talk about hurdles maybe for uptake and commercialization. But this is, for me a really pressing area. I also looked into the literature a little bit and there you find examples where people exactly discuss this lack of commercialization.

One other topic is pediatrics. Using information generated in adult trials to inform pediatric trials where we know we have fewer patients. It’s, I think similar to what Mouna talked about for rare diseases, and there are publications that say the methods have been there for 20 more years until they have really started to be applied in a systematic way in drug development.

Further examples that maybe you can mention are adaptive designs. The theory for adaptive designs exists for a long time, and it’s not so straightforward and so easy to implement adoptive designs in drug development. I’m aware of that, but. Still, I think there is room that we can apply them more.

And that is also reflected by ICH now writing a new guideline on adaptive design. So it seems there is potential to use these more. And then the last point maybe I wanna make is causal inference has been developed for the last 30 more years. And we just start now to pick it up in drug development and in randomized trials.

And the ICG nine estimands addendum helps with that, with introducing Estimands but I think there is still a way to go and more that we can commercialize within statistics and pharma industry.

[00:11:23] Alexander: Yep. I could probably add to that from myself but that would be even more frustrating. But let’s dive into a little bit more the topic of driving this innovation. What kind of typical barriers do you see?

[00:11:39] Kaspar: Maybe I can start this time. Because we at Roche we are writing a paper on this topic. So I looked also in the literature what people identified as barriers and you can bring a long list. So maybe I can just focus on a few things. The first thing is insufficient knowledge. If in a methods group you invent something that doesn’t immediately mean that people pick it up.

They’re very busy within their projects and overwhelmed by project work and the next filing. So that’s one thing. Insufficient knowledge that goes hand in hand with initially when you invent something. There is often the question, and maybe not even from the statisticians, but from partner functions, do we have precedence?

If you invent something new, of course you don’t have precedence because it is something new and this is sometimes. Not easy to overcome.

Another aspect, depending on how involved complicated the invention is lack of software. And that’s why I think this part of, and let me first talk about the hurdles and then maybe we can talk about how to overcome the hurdles. But if you invent something, having high quality accompanying software is a key piece nowadays, because otherwise, if you first need to write the software yourself, validate it of course that’s not gonna work.

And then there is. Yeah it relates to that point about precedence. Of course there’s a lot of inertia. That’s what I see with safety. We have built this huge machinery how to report safety data. We have built processes around it. We have whole departments around it. If you don’t come and say we should maybe try a new approach you need to work on many things.

It’s not as statisticians. Sometimes we feel I’ve put this method out there, I’ve shown it works better than what we do today. So everybody should immediately understand we should now do this. And I think we also as statisticians need to work on that perception. This is not how things work. You need to actively do the change management.

You need to bring stakeholders aboard. Not and there are also other aspects to these things than just statistical aspects and we need to work on these as well. So these are hurdles that come to my mind, and I’m curious to hear what Mouna has to own.

[00:13:46] Mouna: Not there’s too much to add, but maybe a few things. So just following up maybe on your last point. I think that’s a really good one, right? Because we are sometimes so much focused around the quantitative sign perspective, and we don’t think about the whole system, let’s say, and all the interplay with the different functions, the different the different other collaborators, let’s say.

And that’s sometimes a barrier because we don’t bring them on board early enough. We want to have a perfect sort of solution first, and then we start reaching out. And that’s sometimes not the best approach, or, at least from my own experience, right? I feel like that the first impression.

Usually counts a lot with these things. And so having talked with these functions already. So it’s this looking at the broader picture, I think, which is sometimes, or the lack thereof, which is sometimes a barrier. Then also just to echo what what Kaspar said about this risk aversion, right?

Anytime we have something new and we work in a very regulated environment so it’s not only we can’t innovate By ourselves. So that’s why we collaborate a lot with other companies, but also with academic collaborators, with regulators, because in order to innovate anything, any aspect of work development. And that’s, I think very important. If you do it early enough, you can then maybe alleviate some of that concerns and the risks. I think the other thing that I have paced a bit sometimes in my long or short career is the fact that sometimes we focus very much on short term goals, where this long term goals, and I see that a bit as a barrier for innovation.

Because if you want to innovate, yes, it takes time, it takes resources, it is associated with a lot of risk. You may start with several things and not all of them. Play out relevant or important or, you may start certain collaborations with certain universities and so on, but not everything will actually play out and have a positive impact on drug development.

And I guess that’s something that then comes from the leadership also of the organizations, right? If they are if they want to foster innovation and they want to foster long-term strategic thinking, then I think they’re important.

And maybe just one example to illustrate that. For complex, innovative designs, but also for model informed drug development. There is now one on wheel write evidence. And these are all opportunities to learn jointly at the drug development community and how to use innovative approaches and drug development.

We can engage with the regulator in a more expanded discussion, but of course that also means time on the side of the project teams that want to go there. Of course, my perspective is then thinking about the broader impact, right? This has an impact overall on, on other projects, also on other disease areas, but the teams themselves that you need to maybe convince to embrace that journey, that’s a different thing, right?

So we think more about we and the whole sort of drug development community, but the teams themselves have really their own product that they want to develop as fast as possible, and I can certainly understand that, but there’s a bit of tension between these these aspects. And may I stop there?

[00:17:03] Kaspar: I liked one point that Mouna mentioned.

[00:17:05] Mouna: Just one.

[00:17:06] Kaspar: No, I like all points. But I want to reemphasize one point that all this is a matter of long breath. And I think as statisticians, and I observe this with myself, if you do something, you develop something new, you invent something if you want, then for you, the problem is solved.

You, otherwise, maybe you are better suited in an academic environment where the focus is more on invention. And once you have invented something, written a paper, that’s already a good result. But in what we try to move the needle in pharma industry and then we need to follow up on it.

And even if for yourself the problem is solved, you need to put in some energy to commercialize it. And maybe I can mention one example where Mouna and myself are heavily involved. These are estimands. I heard about this, I know, back in 2015 maybe. And then it was a very obscure thing, but talked about by some regulators.

And then you start to look into it you start to roll this out. We had the first courses at Roche maybe in 2018. A lot of people were asking, what is this all about? Does this matter? Is this not just a statistical thing? And just yesterday, I was part of a training academy at Roche for partner functions.

And now everybody is by themselves. They come to us and say, okay, we need to know about this. And that is very rewarding because we had the long breath and the patience and just, you have to wait and and hang in there. And then ultimately you will commercialize this in the whole organization.

[00:18:33] Alexander: Yeah. Completely agree with all these points. And I think there’s there are lots of examples of that. Yeah. If you can think about this fear of doing something new, what happens if we do this new and it doesn’t work out? Of course. People shy away from it because it means change. There’s only a very small group of people that embrace change. People that are always on the first guts, they’re always, wanna make things different. They’re advantageous.

Maybe they’re more courageous as well. Yeah. And these are the first people that take it up. This is a very small part of any big organization. And if you have working a cross-functional team, of course you need to have people from, that are that kind of people from all these different areas.

 And yes, yeah, you can have a great inventive idea, but if it doesn’t take into account that you have within companies, like timelines budgets systems software like you mentioned, yeah. It becomes really hard. And I think that is where this inertia that you meant is really in it.

Yeah. Inertia is a really nice way to think about it. There’s a famous book by Kitz that talks about inertia quite a lot, says that inertia is the always making things difficult and you always need to put a lot of effort into it. Just because we are right doesn’t mean it’ll get implemented. And just because here also simulations here, also facts here, all the kind of, academic case studies doesn’t mean you can convince people. It is really a what’s in it for them. And that’s where, change management is quite a lot about.

So it’s taking the people with you and looking into what’s in it for the person. Yeah. Or for the project team, like you said. Yeah. But the project team is incentivized by pushing their project forward. And that might be, very different incentive to what’s the overall company wants. Yeah.

We work in an area where most of our compounds fail. Yeah. If you think about drug development Yeah. Failure is the default. Yeah. Success is the, sad to say, outlier. Yeah.

Especially when you look into earlier phases and there’s a lot of, risk taking there. However, when it comes here to statistical innovation and sometimes there’s the perception, it’s the opposite.