The Future of Statistical Methodology in Drug Development

Jenny Devenport

Global Head of Methods, Collaboration, & Outreach (MCO) at Roche

Jenny Davenport is a problem solver with more than 20 years of experience as a statistician across the healthcare industry, including public health, medical devices, drug development, medical affairs, and market access. Her work is driven by a commitment to asking the right questions and developing effective solutions that help patients receive the best possible care.

She champions scientific curiosity, the use of diverse data sources, robust measurement strategies, and rigorous study design to support better decision-making and innovation in healthcare.

As a leader and coach, Jenny empowers individuals and teams to exceed their own expectations by encouraging them to step beyond their comfort zones, recognize and leverage their strengths, build strategic collaborations, cultivate extensive professional networks, and create meaningful impact at both local and global levels.

---

David Wright

Head of Statistical Innovation at AstraZeneca

..

.

.

---

Jurgen Hummel

Vice President Innovative Statistics at Cytel

Executive statistical leader and consultant with 30 years’ experience of working within CRO, pharmaceutical and health care industry.  Proven track record for consulting support for pharmaceutical and biotech clients, particularly relating to drug development and study design planning.  Experience of successfully setting up and growing statistical methodology teams. Strong network amongst statistical leaders within the pharmaceutical industry and regulatory agencies.

Transcript

Speaker 1 (00:03.406)
Hello there and welcome to the Effective Statistician podcast. And today we have a very special edition. We don’t just have one guest, we actually have three guests. And not only that, they’re statistical methodology users, which is a subject which is close to my heart as well. And in a moment, I’m going to get them to introduce themselves. But just to say that this podcast is…

is being recorded due to the publication of a paper entitled Statistical Methodology Groups in the Pharmaceutical Industry, which has just been published in statistics in the biopharmaceutical research. And a link to the paper will be included in the show notes. So I am going to ask people to introduce themselves in turn.

Speaker 3 (00:57.664)
Okay, so hello everyone. I’m David Wright. It’s great to be here. I’m the head of statistical innovation at AstraZeneca based in Cambridge in the UK. I lead a team of expert methodologists and other statisticians based in Cambridge in the UK, Gothenburg in Sweden, and Gatorsburg in the US.

Okay, I’ll go next. My name is Jürgen Hummel. I’m a methodology statistician at Sightel in the statistical consulting group. I previously headed up a statistical methodology group at a CRO. That was PPD, now part of Thermo Fisher for seven years and also at a large pharma company, Novo Nordisk, for a year and a half. And I’ve worked in the industry for about 30 years overall, originally from Germany, but I’ve spent my entire working life in the UK.

I’m Jenny Devonport. I’m a statistician by training. I’ve worked in diverse settings for a long time, public health, device and drug development, medium and large companies, individual contributor roles and in leadership roles, and obviously most recently in methodology leadership, really just dedicating my career to ensuring that appropriate methods are available to solve problems that matter.

Perfect. And as I mentioned in the introduction, there’s this paper that’s just been published. And I wonder if you could just give me, I’ll give the audience, I’ll give the listeners, I should say, not just me, some top line headlines of the paper. But not only that, just give me maybe some sort of summary of it.

What we were trying to do with this manuscript, which has been authored by the FSPY Statistical Methodology Leaders Group, is to explore the remit and value of dedicated methodology groups, the organizational considerations that are necessary to address and maximize impact, the characteristics that methodologists need to influence speed and efficiency,

Speaker 2 (03:11.776)
in drug development. And our goal here was just in general to address what these groups do, why they are needed, and how they add value. Juergen or David, do you have anything to add?

I think that’s a very good summary overall, Jenny, and it was really good that you were the main drive behind putting that forward. I think what we were trying to do from our group is sort of explore the similarities in how we were setting ourselves up and also where maybe some of the differences. for example, I work

both in a large pharma setup and in a CRO setup. And there’s a lot of similarities in the sense that the people in these groups have a sort of really deep methodological background in there, but the focus areas, they may be slightly different. So in a CRO setup, it is typically focusing on client concerns to try and help them speed up maybe stages of development with innovative approaches.

in a large pharma setting that’s maybe then targeting specific therapeutic areas more, the areas that the company is working in. And another sort of difference is maybe that in a CRO setting, it’s often a bigger focus on implementation of published innovative approaches and tailoring them to the client needs. in a large pharma setting, there’s maybe

and sometimes an approach of more working with in collaboration with academia in order to develop new methodology. that is a more long-term approach that’s maybe less common in CROs. David, any thoughts from your side?

Speaker 3 (05:12.142)
Yeah, maybe just a little bit of thank you on the longer term aspect of what you were talking about. So yeah, that is important and can be a tricky part of our work is getting buy in from senior leaders to have a sort of a three year roadmap or a five year roadmap towards success in adoption of a new approach. And so that’s and so the paper goes into details about

how we have to work across from invention all the way into implementation. And of course, as you were alluding to Jürgen, there are many different ways of doing that in Big Pharma or CROs, but in Big Pharma, we might be working a little bit more from the left-hand side of that equation, if you like, but we still also have to tailor what’s developed to specific project needs, which is in effect very similar to what you’re doing in CROs.

There’s lots of similarities there. Maybe I could touch a little bit on the training aspect, which is a great opportunity that we have in Big Pharma because we have such a big audience. But obviously that also creates difficulties in how you can educate people or inform people across various different sites across the whole world and then also drive that adoption so that you get

Not standardization necessarily, but the use of these methods in quite a broad fashion, which is what we’re all after.

fantastic and that’s some great aspects of just a general remit of what you know what you guys do in both Big Pharma and CROs. I wonder if you could give me, I’ll give the audience I should say again, give the audience some idea of what’s it look like to be a great methodology statistician and what would a typical day look like?

Speaker 3 (07:25.326)
So yeah, maybe I can start by saying that, I mean, I think the one thing that attracted me most to this area is that I wouldn’t be able to say exactly what a great day would look like because each day will be different. So there’s an amazing variety of what we’ll be working on. So I think that’s one of the things to stress, which I think is fantastic. So one great day might be where somebody calls you up or meets you at a coffee point if you’re in the office and you have a short consultation with someone.

and immediately solve their problem. Or you might be spending a whole morning with a study team where you’re addressing a very difficult problem that you’d be working with them for maybe months or several weeks on. So that that might be another part of the day. And then maybe you’ll have other days where people don’t contact you and then you’re working on new methods or maybe writing that conference presentation.

or maybe making some corrections to a paper that you’ve written. So there’s all of these different aspects that you might be working on. And I think that the only difficulty maybe that some folks find is sort of not having that concrete knowledge of what they will be working on. But I feel that’s a particular strength of working in a methodology group.

I think what you mentioned there, David, about variety is certainly very true. And that’s what I’ve experienced for many, many years. And ultimately, what I found really exciting is being able to solve often complex problems. I’ve always been excited about adaptive designs. they’ve sort of really become a buzzword about 20 years ago.

What can be a great day is when you have a discussion with somebody who comes with a particular problem and during the discussion you then sort of as you drill into it get an understanding of that there may be a solution that is around to help solve that problem. So maybe a particular type of adaptive design and the ability and then to see there’s a path forward to explore that.

Speaker 4 (09:48.568)
further with a wider project team with simulations with more details and then to figure out there is a solution. And ultimately, when you see that, come to fruition and help the project team with the implementation of that. I think that is what I found most rewarding.

Speaker 2 (10:13.836)
Maybe I jump in here too. I agree with both David and Jurgen that consultation can be extremely rewarding. That ability to collaborate closely with teams that are working on projects and help them solve new challenges that always seem to arise in drug development is important. But something that I also enjoy is how to take that problem and recognize that

it’s going to need scaling and it could be even more impactful on the entire portfolio if everybody could do it. And so that process of developing education plans, of creating software, of creating subject matter expert networks that aren’t just methodologists but also project statisticians who can really help advocate and roll out the new methodology within a company.

so that everyone benefits and everyone feels a sense of ownership and responsibility for implementation. That to me is really exciting. And I think that’s a special feature that a dedicated methodology group that’s high functioning can offer.

And maybe I can build on that, Jenny, because and maybe I’m to be a bit controversial here and hopefully not too controversial, There may be senior leaders out there that are listening to this that say, I’m getting methodology groups. Fantastic. I get the idea, but they’re nice to have. How would you respond to those people?

Speaker 2 (11:50.424)
So maybe I make a start. And this is heavily addressed in the introduction of our paper. Our industry has an innovation problem. And it’s related to efficiency. And it’s related to the high failure rates in drug development. The costs of drug development are driven very extensively.

by the different stages of trial investment. And of course, your biggest investments occur in late stage. But the latest statistics suggests that only 60 % roughly of phase three trials are successful, which means there’s a lot of room for improvement. Okay, so we have this innovation problem and this is where methodologists and statistical thinking in general can help, is to help drive decision-making. And so if you think that it’s a nice to have,

then maybe you’re not willing to address this 40%, but it would be really good for drug development if we could.

If I add to that is, so I’ve spent most of my working life in a CRO setting, and it’s probably fair to say that not many CROs do have a statistical methodology group. DPD and Cytel are probably exceptions in there. So many CROs focus more on the operational aspects of delivering study. However, I think that misses an aspect that a lot of

clients want to have an efficient way of running their studies. And if, as a CRO, we can help them in that, that then provides a head start for the actual operational delivery and running. So it’s absolutely not a must-have in a CRO setting, but I think it does provide a competitive advantage by having that ability to do that.

Speaker 3 (13:53.704)
yeah, just to add to that, I think in Big Pharma, one of the advantages of a methodology group is that it can span multiple different parts of the business. Statistics is sort of fairly agnostic to a particular therapy area, but people who work in particular therapy areas think that the methodology they work on doesn’t get used anywhere else. so we’re able to make those links.

And sometimes that can lead to advances in different areas or just awareness. We can translate one problem in oncology into one in cardiovascular, for example, or whatever, different areas. And so we can provide that link. And as Jenny was talking about, when we’re deciding which problem should we scale, obviously we would like one that’s going to be broadly applicable.

across a whole range of different areas of the business. And then I think that offers a very clear way for senior management to see the value of methodology groups.

sorry go ahead Jenny.

was going to mention an additional point that was brought up by our American Statistical Association colleagues in a discussion of the article. They and other discussants were advocating for an even larger remit of methodologists beyond clinical trials in acknowledgement of the fact that a lot of different areas of drug development could benefit from statistical thinking.

Speaker 2 (15:33.986)
decision-making, which I already talked about a little bit, but also operations, which Jurgen mentioned. There are lots of different places where we could gain some optimization or some efficiencies, and that would benefit from statistical thinking. different organizations choose to operationalize that differently. They may have their own groups that do that.

but they may still benefit from the statistical knowledge of a methodologist from time to time.

And that last point, Janice, is really interesting because I remember Andy Grieve talking about this when he first set up the group at Pfizer, talking about how he’d been involved with so many different interactions throughout the company. So I think what you’ve just said there brings all that back to the value of a methodology group. So if we think about the listeners now, and there’s going to be

statisticians who are young in their career or they may be slightly more experienced, they might be sitting there thinking, well how do I become one of these fantastic methodologists? So I’m going ask you the question, what makes a great methodology statistician?

Speaker 4 (16:51.594)
There’s obviously the technical knowledge first and foremost. In order to be a good methodology statistician, you need to have a deep knowledge of statistics and what are the underlying assumptions of different approaches. What’s the framework? What are the limitations? But then also the ability to adjust an approach as and when needed rather than just applying it in a very specific way. So that

technical skill is a great starting point, but I think it is only a starting point because there’s lots of other aspects to that. So Jenny, do you want to add on some others?

I think technical skills are the starting point, but having a deep understanding of drug development so that you know what questions to ask and you know what might be practical is also important. Having the ability and the social skills to be able to interact with lots of different kinds of non-statistician

personnel is really important. Having a sense of curiosity about the basic science, about the medicine, about the different players involved, kind of the intrinsic motivation to pursue problems and to pursue relationships, and of course, an extraordinary ability to collaborate. But David, I think you were ready to speak as well.

Thank you, Jenny. And so listening to this, it sounds like we’re looking for a unicorn or a sort of impossible superhuman. And so I’d just like to reassure the listeners that obviously we understand that people start from different places. so just to emphasize that people in our teams come from very different

Speaker 3 (18:44.088)
different backgrounds and to have different paths to how they’ve joined a methodology group. So sometimes they’re academics who then join industry. Other times, the people have worked in projects for a while or for several years, and then they’ve sort of, they develop some of that drug development knowledge by being in those teams, but also sort of want to go back a little bit to their sort of methodology roots, if you like. And so then they get.

the best of both worlds when they come back into a methodology group. So just to emphasize that it’s, and that everyone in the team is different and obviously having a diverse team is very important. So it’s very unusual to, or impossible for the whole team to have all of these skills that we’re talking about at the moment. So it’s sort of be a team exercise to be able to do that. But Jenny wants to come in at that point.

I wanted to say yes, Anne. So we know that the environment is changing and that there are a lot of additional types of expertise that are coming into the quantitative array in pharmaceutical development. One example is software development, the open source revolution in the pharmaceutical industry towards using

more open source software as opposed to licensed software to address innovative methods and get them implemented sooner. This is something that has come into play. And so some organizations have software development within the methodology groups. Some of them have them somewhere else. And then of course, there’s also artificial intelligence, which is finding its way into different

parts of organizations. In some organizations, they exist with the methodologists in an innovation group, and sometimes they sit in a parallel group. But certainly, perhaps there is benefit to those groups being synergistic, both in terms of the validation and compliance and in terms of just realizing all the value they possibly can. David.

Speaker 3 (21:01.174)
Yeah, so thank you for that expansion of what I was talking about, I fully agree with. But I’d like to say that one of the key skills that is relevant now as the world changes in an ever rapid fashion is obviously you would imagine methodologists will be very curious about new methods. That’s a key part of their role. But obviously, as you

things come in from the AI side, I’ve been delighted to see several people just lean in immediately to understand how these new technologies are working and to obviously be able to then provide expert guidance to various senior leaders about the value some things bring and be honest that the less value that other things bring, which is a part of our role to be a of a trusted partner in those conversations.

and I actually had on my next question around AI which is quite interesting and I’m wondering if we were to look into the future where would you see the methodology groups and AI working together and maybe it’s a case of where do the methodology groups have to evolve?

Speaker 2 (22:20.106)
It’s a question. And maybe I make a slow start.

One of the things that current models are very good at is automation. So when you already know what you need to do and you already know how to do it, you just need to be able to do it better, you know, more and faster. That is a place where today’s models can do a pretty good job of accelerating the work where they’re

aren’t as many questions about how to do it or why we’re doing it. Obviously, that’s not innovation, right? That’s implementation, that’s scaling. And so it’s this innovation piece that we have to see how the models of the future perform and how they can support methodologists and other scientists in their work.

to accelerate invention, to accelerate adoption. I certainly see the potential of these new technologies to help with scaling and automation and adoption. But I think the jury is still out on innovation, but I’m sure that I can be corrected on that.

it.

Speaker 3 (23:45.678)
Thank you. I’m not sure I’m going to correct you, but I just want to say that I think the important thing in one, two, three, five years time will be the driving aspect of our organization. So who is driving essentially? And you might say, well, no one is because sort of these automatic cars exist already. I think the crucial component is that I think it should be people like ourselves and our teams that aren’t driving.

the implementations, so by asking the right questions and then doing the research in the right methodologies and seeing how they can then fit in to the new technology rather than the other way around. Because otherwise we run the risk of looking at those technologies and sometimes them failing for reasons that we would readily be able to have foreseen if we were consulted. But we shouldn’t sit back and just wait for that to happen because we have to be big.

and bold and go for it and then we have the chance to lead in those conversations.

And if we look at some aspects of maybe machine learning as well as artificial intelligence, I mean, there are approaches that are becoming more popular, for example, in modern or variate adjustment that are machine learning based and that do have the ability to make our statistical analysis more efficient. Some of the questions that we hear

from regulatory authorities is ensuring that we still have a good understanding of what drives the treatment effect under certain circumstances in specific populations or even subpopulations. So I think that is where we as methodology statisticians can really help is try and get the understanding from the drug development perspective rather than simply the implementation of those approaches.

Speaker 1 (25:46.124)
And I’m going to add on top of that AI issue, and you mentioned adaptive designs earlier on. My way of thinking is a human is a much better thing designing a clinical trial, particularly an innovative clinical trial, because an AI is only going to be working from what it already knows in the database. Whereas you might be thinking, well, actually, that’s not going to work.

And maybe this is going to work. Maybe this is more innovative and maybe this is a better way of looking at this design. So to me, we’ve got to embrace AI. The gene is out of the bottle. We can’t put it back, but it’s case of working with it. And I think there are human things that we can do. And I certainly think designing an innovative clinical trial is one of those human things.

I’m fully with you there and since you mentioned adaptive design, there is no such thing as one type of adaptive design. I the ICHE 20 guideline is out as a draft and it already mentions many different areas, but there are additional types of adaptive design that are not even mentioned in a lot of detail in there because ultimately what you’re trying to do is try and

build a study design that addresses the main area of uncertainty that you have at the beginning of your study so that you can take corrective action if needed during the study rather than just seeing the result at the end and finding out that, this is not what I needed or what I wanted. So in those situations, you’re absolutely right, Alan, is the ability to try and understand where is that main area of uncertainty and how can I choose

one out of many different types to try and address it.

Speaker 3 (27:37.996)
Yeah, and just to add to that, I mean, it’s a remiss of me not to mention the work we do in collaboration with regulators as an ex regulator. And as Jürgen pointed out, even if people within the company think that they might be able to adopt various things, other people will be asking questions of a more technical nature, which we are very

with the best groups to advise on. So I think that’s very important thing to flag up as well. So I think there’s a lot of scope there that obviously everyone will be learning about new approaches and it will be a sort of roller coaster ride as people learn more about them. Anyway, I like roller coasters, so that’s fine. I look forward to the ride.

Maybe I can, I’m going to move away a little bit here and just, you know, your three methodology leaders, statistical methodology leaders. What, I mean, just a guess, what percentage of companies, this is really difficult because it’s obviously a lot of small companies. If you were to guess what percentage have statistical methodology groups, what would you say?

Speaker 2 (28:56.526)
I can’t give you an exact number and you know statisticians hate guessing and gambling, but I would say that they are fairly common in large pharmaceutical companies and maybe less common in medium and smaller biotech companies.

would completely agree. And if we then look further, I already said earlier that they’re not very common in a CRO environment. So even amongst the sort of medium to large size CROs, they’re not necessarily that common. So I think the biggest proportion is, you’re right, in the large pharma setting.

And it makes sense, right? Because a larger pharmaceutical company is going to likely have a more diverse portfolio. They may be pursuing more complex targets. And so they may have more demands to streamline and centralize methodological development and scaling.

And so this is, I gotta say, this has been a fantastic discussion and I hope the listeners will love this. But I wanna go just some final, as we’re starting to wrap this up and I know we’ve touched on it already, but what is the true goal of a methodology department?

I think ultimately what unites us is that we all want to make drug development more efficient. And I think that’s been the driving force of the FSPY stats methods leaders group being established in order to help us jointly achieve that. Because if we are able to join force and maybe have help with the spreading of new methodology and helping with

Speaker 4 (30:57.272)
some of the discussions with regulators jointly, that is gonna be a big step forward for all of us. for me, that is the biggest aspect.

And for me, ever since I started in statistics, I’ve always looking to apply something as it were. So if it was just theoretical and wasn’t applied, although it might be beautiful and very clever, I would always be deflated in some way that hadn’t been used. And so I think the biggest challenge that we face in the pharmaceutical industry is the slow adoption of a very beautiful things. And so that’s the

you know, the most important thing and goal is to speed up that adoption of these fabulous new approaches being used in the right situations, because as we’ve said, it shouldn’t be used everywhere, just finding the right situations to use them. And so that’s that’s the beauty of the game, as it were, that we’re in. And we just need to get even better at that, because as I’ve been talking about it recently, some of those things have been incredibly slow, which for various reasons. So

we can make those timelines much sharper and then we can innovate more quickly.

And David, if I can probe on that, but how are we going to do that?

Speaker 3 (32:21.1)
Better connections between various groups is one aspect which we’re all working on as part of the FSPY Stats Methods Leaders Group and was at an FSPY Stats Leaders meeting in Munich earlier in the week. So we’re thinking about ways of doing that. it’s partly those connections. It’s also about better communication of methods so we can clearly articulate them to non-statisticians in a better way.

So I think if we do that, we’ve got the potential to vastly speed up the adoption of new approaches.

I’ll take a little stab at it too. You know, the traditional academic route is you invent a new method, you write a paper, you present it at a conference, and then a miracle happens, right? Or everyone ignores it. But I think the goal of methods development in drug development is deployment, not invention, right? So a concept that I have found really helpful.

is it’s not mine, it’s ours. It’s not my method that I developed, it’s our method that we’re using together. And so where you see that play out in groups like the FSPY Statistical Methodology Leaders, our American Statistical Association counterparts, is the collaboration on specific topics. There are special interest groups dedicated to the

kind of further development and implementation of specific methodologies, this collaboration that we don’t have to do things alone. In fact, we’ll be faster and more thorough if we do it together. So all of the steps of the innovation cycle benefit from collaboration, whether it’s internally with project statisticians in your company, which is huge, or also externally with academics, with other statisticians in appropriate contexts with regulators.

Speaker 2 (34:28.174)
You can really take things further faster because inventions don’t add value until they’re adopted at scale.

And so if you mentioned the SPI leaders and the SPI stance leaders, if listeners are listening to this and they say, how can I connect with the SPI? How can I connect with SPI? That’s a question. How do you connect with the SPI leaders and the SPI statistical leaders?

Well, the FSPY have a website and there’s details there about the stats leaders group. There’s details there about the stats methods leaders group with contact names of the people that lead to those groups. So that’s the best way. Have a look there and contact the people that are mentioned there.

and Jürgen I will put that in the show notes. And how about the three of you? What’s the best way to connect with you?

LinkedIn.

Speaker 3 (35:29.068)
Yes, that works.

Yeah, LinkedIn. We’re all on LinkedIn.

hear nods from everybody. So I’d like to take a moment to really thank you for coming along to this edition. like I said, a very special edition of the Effective Statistician. I know the listeners will be delighted with this episode. And actually, what I’m encouraging the listeners to do

is if they like this episode, put some comments in the show notes, put some comments in there, well not in the show notes, but put some comments to say what you liked about this. And if there’s any questions you’d like to ask Jenny, Juergen and David, then maybe put those in there as well and we can ask them on a future episode or in some sort of chat forum. So thank you very much. And I’m going to say goodbye and thank you everybody else for listening.