Integrated Evidence Planning That Connects Drug Development to Patient Needs

Gorana Capkun

VP, Global Head of Patient Focused Real World Evidence at Merck Healthcare

Chris is a now consultant after having a industrial career of over 30 years as a statistician working throughout the drug discovery and development pipeline for multiple different companies. Chris has published widely and is a regular speaker at international conferences. As well as a long background in Clinical Statistics, over recent years Chris has spent time considering how more modern Data Science, Machine Learning and Artificial Intelligence techniques and technology can work in partnership with Clinical Statistics to derive the greatest benefit.

---

Transcript

00:00
You are listening to the Effective Statistician Podcast. The weekly podcast with Alexander Schacht and Benjamin Piske designed to help you reach your potential, great science and serve patients while having a great work-life balance.

00:22
In addition to our premium courses on the Effective Statistician Academy, we also have lots of free resources for you across all kind of different topics within that academy. Head over to theeffectivestatistician.com and find the academy and much more for you to become an effective statistician.

00:49
I’m producing this podcast in association with PSI, a community dedicated to leading and promoting the use of statistics within the healthcare industry for the benefit of patients. Join PSI today to further develop your statistical capabilities with access to the ever-growing video-on-demand content library, free registration to all PSI webinars, and much,

01:13
Head over to the PSI website at psireb.org to learn more about PSI activities and become a PSI member too.

01:30
Welcome to another episode of the Effective Statistician. And today I’m so happy to have Korana Kapkun from Merck on the line. She is the head of the Real-world Evidence Department at Merck, the German Merck. And we will talk about a very, very interesting topic, which I think all statisticians should have a close eye on because it’s very close to data.

01:58
And that is integrated evidence, planning plans. so, Gerana, welcome to the podcast episode. Thank you, Alexander. And hello to everyone who’s listening to us. Can you maybe start with an introduction of yourself? In the pre-chat, we found that we have the same kind of origins and pure mathematics, but where did you go from your studies? From my studies, yeah. So I left these pure theoretical mathematics to touch on.

02:28
computer science and then discovered statistics, which fascinated me because they tried to describe the world by using measurements. And I was first working in different actually areas outside of healthcare when I discovered epidemiology. So after my PhD in applied statistics, I moved into the field of epidemiology and was really fascinated.

02:52
about data that are already collected or that we are collecting to understand what is happening in clinical practice and try to get some meaning out of that or guidances out of that, especially when questions are geographically very diverse and wide. And from there, so I spent some time in academia, I spent some time in public health. So I worked in the Swiss federal office for public health, went really great.

03:19
invitation from Novartis to join their biostatistics department came and Novartis was a fantastic place to be because it was like the best ever school for discovering all different facets and aspects of pharmaceutical industry that opened quantitative sciences to me in all different aspects of decision making, which is one of my passions.

03:42
So after being in biostats, in health economics, outcomes research, market access, real-world evidence, medical affairs, an integrated approach to evidence generation attracted me to Merck Healthcare, where I’m now leading a patient-focused and real-world evidence team. So our team already integrates the classical, more classical real-world evidence with patient-focused evidence.

04:07
which is patient preferences, patients appreciation of symptoms, diseases, treatments, benefits, risks, and the creation of endpoints that a patient derived. So I think this is a little bit about my journey so far. Very good. Yeah. So you really combine all the different knowledge spaces, basically beyond clinical trials. So if you can tell in a couple of words, what actually is an integrated evidence plan.

04:35
And what is integrated about it? I can tell you one definition about it, but I can also tell you that since 10 years or more that this term exists and was invented by pharma companies to respond to the request of totality of evidence by our external stakeholders. It has been implemented in so many different ways, but in my definition, it can be a cross-functional strategic roadmap.

05:03
that coordinates all evidence generation activities across the product lifecycle from the very early development through commercialization and lifecycle management. And then you ask me about what it integrates and this will depend on the organization. So ideally it will integrate or bring together all functions that are part of an organization. It will integrate needs of all geographies.

05:32
It will integrate needs of all stakeholders to whom internal or external to whom we want to share this evidence with. And it will integrate all methods. And this is the beautiful part. now clinical trials are not anymore alone. are surrounded.

05:50
and swimming together with their friends of real-world evidence, patient-focused evidence, modeling, simulations, machine learning, AI, everything is integrated into that plan. So this is how I see it. So you mentioned kind of totality of evidence is needed by the different stakeholders. Let’s first start with the external stakeholders. What kind of external stakeholders do we talk about for this integrated evidence plan?

06:19
Our classical suspects in pharma industry, once upon a time, we were all aiming towards satisfying regulatory needs. And we said, okay, if we get market authorization, that’s it. The situation got more and more complicated and more interesting for us. And if we do have market authorization, we do not necessarily get access. So we need to satisfy needs of payers and HTA bodies to give us reimbursement and to give access.

06:48
Nowadays, it becomes even more complex. If we really want to do what we are all here for in pharma industry and to reach the patient itself, somebody needs to prescribe that medicine. So we need to reach the hearts and minds of our prescribers, of our physicians. They need to understand for which patient, why and when do they need to prescribe. And then lastly, and more presently now, we need to reach our patients with a clear narrative.

07:17
Why is that particular medication or combination of treatments or whatever solution actually fit for purpose for them? And so these are all of our different stakeholders that we need to prepare our evidence and our story for. Yeah. Let’s start a little bit about the life cycle, how that actually develops. When would you consider be a good start for something like an integrated evidence planning? So for me, this starts at research.

07:47
And this is why I’m actually, yeah, already at preclinical and many companies may disagree with me at this stage. I’m proud to be a part of the Merck healthcare that took this strategically per faith and said, how would the world be if we start integrating at the very beginning? At that very beginning, we are distantly thinking of our external stakeholders later on, and we’re not outlining.

08:13
all of the plans in the exact sequence of execution. We are thinking of these questions surrounding us. Is there even an unmet need? What is the burden of disease? What problem do we try to solve? Is that specific mode of action actually the most potent for what we are trying to achieve? Is that disease area known to us to the level of endpoints that we need to measure? How do we measure even the future treatment success or the burden of that disease?

08:43
So all of these questions need some kind of integration of different experts and different domains to, you want to design the best possible next experiment, which might be, can be a clinical trial or it can be of a different nature. Yeah. And of course you need to filter and select, yeah. Which ones do you kill? Which ones do you move forward? And if you see like, well, that might look.

09:07
noise in the lab, but there’s no way we can translate into the clinic or there’s a high risk for certain side effects or whatever. Yeah, indeed. So this integration really goes beyond evidence itself. It serves as a basis of a cross-functional discussion, serving this global product team that many pharma companies would have. And this is like a mini enterprise.

09:33
guided by the global program head or life cycle lead, depending on the company who is trying to see, do we have a molecule? Do we have a disease that we can really bring to patients faster and to more patients? Yeah. So I think integrated evidence plan is really serving that purpose, serving patients in the end. Yeah. I think it provides clarity in terms of what needs to be done for everybody that is involved and also upper management.

10:02
It’s not just sitting in someone’s head. That things like, okay, I know all these kinds of different things and I make the decisions, but if I’m sick or if I leave, then the documentation is gone, so to say. Yeah. No, no. And in the ideal world, if it will give us all like a whiteboard.

10:20
on which we draw and we put together all of the ideas and draw the path forward. And we all see the same picture and we can all comment on it, make it better, change left or right. It really gives us a guidance and it’s a living document. Integrated evidence plan is a living document. Every time something that could influence our thinking and decision-making happens external of the company or internally, every readout.

10:47
Every launch or readout of somebody else’s phase one, two, three influences our thinking and brings us back to that board asking us, do you need to change course or are you good? In a sense, it’s also decision log. Yeah. So later on you can go back and say, well, we decided to cancel that because of X. Yeah. I think that is very often not actually that easy.

11:15
trace back. If you, for example, need to make a decision again, likely if you’re working the same indication over a longer period of time, you will find similar challenges. Then you can look back, okay, why did we make that decision? Why did we move forward? What is now a different decision? Have we learned? That’s really, really interesting. Oh, it is. Yeah. I love the concept. When we now move forward, let’s say into early phase, what kind of

11:44
Typical evidence questions arise now and how far would we look forward, look out, so to say? I think once we find this good marriage between the compound and the indication, then questions are, how am I going to design my first experiment? Who are the patients that need to go in? How do I measure success? Is the success measured in short time connected to…

12:11
Can it be projected to the future success or even how this molecule will behave in the real world in clinical practice? So I think this is then the questions that we could imagine around that period of time. So the central point becomes your first experiment and everything that can help that experiment be successful. And this is where the real integration of different types of evidence happens.

12:40
about types of evidence. Let’s go a little bit through them and you already mentioned a lot of them maybe I can summarize. Of course we have the clinical trials, yeah, and also not just the ones that we do for efficacy and safety but also the ones that we do for PKPDs, these kind of areas. And then of course we have real evidence, not of course from the experimental drug but from maybe other drugs.

13:07
Maybe something to show the burden of disease to better understand how patients are treated, what standard of care we have, how long they are treated, comorbidities, all these kinds of different things. Then we have probably systematic literature reviews, where we can look into kind of what was already out there and every kind of meta-analysis, network meta-analysis in their comparisons that we can do based on that. And you mentioned more patient.

13:37
research like qualitative studies with patients where we have a focus group, things like that, but also patient preference studies. What other kind of data in broadest way, I’m not just thinking in terms of numbers, can we incorporate into our evidence planning? I think there is methods, is designs and there are also data types and sometimes we mix both, but digital data, data from variables could also

14:05
bring us additional information, data from images, omics data, data about specific biomarkers, but now also LLMs are giving us access to some other type of data that are providing input into the overall knowledge and questions that we are asking ourselves. Data around outcomes, so patient reported outcomes or clinician reported outcomes that are beyond laboratory.

14:32
values or any kind of vital signs. So basically anything that describes a patient, but I would even go and say the system through which patient needs to go through in order to have the most positive outcome is part of that ecosystem of measurements that we are doing on patients. To be fair, we probably want to mention all of these animal data.

14:56
and everything that is coming from preclinical. So everything related to safety, toxicity, initial signals of effect, even if they are on types of biomarkers that we’re not going to use in clinic. I think all of that is part of the story and part of our understanding. Would you also think about aspects that are more for the manufacturing pieces or would you say that’s a completely different story? It depends how far we want to integrate.

15:27
But it is an important element as well as operational and feasibility aspects are in on our paths to success. But for the moment, I’m more thinking of patient level data or animal data or system data that are telling us how to achieve that patient, the right patient at the right time receives the right medicine. So at the moment I’m not thinking of operational and manufacturing aspects, but they are

15:56
really important element of success. about advisory boards? So it’s a payer advisory board or QP leader advisory boards or patient focus groups, these kinds of things. How do they inform the integrated evidence plan? They’re important advisors and important data points that are guiding the path. But if you allow me being a quantitative scientist, and if I would have access to

16:25
data like European health data space or all integrated real world data in the world. I would somehow more trust this huge database that is filling me that number of patients waits for that number of time before being diagnosed and then receiving the treatment. Then I would put the weight on the opinion of five fantastic key opinion leaders around the table because they probably do not have access to that vastness of data.

16:55
So I think it is complimentary, but in a pyramid of evidence, I would give it a little bit less weight. And I can give you actually an example, if you don’t mind. Once upon a time in a neuromuscular space, a rare disease, and that disease was considered not shortening patients’ lives. Because in an ad board of five out of 13 world specialists in the domain, people saw a patient aged 70 or

17:24
73. So the common opinion was, yes, there are patients who reach a certain age, so there is no shortening of patient’s life. When we actually got into their databases, we could derive the information that it shortens life. It is not because we saw few patients who ever reached a certain age that the whole disease is not impacting the lifespan. So this perception

17:52
And this is a rare disease. So in their lifetime of treating patients, maybe they saw four patients. So this is small sample asymptotics based on anecdotal relationships with these patients. Yeah. And then you have the survival bias. Absolutely. Absolutely. Yeah. I love that point. I think in terms of these advisory boards, I completely agree they are complimentary to these other datasets. They can inform your research.

18:21
where you want to look for things like this. And maybe sometimes they are faster to get some idea and cheaper than running a landscaping, real evidence practice and then contacting different data sources, doing some kind of deep dive until you get actually to the real study, which might take you a year and lots of money. But that’s a very important point. But they can also mislead you. Yeah, it can definitely mislead you.

18:49
So I’m pretty sure people are clear on what’s now happening in terms of when you get into phase three and launch, you have all the needs from a regulatory aspect and JCA is a big topic, but also local HTA topics. Now, one of the biggest challenges that I see in that space is that you have this explosion in terms of stakeholders. Yeah. When you are in your

19:19
research, early team, yeah, you can probably name all the different people that are working on this. You can easily fit them into a usual office space. But as soon as it goes into this launch moment, the number of people explodes. You have not just the global team, which has also significantly become bigger, but you have all these local teams and functions like

19:49
marketing, medical affairs, sales, HTA, market access become much, much bigger. So how does the integrated evidence plan help there, as you mentioned, roadmap and structure to make sure everybody is on the same page? I think this is where it becomes critical because it is the only anchor that all of these functions and people who are

20:19
running around doing all the right things can come back to and check where we are, can bring information to, and then the more core people representing all of these internal stakeholders can decide on the next next turn or the next direction. So this is where it really plays an important role. But when you mention all of these internal stakeholders, their involvement needs to happen much earlier.

20:47
before they start actually doing this planning is really key. And this alignment is key, especially if you think of evidence of different kinds that cannot be found already in secondary use of data or in the existing clinical trials. If something is missing and needs to be created prospectively, that needs to be planned upfront. If you need a differentiating end point, a PRO, CRO, or any kind of end point, that would needs to be validated.

21:17
that one needs to be accepted by all of the external bodies. And this is a process of several years. So this is why starting early with right core representatives of all of our internal functions is really key. And when these endpoints need to be collected, for example, I just saw an outcome from a German HTA review and they had a comparative study. So that was great. They had the right endpoints. That was also good.

21:47
population was also good. But this comparative study reached only up to week 16. And what the German HGA body requires for chromic disease is at least 24 weeks. So they missed these eight weeks. And there someone either didn’t ask or didn’t listen and didn’t know what was the impact of that decision. And that has an impact

22:16
not just Germany, but all of Europe. And it’s a disaster. reminds me how important a simple definition of an endpoint is and our understanding what endpoint is, what estimate is, what ingredients go into that and how all components of these ingredients needs to be clarified and aligned with our internal stakeholders, who are our point of contact if you

22:43
working from industry to our external stakeholders. So they are bringing that knowledge back to the organization. And then we make that decision. So choice of an endpoint as sounds simple, but it isn’t at all. Or choice of a design. Like, do you know how many years patients in Western countries are sometimes waiting for access because we ran a single arm trial that is accepted by regulators. It’s up to five years.

23:12
Five years of time, five years of no access to treatment, just because we did not align with the design with HTA bodies at the same time we did it with our regulators. So this is something that integrated evidence planning is aiming to fill these gaps in understanding. Things that also speaks to the preciseness that you need in this evidence planning. once worked on a launch.

23:41
of a new drug and it was a pediatric indication and the people, the QP media said, oh, we need functioning data. Then we got to her, yeah, we need functioning data. And I asked, what does that even mean? Yeah, I’m with you. And yeah, in comparison to what, at what time point, what do you want to show with this? Yeah. But it’s so easy and big organizations that these kind of buzzwords come up.

24:09
or these broad needs. I once said, we need more real world evidence data. And I was thinking like, what? Yep. I’m so with you. And I think at Merck, we really have a fantastic team of these clinical measurement scientists and biostatisticians are part of it as well as our team, as well as quantitative pharmacology, precision medicine, our biomarker specialist, our data science specialist.

24:39
I think these kind of teams, especially when they’re integrated together, are really guardians of precision and guardians of scientific rigor and methodological excellence. So we are those people who will ask these questions. What do you actually mean? What is actually the question we are trying to solve and what ingredients need to be there so we can translate it into a design.

25:06
For all quantitative scientists, we should be proud of our knowledge, but we also should be courageous to ask these questions to our colleagues because maybe they do not live in the same world and maybe we are then helping them articulate exactly what needs to be done in order for us to be heard by external stakeholders. Sometimes I ask people, what is the claim or the conclusion that they want to have?

25:33
Yeah. After all this exercise, we create some kind of marketing leaflet. Yeah. What is the end goal? What is the story? Yes. What is the story and why is this different than anything else? I mean, we come back to something we discussed before is if you need to, if you need to explain it to your grandma, why this medication is better than something else. This is what we are aiming for because if.

25:59
the grandma or five-year-old nephew or son understands the story, then everybody will. That leads very good to the next and last phase is probably the post-launch phase. You know, when a grow and get more patients to benefit from the drug. How does the evidence plan there change over time? It evolves. It starts including collecting data. mean, delivering data that are needed for

26:28
post-marketing commitments, but evolves from only delivering what is needed to creating more of a story. Because our clinical trials are just beginning of collection of the data about patients on a specific drug. The real access and the real numbers of patients will happen in real world in clinical practice, where patients outside of inclusion exclusion criteria will be exposed where these

26:58
these differences between patients can actually influence the outcome or where the clinical practice, adherence, persistence, tolerability, switching, previous treatments, anything can actually influence the behavior of that drug and impact on patient. So this is what we are then trying to untangle on top of the classical safety.

27:19
We cannot tell much about safety, especially for rare events or rare important events, serious adverse events with small collections and clinical trials. It’s really interesting that when the rubber hits the road, so to say, you get into real patients, what kind of all different topics come up. Yeah. Or maybe topics get a very different weight as to what you have thought before. Indeed.

27:48
I once worked on a drug that had side effect in terms of that it created pain at the injection site. And from a regulatory point, all the scientists says that’s a non-issue for this disease area. But when it actually came to the market, the patients did care. So there was a demand on research that shows how you can manage that, how you inject.

28:17
What kind of temperature you use, where you inject, how you can decrease the pain. And so that was completely, I wouldn’t say ignored, but it wouldn’t sort of being something that was interesting or important. And then only when it comes to real patients and more patients, well, completely made up their mind. So this is an interesting one. I was once involved in a drug that was an infusion and it was taking

28:46
20 minutes to infuse the treatment and then 20 minutes of preparation and 20 minutes of cleaning devices. The company then developed the five minute inhalator instead, and no one cared except for patients for whom an hour twice a day became twice a day, five minutes. And this is something that you realize only from what is happening in clinical practice. We are happy to have a treatment.

29:15
But is it tailored for patient is something that we’re learning in clinical practice. And also we may see some benefits of certain treatments that we did not expect or combination of treatments that we are learning once the drug is on market. And this is where the magic of label expansions is happening as well, where we see these additional benefits and not only additional side effects. Let’s come to the last point of our discussion today. What can go wrong?

29:44
integrated evidence planning. So what are your top mistakes that happen in integrated evidence planning? There are many. From a strategy, I would say starting too late. Oh, we’re missing an element and that causes us additional delays. Operationally is really being rooted into our silo’s way of working within function. So

30:12
preference to functional excellence over the needs of the molecule and patient that will receive it. If we are talking today to biostatisticians and other quantitative scientists, it doesn’t matter what your title is. It doesn’t matter if you’re sitting in one group or the other. Work together. Ask yourself difficult questions. So this siloed way of thinking, my method versus your method is something we all need to overcome. Oh, yes.

30:42
So that is when communication, communication, communication, internally and externally. You mentioned one of these lost in translation. We need functional evidence on functional aspects and no one knows what these are. So I would say start early to avoid these mistakes. Talk to each other. Don’t fall in love with your own methodology and with your own function. You’re sitting there because it is important.

31:10
important to be excellent in what you do, but your impact is really rooted in collaboration and communication with others and understanding what the problem is and the so what, why we do all this. And this is for the patient. And what we want to do is to really remove all the hurdles on our way together. would say that would be my three topics amongst others. love the rule of three. Yeah. And if we go into more, that would probably be.

31:39
another podcast episode. Absolutely. Lots of topics here. FOM is a fascinating thing, space anyway. Yeah, and it’s evolving. So thanks so much, Gerana, for this great discussion where we talk about all kinds of different things. Evidence planning, where it should start, who should be involved, what kind of data pieces we have, quantitative, qualitative, how the whole thing evolves over time from preclinical research to

32:08
post-launch area, who all gets involved, talked really, really a lot about this and how people understand this is a really important thing and shouldn’t be underestimated in terms of importance and all the people that are listening here should get involved. If you have one takeaway message from today for the listener, what would that be? Work together and communicate. I think this is my, yeah.

32:37
Collaborate. If you want to go fast, go alone. If you want to go far, go together and be proud of your methodological expertise. Engage, talk about it, ask questions. Everybody needs you for success, for joint success. Thanks so much. You’re welcome. And thank you for having me.

33:01
This show was created in association with PSI. Thanks to Rain and her team at VVS for help with the show in the background and thank you for listening. Reach your potential, leave great signs and serve the patients. Just be an effective statistician.