The Evolving Role of Generative AI in Pharma

Manuel Cossio

Global AI Head at Cytel | Agentic AI for Clinical Trials / RWE / Market Access / EU JCA

Manuel Cossio is a board-certified medical geneticist and an artificial intelligence engineer. With over a decade of experience in global medical and clinical affairs within biotechnology companies and big pharma, he specializes in rare diseases, oncology, and neuroscience.

Manuel’s expertise extends to healthcare data, AI, and digital health. He has led numerous global projects involving real-world data, sensors, healthcare apps, electronic health records, and algorithms, including those for computer vision in medical imaging, natural language processing, generative AI, and software as a medical device.

Beyond his professional roles, Manuel advises organizations such as ISPOR, DiME, and the EU Commission on digital health and healthcare generative AI. His mission is to seamlessly integrate artificial intelligence and digital health, driving a significant transformation in healthcare. He is dedicated to promoting a patient-centered design approach, enhancing the healthcare landscape through innovative solutions.

Transcript

[00:00:00] Alexander: You are listening to the Effective Statistician Podcast, the weekly podcast with Alexander Schacht and Benjamin Piske designed to help you reach your potential lead great science and serve patients while having a great [00:00:15] work life balance.

[00:00:22] Alexander: In addition to our premium courses on the Effective Statistician Academy, we also have. [00:00:30] Lots of free resources for you across all kind of different topics within that academy. Head over to www.theeffectivestatistician.com and find the Academy and much [00:00:45] more for you to become an effective statistician. I’m producing this podcast in association with PSI community dedicated to leading and promoting use of statistics within the health industry for the benefit of [00:01:00] patients.

[00:01:01] Alexander: Join PSI today to further develop your statistical capabilities. With access to the ever-growing video on demand content library preregistration for all PSI webinars at head over to the [00:01:15] PSI website at PSI Web to learn more about PS I activities. Become a PS I member today.

[00:01:29] Alexander: Welcome [00:01:30] to another episode of the Effective Statistician. I’m super happy to have a colleague here today because he is working all around artificial intelligence, ai, and many of us have been exposed to [00:01:45] this topic already and it’s definitely not a new topic, but it’s still a very hot topic and a very fast evolving topic. 

[00:01:53] Alexander: I certainly have used it here and there but it’s definitely something different to talk [00:02:00] with someone that is working on it all the time. Manuel, welcome to the show. 

[00:02:05] Manuel: Thank you so much , Alex, for having me. This is such a pleasure to be here. I hope I’m not the black sheep of your podcast because, everyone is a [00:02:15] statistician and I know that this is, but from, an engineering point of view, so I hope that’s near to the table.

[00:02:22] Alexander: I definitely had non statisticians as well on the show, so you’re not the first one . Introduce yourself so that [00:02:30] people understand where you’re coming from and what you’re currently working on.

[00:02:33] Manuel: Perfect. I would like to say that I, as a professional, we can say I am a hybrid.

[00:02:41] Manuel: Because I am a molecular biologist geneticist, that is my [00:02:45] core training. And I work on diagnostics in a hospital at the beginning and then in a pharmaceutical company Sanofi in the space of medical affairs for diseases. So we can say that my career started more or less [00:03:00] in the, medical science space. But then I migrated to ai. And AI now we can say the biggest part of my focus. Because the thing was that when I was working at Sanofi. We started having some very [00:03:15] small projects on ai specifically for some therapeutic areas where we needed to bring innovation to make the patient journey a little more efficient and more different from the one that we had in the past.

[00:03:28] Manuel: So in that moment, I realized [00:03:30] first that I didn’t know practically anything on ai, and that made it really hard for me to have conversations with vendors on technical discussions because ai had a really big grasp of the field. But when we [00:03:45] started discussing algorithms and accuracy and, images and all of the things that, for example, you need to build AI solutions, I was really lost.

[00:03:55] Manuel: And I was deeply passionate about the field. So I said I think [00:04:00] my path needs to take this other direction. So that’s when I moved to Barcelona to study. Barcelona has one of the nicest schools of engineering in Europe. And also Barcelona has one of the European [00:04:15] supercomputers.

[00:04:15] Manuel: So you have both of these really big spaces in AI in the same place. So that is why I moved there and I started my training in ai. It was one of the toughest things that I have done, because I didn’t [00:04:30] come from the computer science space. So for me, learning was very hard. I remember that I spent almost 15 hours a day studying and practicing [00:04:45] coding skills because I actually knew very little on how to code, and the program require a very high coding skills development. So I needed to put my hands on the dirt and do [00:05:00] everything there. It was very tough, but it was extremely rewarding because, thank God, it was the phase. Previous to the boom of ChatGPT.

[00:05:10] Manuel: So that meant that, for example, for you to have an idea, we needed to do all the [00:05:15] coding ourselves. In end, every student needed to write their own code. I don’t know if I like that approach, but we can say that it worked that they wanted us to think about the mathematical processes [00:05:30] behind functions, so they ask us it was actually a requirement that you couldn’t use functions built by package.

[00:05:39] Manuel: That means that, for example , we needed to program, the function ourselves so that they [00:05:45] saw inside of that code that we understood the mathematical process behind. So we program super vector machines. We, very rudimentary and small tool, for you to encourage you to think about what you [00:06:00] were writing. And they were really like focused for us not to copy, the work of other students. So the code needed to be like, developed by you and not a copy of other codes. So it was [00:06:15] actually very tough, but it gave me this mental exercise of thinking the solution.

[00:06:20] Manuel: Before going and coding it, like how the data is going to be in the input space, how it’s going to be transformed, how the output should look like, for [00:06:30] example what could be potential bug in the middle of that function if you have, for example, three dimensions here, and then the output has four dimensions.

[00:06:38] Manuel: So it, it was I think a very challenging to learn but it was very efficient. I think now I don’t know how [00:06:45] they are going to do with, co-generation, because you can generate a task on an exercise in seconds.

[00:06:51] Alexander: Yeah. 

[00:06:51] Manuel: Just to finish and wrap it up, I worked in several consulting companies. I also consulted for other [00:07:00] pharma especially on this journey of, starting using AI for medical affairs first and then co-generation in, for other companies. And now I am at Cytel as a partner, I would say into [00:07:15] bringing. All of these solutions that we have internally and to build a future where AI can be ally for people and to help us, in some sort of way to take all the automatic, repetitive [00:07:30] tasks that we do day by day that Could put us into the burnout space and actually allow us humans to, once we have the data and once we have perhaps the things that we need to analyze, to use [00:07:45] our, human intelligence in ways that the language models cannot at this point or in developing it. 

[00:07:53] Alexander: I completely agree.

[00:07:54] Alexander: It’s yet another way of how to get rid of some mundane tasks and [00:08:00] focus more on those tasks where we have much more opportunity to add value. Let’s speak about where do you see AI being really. released, spend at, used and broadly being [00:08:15] accepted within the pharma space.

[00:08:16] Manuel: That is a very nice question. In the sense that to start like the defining this kind of space, the best. Use cases that we have for generative AI [00:08:30] and agentic AI in these days is where you have something that you repeat day after day that involves the same data, that enters the same data that goes out, the same changes in [00:08:45] format.

[00:08:45] Manuel: That is the best place for AI to come. And help in that process. For example, we have a lot of documents that we need to build, for example, in the clinical development phase for example [00:09:00] clinical reports that you need to produce that are basically understanding data sets and understanding perhaps.

[00:09:07] Manuel: Another report or another a protocol, for example, that has the data but in a different format [00:09:15] that then you need to grab that data and transform that into a new report that are the cases for generative ai because. It’s basically the data is there. The model just needs to [00:09:30] understand the structure of the data, how it is being, used, and especially the context in which the data appears, so that then the data can be transformed into a different format.

[00:09:42] Manuel: Understanding the context, the original [00:09:45] context, so that we don’t perhaps. Use different words and change the meaning in the new document that we are generating. So when you find a repetitive task that you are doing over and over again, that is the case? 

[00:09:58] Alexander: Where do you see [00:10:00] cutting edge areas, where, aI is maybe used here and there, but it’s definitely not yet widely adopted. 

[00:10:10] Manuel: That is a very nice question. I would say in the space of [00:10:15] medical writing, we have a lot of opportunity in the sense that we are. We are slowly approaching that phase. And we have a lot of opportunities there because medical writing is one of the [00:10:30] things that is being used.

[00:10:33] Manuel: It, it takes a lot of people and it takes a lot of processes that are mostly the same. The documents, the structure is fixed. That means that you’re not [00:10:45] going to have a report that you need, for example, for regulatory purposes. Being changed like day after day is the same report, it’s the same data that needs to be filled inside.

[00:10:56] Manuel: The input is also the same. The [00:11:00] problem that I’m seeing now that of course we need to address that also is the hallucinations problem that models because of the nature of how they are trained and how they. See [00:11:15] the world, we can say it’s deterministic. That means that you have a distribution of, things that the model sees, and that is the representation of the world for that model.

[00:11:28] Manuel: When you [00:11:30] go out of that distribution, then. 

[00:11:36] Alexander: Yeah. Information except 

[00:11:38] Manuel: Yeah, exactly. We are subjected to basically randomness. That means that the [00:11:45] output could be a good one or the output could just be outside of the definition of true that we have for that answer. Because. In repetitive tasks, sometimes you [00:12:00] can more or less, narrow the deterministic part because they are, things that are being, repeated step after step.

[00:12:09] Manuel: But it could happen sometimes that the model cease something that doesn’t have [00:12:15] all the knowledge to understand. And then, the output is not, it’s not something that is valid, yeah. So hallucinations is something that we need to, first of all, define then understand, and then learn how to mitigate for [00:12:30] each one of the use cases.

[00:12:32] Manuel: And the other thing is that we also need to learn more on data protection and data governance around models, because the case is [00:12:45] that. When you work with, for example, your GBT license in your laptop and another person is working with another GBT license, but for the same company all of that information travels.

[00:12:59] Manuel: That means that you [00:13:00] upload something a query, for example, and perhaps you also upload a document. If you are going to use a retrieval document generation, you need to read that document to extract information. So once, once you’re upload that, the data travels where [00:13:15] the server is, because in that server is the model, the foundational model hosted.

[00:13:21] Manuel: So the data needs to be encrypted when it is traveling. Then when it arrives into the place where the server is, [00:13:30] it needs to be encrypted again so that the model understands the data, it is processed, and then it goes back to the user where the user is located. The problem is that, for [00:13:45] example, with the European data space, you are not allowed to take data out.

[00:13:53] Manuel: Of the European Union in, for example, a server that is in the US or in another [00:14:00] country. So that make things complicated to build models and to use perhaps some companies or solutions that do not have servers in the European Union. And also how the data is being transformed. Will data [00:14:15] remains a part of – we can say learning data for the model. We also need to know that because if a document needs to be protected, you cannot have that data being part of the training data of the model. The [00:14:30] model needs to operate and forget about the data it just received. Yeah. And all of that. Because this field is very young.

[00:14:39] Manuel: We are learning as we walk. We see a problem, we go, we [00:14:45] discuss, we adjust, and then we repeat. We iterate. That is the process more or less. 

[00:14:51] Alexander: Yeah. You mentioned already coding, what are your experiences with using generative AI for coding? How does [00:15:00] that especially work out in our regulated pharma environment?

[00:15:04] Manuel: The thing is I dunno if I’m going to get in a lot of trouble for say this, but we have two, I think visions on this. On coding with [00:15:15] generative ai, we have one vision that I think is a very conservative one in the sense that coding needs to be restricted only for AI and data scientist engineers.

[00:15:26] Manuel: So you come as a normal human being [00:15:30] of the world with an idea, then you transmit that idea to them and then they build the code. And we have the other vision that I am a little more inclined into that vision that actually generative AI is making coding [00:15:45] available for people that do not have coding backgrounds.

[00:15:47] Manuel: That means, for example, I would like to develop a webpage that has a lot of coding, in the backend. So I just write what I want. How I want the webpage to [00:16:00] look like, and then I will have that code being generated. First of all, it helps a lot of people that want to go into coding to just do small steps and to perhaps when [00:16:15] you have a back in the code that you are, trying to learn and to produce, to have a model explained to you why you are having that problem.

[00:16:24] Manuel: That was something that, for example, I didn’t have when I was learning to code, and that meant that [00:16:30] sometimes for some functions that were, for example, you were working in a pipeline, right? And then for some reason the environment updated the versions of some packages. For example, [00:16:45] you had a small change in the way that the input was, received, or for example, you have this kind of, okay, in this first part of the function, it goes the dataset. [00:17:00] Here it goes, which part of the dataset you want to analyze. And perhaps because of design, they just switched to this, so they would have first to choose which part of the data set you will need to use, and then the data set. [00:17:15] That thing, from one moment the pipeline was working from the next day you just, open your computer. You just said, okay, I’m going to continue working with this code. You just hit it, run. I give you a back bear, and you [00:17:30] spend years. We can take weeks, reading things on blog seeing other, things related to that code until you see someone said, remember that when, for example, I don’t know, we’ll call up, [00:17:45] change this new version of the environment. They change this package for this one. And the positions are inverted.

[00:17:52] Manuel: Once you realize that perhaps you wasted two weeks, now you just put that function [00:18:00] into the chat GPT, for example, or clo and you can say why this is not working. And then it’s going to say, because this version of this function has deposition in. Yeah, you just go and change that. No, [00:18:15] even you can ask GPT to, can you please give me the correct version of this code and it’s going to rewrite the code with the new, fixed function.

[00:18:26] Manuel: So then your problems go away. So in my case, I [00:18:30] think that is, you wasted a lot of time in something that was so mundane of just a change in a position. And I think generative ai, there is a very helpful ally. Or for example, when you have the [00:18:45] idea in the sense, you are speaking with someone that wants to develop an algorithm for, image screening for ABCs and they are telling you? Yes. Okay. So we have four line images that we need to [00:19:00] identify. So this image, we have the data sets with the labels. And we would like perhaps a neural level because that is some algorithm that works super nice with images.

[00:19:11] Manuel: So you know in your mind, okay, the images are [00:19:15] going to come here, the director is here, the data set with the labels is here, so I need to combine this and this. So if you need to sit and write it the old way, perhaps you would waste, I don’t know, finding a repository that has a [00:19:30] pipeline similar to the one that you want to develop, then write it on your code processor.

[00:19:35] Manuel: Then you know, testing. But now you can say, chat GPT, hi, here is my problem that I need to solve. I need to classify these four categories [00:19:45] with a neural lab board. It could be this one or this one. Here is the format of my data set with the labels. And here is the directories.

[00:19:53] Manuel: Can you build me a pipeline for this? So then you have your entire pipeline in [00:20:00] seconds, and then you put that into your code processor. You hit run. It works. Then of course you made slight changes. Perhaps when you see the accuracy that is not working the way [00:20:15] that you want. You could add augmentation techniques or things to improve that.

[00:20:19] Manuel: But you just had, we can say a 65, 70% correct. Pipeline in five seconds. Yeah. [00:20:30] Yeah. So there is the the nicest things about this, you want, for example, to write or to do research right in a specific area. You now can use, for example, open [00:20:45] AI to do what is called a deep research. So you can say Chat GPT, can you please build me a landscape of these and these subjects. What is done and where are the unmet needs, for example, and then it is going to go and bring [00:21:00] papers, blocks. Any piece of information with a reference and tell you from these subjects that you want to know here, and here are just two publications and they are not addressing this and this [00:21:15] problem there, you have your way to go for research, for example.

[00:21:19] Alexander: That is super helpful. For example, when you, write about the introduction of a paper and these kind of areas. And I love that it comes now with the [00:21:30] references. That was a big limitation in the past that you didn’t know where everything was coming from.

[00:21:36] Alexander: And so it was really poor from a from a transparency perspective. And of course. You couldn’t really use it [00:21:45] for cause for research. And now it’s much easier to identify what is actually fake and or, made up by the model what is actually real data, real publications.[00:22:00] 

[00:22:00] Alexander: I also think that when it’s about coding, I know that many struggle. With coding for figures, for nice figures. You can spend a lot of time fine tuning figures. And I think you do actually need to spend some [00:22:15] time fine tuning figures. But of course, with AI it can be much faster

[00:22:20] Alexander: to explore a couple of different ways how to approach the figure and you can get suggestions on improving figures and so on. So that [00:22:30] hopefully leads to much more use of data visualizations in the future. 

[00:22:36] Manuel: Exactly. Actually Alex, you hit one of the hot topics also for ai. That is this kind of data quality control [00:22:45] and data standardization.

[00:22:47] Manuel: When you have, for example, in some cases that we have this kind of migration that is happening from SAS towards R. So yeah, it’s going to take [00:23:00] a while because SAS is a language that has a lot of validation behind and a lot of years of use. So we practically know anything that can happen inside SaaS and R is just the opposite, is something that is starting [00:23:15] to be used.

[00:23:16] Manuel: The thing is. Some things, as you probably know in clinical trials are done in SaaS and some things are done in R, right? Yeah. And there is the moment of truth when you are building the [00:23:30] reports and building, the documents that you see, perhaps a table that comes from SaaS and then a second table or a figure that comes from R.

[00:23:42] Manuel: They have different data inside and is the [00:23:45] same study, is the same data set. Yeah. But they are different. And the amount of human hours that it takes to go line by line and say, here it should be 15, why is 13? [00:24:00] Like where is this number coming from? And then you go and see and perhaps. The, when the function was written in R, there was a skip in the line.

[00:24:08] Manuel: So you’re not seeing the same role. You’re seeing a role behind. Yeah. Those things are the use [00:24:15] cases for ai also in the sense that you say, here is this other figure table. Can you read both and tell me where the mistakes are? the language model is going to say here line, I don’t know, two in this table should [00:24:30] correspond with line two also in this table.

[00:24:32] Manuel: But in this table is line three, so you need to mine, the problem. So instead of taking, I don’t know, perhaps sometimes 15 hours and this is something that is also very interesting, that as humans. The [00:24:45] more tired that we get when we are reading the same thing, the harder it is for us to spot problems.

[00:24:52] Manuel: Oh, yeah. And ai, it doesn’t get tired. Yeah. So it just go over and if you say, can you [00:25:00] please do the same analysis, iterate 10 times, it iterates 10 times. So then you see if there are some different answers. You can say, okay, from, I don’t know, 10 attempts, eight, give the same, response.

[00:25:12] Manuel: It could probably be that the right [00:25:15] thing, but imagine a human, can you revise this single 10 times like. Yeah. That’s the main difference. So that is another case. And also building, for example, very mundane things that are, that take a lot of time from people, very [00:25:30] highly trained people with PhDs and with, I don’t know, 20 years of experience, writing the footnotes or writing captions for figures or for tables.

[00:25:40] Manuel: That is the case for ai. 

[00:25:43] Alexander: Yeah. Completely agree. [00:25:45] Thanks so much for that great discussion. We touched about various aspects within biometrics with, medical writing, statistics, coding heavily talk about medical writing and coding and how that can help [00:26:00] us to become more effective.

[00:26:02] Alexander: What are the different use cases there? Now you also talked about limitations that we need to take, have into account where we need to be a little bit careful in terms of uploading data, for [00:26:15] example. There is lots of opportunities. But we need to also be aware about our space that we are working in and the specific constraints that come with this.

[00:26:26] Alexander: Now if a listener says, [00:26:30] ah, my company would really benefit from all of that, and, we are not a huge company with, lot, lots of AI specialists already. How can that company benefit from you and your colleagues services? 

[00:26:44] Manuel: [00:26:45] The first thing that, that I would say to them is that of course they, they should come to to me to ask Cytel with any kind of questions that they would like to know.

[00:26:55] Manuel: But the thing that I would say is. That [00:27:00] they just imagine what they would like in the sense because, so sometimes when these. Fields of ai feeds a lot from, abstraction and imagination in [00:27:15] the sense that when you know the process, you are the expert in the sense that I am an expert in ai, but I’m not an expert in the process.

[00:27:24] Manuel: That means that each company, each place has their own [00:27:30] processes, for example. Data governance processes, people that need to approve some data flows to move to the next stage. So each company is a world inside. And the more you [00:27:45] know and the more you tell us about how the process goes, the better for us to say how we can pull AI into that process in a way that is organic.

[00:27:57] Manuel: That it respects the data [00:28:00] governance processes of the companies. I have seen a lot of very nice solutions in the sense, very well thought and with a lot of justification behind as use cases to fail [00:28:15] because the processes inside the company that these solutions was going to be applied on.

[00:28:22] Manuel: Weren’t totally known and were, took for granted in the sense, oh my God, it’s going to be [00:28:30] just the same as this other company. No. Big mistake. We need to know, and we need to perhaps meet a couple of times where you tell us yeah, we have this server. This server is located here. The data [00:28:45] from these trials come.

[00:28:47] Manuel: Into this way here, and then the data needs to be approved by this and these and these people so that it is available for us. All of that makes us to say, okay, so [00:29:00] perhaps the language model needs to be hosted here. It needs to have access to this and this data. We need a person to approve this process so that we also protect.

[00:29:11] Manuel: Very carefully the data of the company [00:29:15] so that you also have patients trusting that their data is protected. Because one thing that is really damaging for AI is when the data is not [00:29:30] protected. Because a solution could be amazing in the sense of, predicting what you need to be predicted.

[00:29:36] Manuel: For example, a disease risk or I don’t know, a time to an event that you want to prevent in a patient. That is amazing. But [00:29:45] what happens if, you know that algorithm is leaking data Yeah. To the outside, and a patient then says, no I’m not going to be using this algorithm because I don’t trust that my data is going to be [00:30:00] secure There.

[00:30:01] Manuel: Yeah, that is a thing that we cannot afford because patients need to trust that the algorithm is going to take the best possible protection to their data, and also that the output is going to be [00:30:15] something useful for them so that they can use that and we can improve health systems with this. Everything needs to be needs to be considered. And the other, the last thing I’m going to say about this is that [00:30:30] for me especially this feel is also filled with magic a little. So you need to bring your happiness here in the sense, what would you dream of [00:30:45] having, with ai, what would you like this to look like at the end?

[00:30:49] Manuel: For example, I would like. This document to be completely automatic though, with those things we can work in the sense, okay, [00:31:00] let’s see how the document is being produced. Perhaps we cannot produce it completely in the first iteration of the process, but we can say, okay, let’s do the synopsis. So once we validate that, we continue with the rest.[00:31:15] 

[00:31:15] Manuel: And in no time you have. The prototype, they are ready to be used. But bring the thing that you want, bring the thing that makes you happy. Like the sky is the limit. Yeah. With that kind of [00:31:30] approach, we can speak, we can iterate, we can, if we don’t have the answers for this just now. We can have it for the future, but we don’t.

[00:31:40] Manuel: We need to come with our hopes because that is the only thing that is going [00:31:45] to give us the strength to push, forward when we want something to be built and change the life of people. So that is basically what I would say to anyone that wants [00:32:00] to do ai. Awesome, 

[00:32:02] Alexander: awesome. And we will link to Manuel LinkedIn’s profile and his email address on CEO homepage.

[00:32:11] Alexander: So just check out the effective statistician.com [00:32:15] and then the podcast. You can find all, see episodes if you scroll a little bit or, and in search for man across you. Thanks so much. For that. Thank you so much, Alex. Out ai. I have the feelings. That was [00:32:30] potentially not the last one. So let’s see.

[00:32:36] Alexander: This show was created in association with PI. Thanks to Rain and her team at UVS. Well with assurance’s background and [00:32:45] seek you for listening. Reach your potential lead great science and serve patients. Just be an effective [00:33:00] statistician.