Taming AI for Biostatistics: Darko Medin on Bio AI Works & Reliable AI Models

Taming AI for Biostatistics: Darko Medin on Bio AI Works & Reliable AI Models

Alexander: [00:00:00] Welcome to a new episode of The Effective Statistician, and today I have someone that is actually quite well known on LinkedIn. How are you doing?

Darko: Thank you for a fantastic introduction, Alexander, [00:00:15] and I’m doing great. Really happy to be the guest, your podcast, and really looking forward to discussing some interesting topics with you.

Alexander: Yeah. What have brought you to use the space where you are now? Maybe you can give a short [00:00:30] overview of your cv.

Darko: Of course. I’m someone who is working as a biostatistician for various companies around the world, but I’m also someone who enjoys building digital products, especially in a field of ai. [00:00:45] So that’s related to one of the topics that we will be discussing today, which is a specific platform called Bio AI Works.

I would say that the time we live in and the developments that are happening in ai, [00:01:00] everything is moving so quickly. So I would try to speak about different items which may be related to specific solutions for specific problems that exist today within different research domains. And [00:01:15] again, I would like to thank you for the chance to discuss this.

Alexander: Yeah, thanks so much. Before we dive into the content, I can highly recommend you to follow DA on LinkedIn. You can join lots of others that [00:01:30] are following him for his continuous other sharing and of first we’ll put the link to daco LinkedIn as well as to the tool we wanna talk about into the show notes.

So yeah, you mentioned already your [00:01:45] platform, bio AI works. Can you tell us a little bit about the problems that it solves?

Darko: Yeah. Bio AI Works is a platform that I started building, I think over two years [00:02:00] ago, and initially it was envisioned to create a connection between biostatistics and artificial intelligence.

The connection between biostatistics and artificial intelligence involves a lot of biological fields as [00:02:15] well. So that was my initial idea for Bio AI works. However, as the AI development and evolution started to become faster with much more dynamics, many new ideas came to my [00:02:30] mind. And right now the focus is on specific problems that AI models, which are, by the way, they’re fantastic.

So everything that we are seeing with. Large language models such as Pat, GPT, open AI models, other models [00:02:45] such as Lama Grok. It’s really a field where a lot of fantastic developments are happening. However, a large language models have a specific set of problems that are related to hallucinations and accuracy, even though large language [00:03:00] models can do a lot of fantastic, perform a lot of fantastic tasks, especially in the field of generative tasks, generative ai.

They have been struggling with that scientific, statistical accuracy [00:03:15] that us as statisticians usually want to have. The specific problem where Bio AI works is it’s a significant portion of Bio AI works, is focused today, is trying to be different in that [00:03:30] way so that AI models that are developed at Bio AI works.

We’ll have a higher level of reliability. So there will be a lot of different checks that AI models need to pass to produce a specific [00:03:45] out. So that’s one of the main problems that we are trying to solve today. When I say we for Bioworks, I’m partnering, even though I started building Bioworks a few years ago.

In development of certain AI models. I’m [00:04:00] partnering with a great company O Analytica, and we are looking to build a diverse set of AI models. So one approach I just mentioned, but there are other approaches that we are also looking to explore, such as precision [00:04:15] medicine research, precision drug discovery, using generative AI in drug discovery as well, and also AI models which would be able to solve one problem.

Which is often found in statistics and [00:04:30] data science, which is the hidden patterns that are in the data that we don’t see using the standard approaches. However we are now in the process of development of an AI model called the universal Constructor, [00:04:45] which main ability is to try to identify those difficult, to find patterns in the data.

Then produce entire data sets of hidden patterns of what we don’t see just [00:05:00] by looking into correlations, associations, and other standard statistical approaches such as linear istic, regression and so on. So we are trying to combine the foundation that we have in biostatistics and statistics [00:05:15] and the rigor that we have in those scientific fields with the capabilities.

Of artificial intelligence and try to improve artificial intelligence by making it more [00:05:30] accurate and more reliable.

Alexander: When you talk about reliability, can you expand a little bit on what that means to you?

Darko: Yes. There are actually multiple fields we are exploring when we speak about [00:05:45] reliability. So one of those fields is obviously statistics, basic domain, which is always needed in some way, the basic accuracy, the receiver, operative characteristic.

Other metrics that we have in [00:06:00] statistics. When we create outputs, we want to know that these outputs are statistically accurate and cross validated, which means that any output that is created. It’s validated in a certain way. It’s [00:06:15] not just generated as an output and produced. It needs to be validated in this process that we are trying to build at the core of AI models.

But of course, this is just one side of the story. There are other approaches that we are [00:06:30] experimenting with in improving the validity of outputs with the domain specific checks and balances partnering with. So Analytica we have. A whole team of scientists from different biological [00:06:45] spheres, including molecular biology biostatistics, statistical programming, other fields of life science.

And we are trying to create a domain specific set of rule that AI model [00:07:00] needs to follow. So that the outputs that it creates are more valid, more accurate, and less prone to that. Very frequent problem with the daily AI models, which is hallucination. So that’s one issue that we want to avoid at [00:07:15] all costs, which is hallucination of AI models.

Well,

Alexander: what’s a hallucination? So I’m just thinking about kind of psychiatry and hallucinations, but if you’re talking about hallucinations, you probably mean something [00:07:30] different. What’s in the hallucination of an AI model?

Darko: Good point. So halluc in AI models is a completely. Different item, different research field compared to psychiatric and psychological terms.

So [00:07:45] in AI models, hallucinations are very frequent. Almost every AI model built today has some form of hallucinations when specific prompts are prompted towards that AI model. This is happening [00:08:00] mainly due to the fact that most AI models, at least today, are generative. Which means that at the core of their functionality is that they’re trying to generate new pattern and new [00:08:15] sequences from text.

But still trying to make sense of those sequences. So when we ask a question to an a AI LLM model, it’ll try to answer that question as accurately as possible given all the [00:08:30] knowledge that it has acquired during the training process. At the same time, it’ll try to be creative a little bit. It’ll try to be generative and try to generate potentially patterns that don’t exist in the original data.

Now, [00:08:45] this is from one angle, okay? Because generative AI models allow a lot of flexibility. That’s the foundational principle that enables those impressive answers that a lot of AI models can produce. [00:09:00] However, it has a downside. The downside is that given all that complexity, all the training data that is included in training those AI models combined with its creativity and [00:09:15] generative capabilities, trying to create some new patterns, sometimes actually very often I would say these new patterns are randomly created.

Well,

Alexander: it could be said, don’t know. It makes up. [00:09:30] Clinical trials that doesn’t exist. Yeah. Says one arm studies that are existing and there combine these in into a clinical trial and says, compares these kind of different things. [00:09:45] Something like this,

Darko: I think you, you made a great example. So a lot of times in AI model will try to combine the information that it has and in specific situation it doesn’t have this critical.

[00:10:00] Angle, maybe sometimes there is not enough information to make a specific conclusion, but an AI model will still try to use its generative capabilities to create a specific story for that answer. And [00:10:15] sometimes that may actually be just randomness, just as you said, her, a random hallucinated clinical trial, let’s call it that way.

And that would be one case example. However, there are many other examples [00:10:30] where generative AI model, this is especially the case for large language models because they are trained very often on large amounts of textual data. Most large language models today actually ingest majority of the text that exists on [00:10:45] the internet, and that’s how they’re trained.

We can just imagine how many possible combinations can those. Models create from that text. Sometimes that text is inaccurate. Sometimes it doesn’t resemble what we are [00:11:00] trying to find. Sometimes it doesn’t have a scientific foundation. Sometimes it has scientific foundation, but large language models really struggle from differentiating this.

If we take scientists can take a critical [00:11:15] perspective. When he or she reads literature, we can say, okay. This is just a blog post that someone wrote, or we can say, oh, this is a peer reviewed clinical trial study, which is very [00:11:30] rigorous. Then I will trust more that peer reviewed study. And there are so many nuances there.

So our human intelligence naturally has built in mechanisms to critically evaluate things. And AI models are still struggling there. [00:11:45] So to be able to prevent those hallucinations. We need these mechanisms that will critically evaluate data. Or anything that’s generated, anything that’s generated that novel to differentiate what could be [00:12:00] reality and what could be a hallucination for that model.

Alexander: Okay. So one thing said, by the way I want do is less hallucinations. The other thing you mentioned [00:12:15] is reliability. You mentioned like example, like receiver operating curves. In order to create a receive operating curve, you need to have something like a gold standard. Yeah. That you compare [00:12:30] against. How do you create a gold standard?

Is that based on simulations or something like this?

Darko: Yes, so this is one of the approaches that we are exploring. So using AI models to. [00:12:45] Identify causal patterns in the data and for this specific field, we are using a lot of simulations to simulate co causal associations, mathematically simulate them and then try to transpose this into a real [00:13:00] world situation.

So that’s one of the approaches. The other approach that we are exploring is, even though we are developing a lot of advanced, they. Technologies because these are more technologies today than just saying, oh, this is a [00:13:15] model. We know from statistics and from data science that a model is not very useful unless it’s a part of a specific product that can productionalize that model and bring real value to its user.[00:13:30]