R-Tool For Bayesian Statistics in the Early Phase

R-Tool For Bayesian Statistics in Early Phase

[00:00:00] Alexander: Welcome to another episode of the Effective Statistician. Today I’m talking with Audrey from Roche. How are you doing, Audrey? 

[00:00:10] Audrey: I’m doing really well. Thank you. How are you, Alexander? 

[00:00:13] Alexander: Very good. We are recording this right after the PSI conference, which we both attended. [00:00:20] Awesome event. How did you feel about that conference?

[00:00:24] Audrey: Very smoothly run. I think the topics were so interesting, and I think it progressed a lot of conversations we would not have had because it would just be offline or online if you like so productive as well. I’m very impressed. [00:00:40] 

[00:00:40] Alexander: Okay, cool. So, before we dive a little bit deeper into the topic of today, maybe people who don’t know you, you can introduce yourself.

[00:00:50] Audrey: Oh, absolutely. I’m Audrey Yeo. I’ve got three years experience in pharma. I am also a statistical software engineer. I’ve made a [00:01:00] package which we’ll talk about today as a background. I’m a registered nurse from Australia. I’ve worked in oncology, adult care pediatrics, and four years ago between my master’s and starting at Roche, I worked in the COVID ICU in my old hospital because they let me.

[00:01:18] And I’ve, I’ve had a [00:01:20] ball so far both in my nursing career and my pharma career as a statistician and a statistical software engineer. 

[00:01:29] Alexander: Wow. Having worked in a COVID ICU, that must be no. interesting experience. 

[00:01:36] Audrey: I think I can admire the comradery [00:01:40] and the staff there. I pick up a lot of good skills and good experience of what great teams look like.

[00:01:49] So I’m looking forward to it. Yeah. 

[00:01:55] Alexander: Great teams make a huge difference and pretty much [00:02:00] all that we do is teamwork. And I just talked recently to some people in my leadership program about teamwork and what creates great teamwork. And one of the common denominators has always said. It’s very satisfying, very rewarding and often fun [00:02:20] to work in great teams.

[00:02:22] Audrey: Absolutely. I can attest to that. 

[00:02:24] Alexander: Yeah. So let’s talk a little bit more about your tool and how that came about. So it is for better decision making in early development. So. [00:02:40] What are the biggest challenges that we face in terms of decision making in early development? 

[00:02:46] Audrey: Yeah, that’s a really great question.

[00:02:49] This tool uses a Bayesian framework and it looks only at binary endpoints. What’s really great about the Bayesian framework is that it mimics a clinical [00:03:00] decision process. By using prior data. So as a clinician myself, I can attest to, you know, updating myself on the clinical decisions I make.

[00:03:09] And that’s what I like about this Brazian approach. 

[00:03:12] Alexander: So what kind of decisions. Are we talking about here? 

[00:03:18] Audrey: Yeah, that’s a good [00:03:20] question. So we, I think it’s very specific to early development statisticians where we are monitoring the data continuously. And at any point, the program or the molecule can be terminated, because it’s doing so well and it can go straight to phase three expanded cohorts or it should be [00:03:40] terminated early because of utility reasons.

[00:03:42] Thank you. 

[00:03:43] Alexander: Is that only for the oncology area or is that for other areas as well? 

[00:03:48] Audrey: Yeah. So this, this package looks at binary endpoints and we will use, we use response rates, but we also use the beta prior. [00:04:00] And if that suits any other therapeutic area the Phase 1B package, that’s what it’s called, is not restricted to oncology examples.

[00:04:09] I do not have any other examples for other therapeutic areas. And since there are so many oncology drugs on trials, on investigation in the early [00:04:20] phase, It’s not just at the Russian Genentech organization. I feel this could be very useful to other organizations as well. And it is open source, so we’re very happy to share, share that.

[00:04:32] Yeah, that 

[00:04:33] Alexander: is actually one of the great things about Roche. That’s a really committed to developing [00:04:40] everything open source. So people can benefit from it. Outside of Roche as well. Kasper Rufibach just talked about this at the PSI conference as well, and mentioned that this offers a lot of opportunities for the community.

[00:04:59] But also [00:05:00] for Roche itself. So that’s really, really great. Now let’s talk about the properties that we need for better decision making. What, what do we care about within this area? 

[00:05:14] Audrey: Definitely. Good question. I also want to acknowledge that this package [00:05:20] has been used by the Russian Genentech organization for the last 10 years, and it has changed a bit due to the questions that was raised internally by two large organizations.

[00:05:32] And it was led primarily by Daniel Sabin and Bovet. And currently, he is my [00:05:40] engineering lead. I started last year to refactor, rename, and add unit and integration testing. The last two are features of today’s state of the art software engineering practice, which means not just, you know, It’s not, it’s not only state of art, it’s also open source and, and it can always get better. And [00:06:00] that’s what I want to talk to your point about open source, if I may. 

[00:06:03] Alexander: Yeah, sure. 

[00:06:04] Audrey: It attracts, well, I, I’m hoping it attracts a diverse audience. To contribute the issues list is is open. People can add issues of Oh, I wish this feature was there. Please make that [00:06:20] and also collaborate. So I hope that with this openness to iteration that it will only get better, including including other endpoints.

[00:06:30] And so how, how can we make better decisions? Or what does this tool package this tool offers? I, I like that it’s at your fingertips of the [00:06:40] study statistician. who in the early oncology study, the dynamic is very fast. Things can change very fast. Molecules can be terminated just the next day. So there’s a lot of efficiency gain to be had.

[00:06:54] And what I try to do there is make documentation as accessible as [00:07:00] possible. We know statisticians are diverse. We are working in a neurodiverse atmosphere. So I try to make the documentation Readable and come from the user perspective. And I also have peers who helped me do that. For example, I discussed design [00:07:20] questions with my engineering lead and also from peers to understand if they would see it the same way.

[00:07:29] And how do we make this too? Sorry, you want to go ahead. 

[00:07:31] Alexander: No, no, go ahead. 

[00:07:33] Audrey: One more thing I want to add to that is how do we make this a better decision making tool is that [00:07:40] I might be the lead developer of this tool, but I don’t have all the clinical trial experiences in early development that others have and can add value to this package by saying, well, it doesn’t speak to me like this. That’s what I wanted to say. 

[00:07:55] Alexander: Yeah, I think. Developing these tools always [00:08:00] connecting with a user is really, really, really important to make sure you have a great user experience. What, what do you do to, or what actually is a great user experience? 

[00:08:17] Audrey: Yeah, wow. I’m so happy to get [00:08:20] this question as a statistician slash, you know, I’m making statistical software, but what makes it a good user experience?

[00:08:28] I think, like you said, feeling like the creators had known what you were looking for, being very connected to the user. And I think that is a [00:08:40] relationship over time. For example, engaging the user in small seminars having, having a possibility to To interact with the users through, I guess, now with it being open source, getting feedback the user, our conference in Salzburg is going to help with that then [00:09:00] understanding the product and us understanding their problem.

[00:09:05] So I’m a clinical early clinical trial statistician myself. So I hope I can lend that experience to, to the future user. 

[00:09:14] Alexander: You mentioned you can submit issues to basically make [00:09:20] suggestions, improvements, things like that. How does that exactly work? 

[00:09:25] Audrey: I think anyone with a GitHub account can log in, go to the phase 1b GitHub repository, go on the issues tab, and submit an issue.

[00:09:39] It is [00:09:40] really that simple. I guess the hurdle could be getting a GitHub account. I would hope that with this open source movement that we would all have one, even if it was the one attached to our organization. So I have a private one, which I work on 

[00:09:59] Alexander: with 

[00:09:59] Audrey: this Phase [00:10:00] 1 BA. 

[00:10:01] Alexander: How do you recognize these kinds of issues? So, in terms of how can you make sure that you understand the issue correctly? 

[00:10:12] Audrey: Oh, so I also want to add that I guess the users can contact us via LinkedIn as well. [00:10:20] It’s always an open, open Open door there. It’s important to prioritize the issues when it becomes a business continuity issue that becomes urgent.

[00:10:31] So a sense of prioritization on the issues is currently on the issues list. If you see it, we have things like the next [00:10:40] function we have to refactor, Oh, do this plot this way, kind of a thing. So, so those are secondary at the moment, but important in completing the package. 

[00:10:50] Alexander: Okay. Yep. Yes. So and of course, as I understand it, it’s not just you who can work on these issues. It’s also [00:11:00] others who can work on these issues, isn’t it? 

[00:11:02] Audrey: Yeah, so that lends us to this. Developing a product the dependent developing a software kind of workflow where we can have different branches so someone can work on it. It’s like an app. Someone can work on this page of the app and the other [00:11:20] person who work on the other page and they don’t conflict or they shouldn’t anyway.