What is the ASAP and what you should know about it

Transcript

[00:00:00] Alexander: Welcome to another episode of The Effective Statistician. And you can hear I’m not 100 percent well today my voice is a little bit rough. That’s why at least rough as than usual. But we have a not rough topic. Today we have actually a very interesting topic today. And the good news is we have an absolute expert on the line to talk us through this really interesting topic that I would say pretty much everybody in our industry, or nearly everybody in our industry needs to talk about and needs to learn about, and that is all around safety. And specifically, ed Safety reviews. So, hi Greg. Nice to have you on the call.

[00:00:48] Greg: Thanks, Alexander. Nice to be here.

[00:00:50] Alexander: Very good. So, for those who don’t know who Greg Ball is maybe you can introduce yourself on a little bit how you came into being passionate about safety statistics.

[00:01:05] Greg: Yes. Yeah, so I’ve been a biostatistician now for over 20 years. . But in the last 10 years or so, I’ve specialized in safety statistics. This really came out of my dissertation on blinded safety monitoring procedures when I was getting my PhD. And it’s kind of interesting. I was a non-traditional student, so I was already working and I was going to school part-time. And it was difficult. I gotta tell you I think that’s probably why the universities dis weighed people from traveling down that path. But it gave me a big advantage when it came to my dissertation because I had already been working at MD Anderson and. I was familiar with some of the issues in clinical trials and with safety, and I really thought about it for a long time, and I wanted to come up with something that was going to be useful. And and so I came up with the blinded safety monitoring procedures and it, it took a little while. Really get going. But there’s been quite a few publications recently on blinded safety monitoring procedures. But I developed that when I was at AbbVie. And there’s people there who continued to develop it. But I was recruited to Merck to start up a safety statistics group there, which I did with Bill Wong.

He was the head and we developed an aggregate safety assessment planning process which also was being developed in parallel with the D I A A S a interdisciplinary safety evaluation scientific working group. But that’s really where I got my start at Merck. And more recently I was recruited to Novavax to start up a safety statistics group there.

But now I’m venturing out on my own. I’m gonna be starting a consulting group ASAP Process Consulting.

[00:03:08] Alexander: Cool. Yeah. And ASAP here doesn’t stand for as soon as possible. Or maybe it does.

[00:03:16] Greg: It’s an interesting story. When we developed the ASAP process at Merck initially we called it ASAP, but we sent out the draft version for review and somebody saw ASAP and they got.

At Merck they call it the AGSAP they put a G in there so people don’t mistake it for urgency.

[00:03:34] Alexander: Yeah, but, and actually you need to work asap on the ASAP if you don’t have one yet. So from that point of view, I think it’s a really nice acronym. Let’s step a little bit back.

You wrote about a changing landscape in the safety culture of moving more towards aggregated data. Where’s this coming from and who’s driving that in terms of looking into this differently?

[00:04:06] Greg: Yeah. In the pre-market space, it’s being driven by the US FDA . And it’s not anything new, right?

A lot of people think it’s a bit controversial, but FDA has been advocating for aggregate program level safety evaluations from at least as far back as 1988 when the ISS was introduced. So this is consistent with FDA thinking on. Safety evaluations, although I would say that in the post-marketing space the E M A and the other EU regulatory agencies certainly advocate for aggregate analysis of safety data.

[00:04:46] Alexander: Yeah. And we, when we say aggregate, Analysis of safety data that is the opposite of all the individual case reports that are filed, isn’t it?

[00:05:01] Greg: Right. So, there’s other existing processes that monitor the safety that you have. Medical review of individual cases. . You also have medical monitoring of individual studies.

But what we’re talking about here is aggregate program level safety evaluation. And it doesn’t necessarily mean that you are integrating the data together. But you’re looking at it holistically. So, If it makes sense, and you can, you should integrate data together. There’s more power in precision in a bigger sample size.

But you have to look at everything and, you should not blindly pull all of the data together. Yeah. You may have a collection of studies Where you have a similar patient population and it would be beneficial to pull those. But oftentimes development programs have studies in special populations like renally impaired or adolescents where the populations are identifiably different and then it doesn’t make clinical sense to pull them together. But you should look at them, right? You should look at everything, maybe have a dashboard, look at them side by side. But whether or not you pull them together you do need to look at everything.

[00:06:20] Alexander: Yeah, I’ve also worked in on compounds with over different indications, which differed by relevant co medication. Yeah. So in certain indications everybody got methotrexate and then it had an impact on the side effect profile and will certainly for certain side effects or for certain safety signals. It doesn’t make sense to pull, but yet for others, maybe it does. So it’s not a, the pulling strategy is definitely not something that is straightforward. It’s something that you really need to have a closer look into. Okay? So that comes from the regulatory side, and there is.

There’s this rule with this interesting name, the IND safety reporting final rule. What does it mean? And where’s the word final coming in there. Final language.

[00:07:19] Greg: Yeah. That threw me off for the first time I saw it. Final rule. Why is this the final rule? What if sometime in the future there’s a different way of thinking about things or better ways of looking at things. And final rule means at this point in time, it’s the final rule, right? Okay. So the FDA is saying this is the final rule on the IND safety reporting. There were rules on safety reporting before that, but this is the current final rule. And sometime in the future, if Have a reason to change that rule, right.

To update it because of new information or new technology. They will do that. But the final rule right now really clarifies what is needed for making an expedited report. So there’s three things you need you need it to be serious. You need it to be unexpected and you need it to be reasonably, possibly causally related to the drug or vaccine or what have you. So, there’s not really any controversy with Sirius.

[00:08:19] Alexander: Yep. That’s pretty clear.

[00:08:20] Greg: Yeah. Unexpected. There’s then it’s not expected. It’s pretty straightforward. The hard thing is the reasonably possibly causally related right now. In the past this all really focused on individual case review or case series. And that works really well if you have a single event, which is important or even a small cluster, right? So you might have an event that you wouldn’t surprise be surprised to see a couple.

But if you saw a few , that would be informative. And in both of these situations.

[00:08:57] Alexander: Can you give an, I’m sorry. Example. Can you give an example for. .

[00:08:59] Greg: The one that the F d A usually use is tendon rupture, right? So that’s not normally associated with drug exposure, but it would be uncommon in any population. It can happen depending on the population you’re talking about, whether they’re young or old, healthy or not, maybe seeing one or two wouldn’t be surprising. But if you saw a cluster of ’em, that would be informative.

[00:09:23] Alexander: Yeah.

[00:09:24] Greg: The problem is anticipated events, right? And this is a term that really as far as regulatory agencies are concerned, only the FDA recognizes. But it makes sense, right? In an elderly population. You might have MIS right?

[00:09:39] Alexander: Yeah.

[00:09:39] Greg: They might have heart attacks. So the whether they get the drug or not. So if you see that, that’s not informative, right. Maybe you’re studying a cardiac medicine and you expect what? No, I be careful. You anticipate that 5% of your patients over the course of the trial are gonna have a heart attack. . So if you see a heart attack, how do you know that’s reasonably possibly causally associated with your drug, right?

[00:10:07] Alexander: Yeah.

[00:10:07] Greg: And so now you have to analyze it differently. You need an aggregate analysis. You need a way of determining whether there’s a relative risk increase, right? not just that a patient had a heart attack, but , considerably more than 5% of the patients have a heart attack, or if you’re have an unblinded analysis are there more patients having heart attack in the active treatment group than the control group?

Right? And this is particularly a problem in ongoing blinded studies. How do you make that comparison? How do you make an unblinded comparison? Do you do an unblinded comparison? So there’s a lot of questions here, and I think that’s why the EU and other regulatory regions are have really stood back on this have not harmonized with the FDA on this issue.

[00:10:57] Alexander: Yeah. So that’s one of the topics. It’s currently FDA specific, although as far as I understand, there are efforts to in the future harmonize it and get it into I C H, isn’t it?

[00:11:12] Greg: Yeah. I think there have been efforts to get it into I C H, but I think the and this is just me talking, this is not any of the companies I ever worked for or the F B I really. The issue is the same issue that sponsors in industry have struggled with, this whole idea of unblinding, right? There was a controversial draft guidance that came out where the FDA recommended that you use a safety assessment committee to do broad and frequent unblinding of serious adverse events.

And we were very uncomfortable with that. That was back in 2015. There’s been a lot of collaboration between sponsors and the F D A presenting at conferences, getting feedback, and I think we’ve come to a alignment and in fact in the most recent draft guidance, the F d A put a out, a new draft guidance in 21. Where before the preferred method was the Safety Assessment Committee. Now the preferred method is the trigger approach where you look at the blinded data and if there’s evidence in the blinded data to suggest that there might be a relative risk increase, then you could do An unblinded analysis, and you can use a safety assessment committee.

It doesn’t have to be a whole committee. Now they say it, it could be an entity which, where an entity might be one or more experts for addressing these issues. But now, that there is a managed and controlled way to do this, right? We have achieved alignment and we’ve documented that in a paper that was published last year on this, on interdisciplinary safety evaluation.

And there’s different solutions for meeting the final rule that, that are published from several different companies.

[00:13:02] Alexander: Yeah. And by the way, and we’ll put these references into the show notes you’ll find everything there including more information about Greg and the ASAP and how he can actually help you with these kind of things.

[00:13:16] Greg: But just to close that thought, I got a little bit away from it. But the same issues that we struggled in the US to meet the spirit of the final rule, right? I think were issues that other regulatory agencies were not comfortable with. But now that we have the aggregate safety assessment planning process, right to, to plan and coordinate all of the aggregate reviews of safety data, to identify the safety topics of interest to come up with a pooling strategy and to have a managed and controlled approach for monitoring these anticipated events where you don’t have to immediately unblind, but if there’s reasons, unblind, of course, you would want to do that. I think other countries are gonna become more comfortable with it now, just we did here.

So I, I really think. In the near future as the aggregate safety assessment planning process gains traction and more companies have a more systematic approach and have a managed and controlled approach for these anticipated events and IND safety reporting, I think other countries are going to harmonize with what the FDA has been doing.

[00:14:32] Alexander: Yeah. So there was also recently a meeting in Germany, the APF meeting in Berlin that is organized by the German. Working group of statisticians in the pharma industry. And that topic was also pretty big there that, you have this trigger rule or maybe you have directly have something that, where you have an unblinded assessment all the time.

The important thing how that I took away from it, you need to have a very good ASAP in place. Yeah. To make sure that everything follows a clear process and that everything is in line.

[00:15:11] Greg: Yeah, absolutely. That is the secret. It is leveraging the scientific expertise in the medical judgment of multidisciplinary safety management teams to have a safety strategy and a systematic approach for looking at these events. Right? And now you have to look at everything. So there’s two parts to this, right? So you have standard analyses, like your standard AE table by system, organ class and preferred term and other analyses. But you can’t look at everything closely. But you have to be, have an awareness of everything but you need to focus on. Those safety topics of interest, which are important for some reason. And this really needs to be driven by clinical judgment. And on these events you can look at them more closely, right?

So you’ll have a relatively small collection, a dozen, maybe a couple dozen that you’ll be looking at closely and you’ll understand, right? And these can be your anticipated events for possible IND safety. . And if you have a systematic approach and a safety strategy where you’re looking at these safety topics of interest, then that really clarifies things. And it ensures that in these drugs that are being investigated that the patient’s safety is being well thought of. And of course it continues in, into the post-marketing that patients that are taking these drugs we have a really good understanding of the safety profile, right? This process is really gonna develop a better understanding of the safety profile and characterize the safety issues with the drug, so it can be communicated to the patients and the caregivers.

[00:16:54] Alexander: Yep. So for the statistician that is now listening if his project statistician that oversees the molecule and the developer end of that what would you recommend? To him doing about ASAP?

[00:17:10] Greg: Right. So the ASAP helps coordinate and make consistent the safety endpoints in studies across the program, right? So, there has to be very strong communication and collaboration. Crossdisciplinary scientific engagement, I call it where these analyses are thought about on a program level and they’re implemented on a study level, right?

So that the endpoints and the data are all consistent so that when you bring it together for the I S or for answering regulatory queries. That they come together and you can make a consistent and authoritative assessment of all of the data, right? If you have different endpoints and you have different definitions and you have the more differences, the harder it is to bring together, right?

Yeah. For example when you have an anticipated event and you want to maybe do a blinded analysis, right? So you need to come up with a background rate, right? Yeah. So maybe you’re not first in class. Maybe there’s other drugs out there that have already completed and publish.

Studies, right? And they are in a similar patient population, but that’s a different company. And they have different definitions and they have there’s differences, maybe different geographic regions and you want to come up with a background rate. You, it’s not identical patient population and there’s no standard collection of terms. So, they may have a different collection of terms and their grouping and so coming up with a background rate becomes a complicated thing. And that’s really been the issue.

[00:18:54] Alexander: Yeah. And it’s not something that you come up with just by thinking about it for an hour. So having that discussion earlier on is for sure something that is important. So, If you are in this role and you’re not working in a company where, looks like that is already rolled out. And there’s a couple of companies things that are pretty much leading the pack. I heard Lilly is pretty good. And this if you work at Avie, that’s probably another candidate. But if you don’t know about the ASAP, for your compound, I would say ask ASAP about the asap. So as you can move forward.

[00:19:33] Greg: Yeah, absolutely there’s starting to be some good literature on that. There’s a paper that the day’s working group wrote recently on the asap where there’s a lot of companies working together to create that manuscript.

[00:19:46] Alexander: Yeah. I have one final question. We talked a little bit about blinded and studies. So if in my program I only have unblinded studies, what happens then? Do I actually need this?

[00:20:00] Greg: Oh yeah, absolutely you need this ongoing blinded studies is a special situation, okay? And they don’t stay blinded forever. Eventually the study’s complete and then you have unblinded information, right? Okay. But what we’ve learned is that safety evaluation is on a continuum here.

You’re continually learning. And for example, historically the first time you would look at aggregate safety data was, is the time of the submission with the I s and the C T D. Right. And before that you didn’t bring the data together. You didn’t pull it for any of your analysis and then oftentimes that would be the end of it. You pooled it for that, and then maybe you go on and you study it in new indications to expand the label, but you don’t have a formal process for integrating the data again, right? But the knowledge continues, it builds on, it doesn’t stop at the submission. And then you have a new knowledge the knowledge. Accumulates and grows, and our understanding grows. So really in pre-marketing safety monitoring and at the time of the specification and in post-marketing, safety surveillance, these are not all different things. This is a continuum and you have to learn throughout that process and you might have blinded studies for a time you might have more unblind studies as they accumulate, but the knowledge is accumulating all this time. And as the knowledge grows you, you need a systematic way of synthesizing all of that information together. So whatever program you have an ASAP process is critical in order to have a systematic approach and to streamline your analysis. You one, it saves money, but more importantly, it improves your understanding, right? The way you pull all of this information together think about having a study without a protocol, right?

[00:22:00] Alexander: Yeah, that would be, that would drive me having sleepless nights.

[00:22:05] Greg: Yeah, so this is the same thing, but on a grander scale, right? Because the safety is continuing and it’s not just one study, it’s all studies. So you have to have some kind of planning I call it planning process. I don’t like to call it a plan because it’s not. It’s an ongoing process.

But it continues and if you have that plan not only is that going to save you money in the long run because you’re going to be much more efficient in how you plan and coordinate all of these aggregate reviews of safety data, but you’re gonna do it in a strategic way. So you’re going to do the analysis that help you understand right? You’re you’re not gonna have a whole bunch of analyses, most of which aren’t helpful. You’re gonna focus on those analyses that really help you to understand and characterize those safety topics of interest.

[00:22:54] Alexander: Yep. And I really love the picture was study without a protocol. Yeah. A SAP process helps you to have a, let’s say, compound wide protocol for all your safety stuff, and that helps you. To have a good feeling about all the safety aspects all the time so that you can say, okay, things are under control and things are managed and things, when we have new studies, when you have new indications on ongoing basis. We know where we are in terms of the side effects. Any kind of, we can know how we can manage. Unexpected things. And yeah, that definitely takes out the panic factor moving forward. Awesome. Thanks so much Greg, that was a great discussion about where we started with more than 20 years ago, starting with ISS how that all developed.

What was the IND safety reporting final rule through all of statisticians in there. And yeah, pretty cool thing about the A S A P. Is there one thing that you would like as a listener to take away from this podcast episode?

[00:24:13] Greg: Yeah. So I think there’s probably a lot of smaller companies that are struggling with this idea and they don’t have the resources and the infrastructure to do some of these things that are being talked about in the literature. All that is needed really is the asap. If you have the ASAP process in place that satisfies 90% of all of the issues. And now if you have a bigger program with a lot more information coming in then you need some safety assessment committees and you need some more sophisticated quantitative tools and methodology. But probably for the smaller companies with the smaller programs, just having an aggregate safety assessment planning process in place to help guide investigations on the safety of their products.

As long as they have a multidisciplinary process, they have it’s designed specifically for that drug and that patient population. They have some quantitative ways of measuring the evidence not p-value and decision rules. And a process that’s driven by medical judgment. If you have that right which the ASAP process delivers, then you’re meeting the spirit of the final rule and this is all very attainable even for small companies.

[00:25:39] Alexander: Yeah. And if you need some more help with that want to check with everything is fine. Just reach out to Greg. You can find all his details in the show notes, and there will also be a special section on the effective decision homepage about what he helps with and all the different things you can get from him and his company so you are in the safe side with safety. Thanks so much. Have a nice time, and thanks for being on the show.

[00:26:10] Greg: Thanks a lot Alexander.