Today I’m looking back at the PSI conference 2018 in Amsterdam. You’ll learn about my personal highlights and key take aways from the conference.

In this episode, I’m  covering the following episodes:

  • Nupur Kolis key note speach in the plenary session  about “The Future of Healthcare: Trends, Opportunities and Challenges“
  • Interview with Nelson Kinnersley about the workshop “Owning Your Own Development”
  • The session “Not Just Another AE Table” with an interview with Maria Costa about “Personalised Benefit-Risk Assessment”
  • The workshop “Learn How To Swing: Hands on Workshop on Preference Elicitation in the Age of Personalised Medicine”
  • The session “Estimands Case Studies”

  • The session “Gone in 60 Seconds (Poster Review)” including an interview with Jules Hernandez-Sanchez
  • The keynote by Steve Ruberg about “Statistics and Data Science: Is Six the Same as a Half-Dozen?”
  • The workshop “Improving Your Communication”
  • The session “Patient Centricity”
  • The session “What Matters Most? – A Scientific Advice Role Play” including an interview with Mouna Akacha
  • The session “A Picture Says More Than 1000 Tables – Interactive Data Review”
  • The session “Regulatory Town Hall”

Transcript

Best of PSI 2018 – my personal view

00:04
Welcome to the Effective Statistician with Alexander Schacht and Benjamin Piske, the weekly podcast for statisticians in the health sector designed to improve your leadership skills, widen your business acumen and enhance your efficiency. In today’s episode we’ll chat about the 2018 PSI conference.

00:26
I will share what are my best takeaways, key takeaways from this conference, which was a really, really nice, great conference beginning of June in Amsterdam. So I will talk about what sessions I attended, what I learned from them, as well as some interviews with some interesting guests and speakers from the conference.

00:49
So stay tuned. This podcast is sponsored by PSI, a global member organization dedicated to leading and promoting best practice and industry initiatives. Join PSI today to further develop your statistical capabilities with access to special interest groups, the video on demand content library, free registration to all PSI webinars and much much more.

01:14
Visit the PSI website at PSIweb.org to learn more about PSI activities and become a PSI member.

01:32
Hey, this is Alexander Schaacht with a new episode of the Effective Statistician. And this time, it’s the first time that I’m recording this completely alone, actually not completely alone, on a special topic. And this special topic is the 2018 PSI conference that happened early June in Amsterdam. And I want to give you a little bit of a summary from my perspective, my personal perspective.

02:01
on this pretty amazing conference that had lots of goals into it. So if you have missed the conference, then maybe you can get a little bit of a sense of what the conference was about and also a little bit of the learnings from the conference. Thanks a lot to Lucy Rowe, who helped me with this in terms of saying that’s completely okay to repeat it here.

02:30
I hope that also this gives you a little bit of a flavor of the PSI conference if you haven’t yet attended one and you sign up for the next one. This year there will be again of course abstract submissions later this year. So have already some thoughts about what you might want to submit to your as an oral presentation or a poster presentation.

03:00
to get, for example, travel approval and these kind of things easier. OK, so the conference kicked off with keynote speech by Noopur Kohli, who is the expert in the future of health care, coming directly from the Netherlands, where the conference was. And she talked about the future of health care.

03:29
trends, opportunities and challenges. And of course it’s quite difficult to kind of speak about the future, but she talked about a couple of trends that help us to kind of extrapolate into the future. And some key learnings for me were, or some key highlights were, that information on health and education becomes more and more important.

03:59
especially as across the world, all the people get older, all the populations get older, and we have more and more patients with chronic diseases. Also, data becomes more and more connected in the health sector, and there’s more and more apps out there that help you to better interpret your data or make some decisions based on your data or…

04:27
help you to make decisions based on your data. But we don’t currently have anything to distinguish the good from the ugly. So it’s very, very hard for the consumer to see what’s a quality application and what’s actually pretty bad and doesn’t really tell you the right things based on the data.

04:57
Also, I think the patients will take more responsibility in the future, more and more responsibility already now. So we move much more toward a co-decision making between the physician and the patient. And I guess in some cases, maybe just a patient decision. And with all this data and this connectivity,

05:25
She talked that maybe we’d move from purely health care companies to much more kind of technology companies. And I think there is already some trends in there that some of the big players in the tech space enter into health care. Okay.

05:49
And then I went to, there were a couple of sessions, actually after that, where it was very, very hard for me to decide where to go. There was one workshop where I was really, really interested to go. It was about owning your own development. But as I couldn’t attend it, I interviewed Nelson Kindersley. And here’s the interview.

06:17
So I’m standing here with Nelson Knoesli, and he this morning ran a session about owning your own development. For me, I think it’s a quite important topic. What actually led you to have such a session?

06:34
Thanks Alexander, yeah and certainly it was not just myself, I must also credit Margaret and Gemma for their workshop as well. But yes, you’re right, I did open it with a few of my observations from various roles that I’ve had as a statistician, both as an individual contributor but also managing teams as well. And I think that, you know, in our formal education there’s very little coverage of many of these areas of leadership.

07:04
I think as the audience saw, that we’re working through that at Roche and we have come up with 14 skills that we believe statisticians can be most successful in addition to their more technical skills. And why do you title it owning your own development? Thanks, yeah. So one of the things which is certainly often talked about is…

07:31
What’s the role of the individual? What’s the role of the manager? What’s the role of the organization? And what I was briefly laying out were, certainly one can rely on some of the organization and some of the things with your manager, but for this to be truly successful, I and I’m sure others believe, that you have a role to play. And so some of the things I was describing were, what are the actions? What are some of the tactics that you can take?

07:57
to actually be successful and take accountability for some of your own developments in terms of non-technical or soft skills. And if you would give our listeners one kind of key takeaway from this session, what would that be? Wow, you do ask tough questions.

08:17
I think by the end of the workshop session, I think they, particularly Margaret and Gemma’s examples showed how sometimes we forget about the big picture. It’s quite easy or maybe it’s quite tempting to jump in and feel that you’ve been asked a question, jump in with your action plan, but often pausing and wondering actually what’s the big picture? What are the big decisions that might be taken by our work can actually help you in the long

08:47
and certainly in a team environment they can help you in the long run. So first, thinking it through a little bit more and seeing how my specific work fits into the overall strategy of the overall team.

09:03
and connecting these to the bigger picture. I certainly agree with that. And also, if necessary, clarifying, because often we get questions and we maybe don’t fully understand the context. And I think as part of the workshop, what we were hearing was, do take that time and then get that bigger picture. And then take your accountability seriously, work on your deliverables, and hopefully with the rest of the team, you can be successful. Okay, thanks a lot, Nelson. Thanks, Alexander.

09:30
So this was the interview with Nelson, which session I couldn’t attend, unfortunately. Instead, I actually went to the not just another AE table session. And behind the session is actually mostly it was about benefit-risk analysis.

09:54
There were three of the four presentations coming from members of the Benefit Risk Special Interest Group and that was a contributed session. Alberto Garcia Hernandez made a very, very nice presentation about addressing intercurrent events with a treatment policy strategy in survival analysis.

10:17
So he showed kind of how you can use flow diagrams to highlight the different flow of the data for the different intercurrent events, and showed how you can use then inverse probability weighting to manage the censoring. There was also a little bit about discussion with these weights. For certain patients, you may get in certain situations very, very extreme weights. So they get an

10:46
overly emphasis and a lot of impact on the analysis. One way to actually deal with these kind of waiting problem, and I think this is similar to lots of different other areas where you have these re-weighting of patients, for example, also in matching adjusted indirect comparisons or in, you know,

11:14
propensity rescoring or a couple of these other things where you reweight patients, you always have these problems with these very, very impactful patients. And one way to address that, which I think is very, very clever, is to put different caps into your analysis. And of course, you know, where you put this cap, so let’s say…

11:43
You don’t allow the patients to have a weight of more than two or more than three or more than five or more than 10. And you can basically do sensitivity analysis for these kind of things and show how the cap actually changes the results. So you can see at least how robust your data are with respect to these very, very influential patients.

12:14
Of course, one of these topics within the Benefit Risk Session was also about preference elicitation and, for example, using discrete choice experiments. And currently, every company is more or less doing their own studies there. However, for certain indications where there’s lots of research and it’s…

12:43
let’s say rather stable in terms of what kind of variables and attributes you need for such experiments, it would be great to have something like company independent results. So some kind of indication standard for weights. I think that would help a lot to make patient preferences more being applied across the industry.

13:14
The last presentation in this session was by Maria Kostar, and she talked about personalized benefit-risk assessment. And for that, I have another interview. So now I’m standing here with Maria Kostar. She is the chair of the SIG for benefit-risk, and she gave a very interesting presentation about personalized benefit-risk.

13:43
approaches. Could you tell us a little bit about the key takeaways from your presentation? Thank you so much, Alexander, for

13:53
your interest in the presentation. So I would say that our interest in this topic spiked because we started discussing the problem that when you do a benefit-risk assessment, it’s not always the case that one size fits all. And so we started thinking about how

14:23
patient with certain characteristics. So with that in mind we developed an algorithm where you say okay given your specific characteristics typically referred to as covariates what is the probability that a treatment that is known to work only in a specific subgroup by design, note this is for

14:53
What is the probability that that treatment provides a positive benefit risk? So we are still very much in the early stages. I think what I would like people to take home is that it is important to think about benefit risk as a very heterogeneous and complex problem. And that heterogeneity obviously can also come from the fact that patients themselves are different

15:23
to target our medicines to the individual patient characteristics. We are still very much in early stages. I had very nice conversations with people during the coffee breaks here at PSI and have a couple of ideas on what to do next. So yeah, stay tuned. Yeah and I think that fits very well with the kind of overall topics that we had during this session where also

15:52
Daniel Saure presented about subgroup analysis based on the patient preferences, so not on the biologic characteristics of the patients and pre-treatment and stuff like this, but more on the personal preferences and on trade-offs between benefits and risks. So I think the two layers in terms of the heterogeneity, it’s on the biologic kind of things.

16:21
as well as the personal preferences where you can basically define patient-based benefit risk decisions. Absolutely. And it’s interesting you mentioned that because as you know we also had a hands-on workshop on preference and recitation. What I liked about the conversations that took place there was that through the interactive tool.

16:50
it was very clear which, in our simulated setting, which drugs performed well on which criteria. And therefore, if you have a patient that has certain specific preferences or because of their more biological characteristics, maybe this person is more frail, et cetera, et cetera, you can then say, okay, if this drug performs better on the safety side, then if this patient is more frail.

17:19
and they don’t really want to have any comorbidities associated with the treatment, then you know that you are making a more informed decision. So I think that’s really interesting how everything tries together. Thanks a lot, Maria. You’re welcome. Thank you. The next session I attended was a workshop and was a workshop that was organized by the Benefit Risk Special Interest Group. You see, I have a little bit of bias regarding Benefit Risk.

17:47
because I’ve been the chair of ZSIG, and that was a hands-on workshop on preference elicitation in the age of personalized medicine. It was a really, really great workshop with an introduction to swing weighting, which is a particular method to elicitate these patient preferences.

18:14
There was a case study presented for everybody to kind of understand. Also an introduction into actually how to do swing weighting. It’s a very, very easy technique that you can use. And then all the different tables were given different perspectives. So one table was a payer, one table was a patient perspective, there was physician perspective, regulatory perspective.

18:44
And then people really kind of dived into this role play and thought about their personas and then had a really, really good discussion about how they wanted to weigh the different attributes. And not unsurprisingly, all the different tables came up with very, very different ratings.

19:14
And I think that is quite good to understand that different stakeholders will have different perceptions about how to assign trade-off weights between efficacy or different efficacy endpoints, as well as efficacy versus safety endpoints.

19:39
Then I went into another session that was very, very, of course, good attended, well attended, and that was the estimates case study. Estimates, of course, one of the hot topics of this conference. There were a couple of presentations about that. One was about specific working group in Alzheimer’s disease.

20:08
that looked into how to best come up with estimates in this setting, also kind of for studies that look into the more milder forms of the disease. And of course, these Alzheimer’s studies are very, very long-term studies due to the slow progression of the disease, and therefore lots of things can happen in these long-term studies.

20:38
Another very, very good presentation in this estimates case studies was about the composite estimates strategy in confirmatory clinical trials given by Oliver Kean. And what I loved about this presentation was that he showed that

21:06
This composite approach is usually thought of as you do a binary approach. You are a non-responder if you drop out, and you are a responder if you’re still on treatment and are actually a responder on some kind of different scale. But he showed that, well, actually you can also, you know, depending on the intercurrent event, you can have different ways to categorize you.

21:36
So an AE or loss of response might be something different than, for example, if you die or if you drop out of the study because just you need to move the city and you can’t continue with the study. So that, I think, was very, very good to have more categories in terms of this composite approach.

22:06
Then, of course, you kind of have the problem that you don’t have a binary endpoint, but you have maybe an audit categorical endpoint. And for that, there’s a couple of non-parametric ways to analyze that. And there’s lots of research by Professor Brunner and his colleagues from Göttingen that has published on that. So if you look for publications,

22:36
Yeah, with Brunner on it, Edgar Brunner, you’ll find all the different things in it. He has published with various authors on this topic, and there’s lots about how to analyze such data. So that was a very, very good one. And the last presentation in this session was by Luvisa Berggren. And I’m not talking too much about this today,

23:05
That will be next week’s episode. I have her as a guest.

23:12
After that, we had the poster review session, Gone in 60 Seconds is the title, where all the different poster presenters had 60 seconds to present and pitch their poster. And it was a plenary session and it was very well attended and it was really a lot of fun. So there were lots of people coming up.

23:42
with very, very good, well-prepared, short presentations of the posters that really pitched people to come to their poster. And there were also a couple of people that made an extra effort to stand out from the crowd. Because, of course, there were lots of 60 seconds poster reviews. And if you want to stand out from the crowd, you need to do a little bit of extra work.

24:10
So that was also a lot of fun. And regarding the poster session, I have another interview. OK, and now I’m standing here with one of the winners of the poster exhibition, Dr. Jules Hernandez Sanchez. And he got an award about his very, very nice poster about alternatives to comparing survival curves at the median. So what are your kind of key takeaways?

24:37
What are the key takeaways from this poster? Hi, thank you Alexander. It was a very nice surprise actually. I wasn’t expecting to win anything. It’s my first conference as well. So I was involved in a project where we based all our inferences in median differences. And for that particular project it was okay because there was no difference in the treatment versus the standard control arm. But…

25:03
But what happens if the survival curves are very close at the median but different anywhere else? If you base your estimator inference on the median difference, you would say there is nothing going on. But if you look at the curves, there is a lot of things going on. So I proposed to apply simultaneously four different statistics, which are very simple. They’re out there already, log-brand tests, pairwise, generalized comparisons. By Buies, you can Google it.

25:32
restricted mean squares, survival time, and something I came up with, which is a Hodges-Liemann estimate of the difference between medians. And it’s a very simple implementation. I mean, you can have a lot more sophisticated methods. In fact, next to me there were two posters with a lot more sophisticated methods, but what happens is when you go back home, you don’t implement them. And this is very simple. And every test extracts…

25:57
a different amount of information from the comparison in curves and you start gaining more insight into them. And for instance restricted means survival times tells you the average life expectancy up to a certain point, pairwise generalized comparisons compares every single time from one arm against the other and gives you a proportion of which arm has more in favor than against, etc.

26:24
So you propose to always, if you want to display some differences in the means, to also display all these other things together. You would power your study for one main analysis. But you want to describe your overall… Exactly, exactly. You can complement that main analysis with these four estimates and it gives you a lot more information, different overlapping information, not exactly the same, but they start building a statistical.

26:52
a picture of what you see. Very nice. Thanks a lot. You’re welcome. Thank you, Alexander. Fun was also the networking event in the evening. So we had a very, very nice restaurant bar directly at the water, the Hannigan’s Booms. It was in walking distance from the conference center and it was really great meeting old friends.

27:21
and also getting to know some new folks. So that was very, very, very nicely organized from the conference as well. So next morning, there was another plenary session where Steve Rueberg, formerly Lilly, now has his own one-man show company, has talked about statistics and data science.

27:51
is six the same as half a dozen. And that was a very, very, very good presentation. So Steve Rugeberg has built a complete kind of, you have what many would call a data science department within Lilly. And he talked a lot about these kind of areas where you can have an impact as a statistician beyond drug development.

28:21
And he challenged the organizers of the conference that choose the title, Breaking Boundaries in Drug Development. Maybe we should have chosen Breaking Boundaries Beyond Drug Development. So that was very, very nice. It was very thought provoking in terms of that we need to step outside of our kind of comfort zone, that we need to look more into these kind of spaces beyond clinical research.

28:51
And he bolstered that with a couple of examples from pretty prominent examples, also from Google and other kind of prominent players, where they had bad examples of data science. That, in the beginning, looked very, very promising, but actually turned out to be complete flukes. So he’d

29:17
really asked us as statisticians in the healthcare sector to step up and go beyond. And what was also very, very nice of him, he showed how he actually did that within Lilly. So he shared a couple of examples how he carved out some resources, put it onto projects.

29:48
made a success, created further demand, that led to more resources, and through that, really, really established that. So he had all the different kind of features of a change initiatives in his presentation, and he talked to them why you need all these kind of different features of change management in order to come up with a robust plan and just

30:17
kind of a strategy. He talked also about the visions that you need to have in order to get there and these kind of different things. Very, very rounded presentation. Lots of applause afterwards and I heard lots of very, very, very good feedback about that.

30:39
After that, I went into another workshop. So there were actually a couple of really, really nice workshops in this. And this one was about another of my favorite topics, which I actually only discovered afterwards. So the workshop was called Improving Your Communication. And they had some really, really nice, so to say, games.

31:10
that showed the people on the games the power of graphical presentations, the power of figures. So there was one game where the different tables were given all the same data, and then the tables were asked to compete to get some kind of features from the data, you know, the maximum or the minimum or things like that.

31:39
So some tables always kind of first, much faster than the other tables. And I was sitting in these lower tables, and wow, how are they doing this? They’re so fast. And only later it was revealed that some of the tables were given just the results tables.

32:09
that we usually have in our clinical studies, and some had graphical representations of these. And of course, the tables with the graphs were much faster. Very, very nicely done. Also about what to look into the different graphs, what to take care of. Perfect. Very, very nicely done by.

32:37
Gemma Hudson who presented that, as well as a couple of other people helping her. Great job. The next session, I was actually the chair and I organized this invited session about patient centricity. And it was an invited session on the topic of benefit risk. One

33:04
presentation I would like to expand a little bit more on. That was an update from Prefur, which is actually an IMI project looking into patient preferences and patient preference elicitation. And Chiara Vicello actually did a very, very nice presentation about the first outcomes of this Prefur IMI project.

33:34
She talked about how to categorize all these different patient preference elicitation approaches and which recommendations the team came up with how to use this. So I think this will be really, really important because that basically sets standards for the industry in the future. And I think in the future…

34:04
we will need to follow these guidance because it’s pretty broad expertise that was there. And

34:14
If we deviate from it, we probably need to argue and discuss it while we deviate from it. So I think it’s, I’m not sure whether I would call it a points to consider document, but it probably goes into that direction. So if you work on patient preferences or if you, I think I would encourage you to have a look into that.

34:43
if you work in track development, have a look into the updates from Prefurb.

34:51
That was the last session I attended that day. And in the evening, we had the Gala dinner. The Gala dinner was actually at the same place where also the conference was actually a beautiful building. It’s a very, very old building in the center of Amsterdam that had these amazing rooms, really, really large rooms. I think one of the last royal

35:20
weddings were also there. And so obviously, really great location. But not only a great location, also food was very, very good. And afterwards, there was a really, really nice big party. So lots of people dancing, lots of people making party up to really, really late. And it was great for meeting friends and having lots of fun.

35:49
We started a little bit later with conference due to the gala dinner knowing to take a little bit longer, so people have a decent time to rest and we also started with a little bit of a different session. So the session was about what matters most, a scientific advice role-play. So we had a couple of folks

36:19
on the podium at the conference. I was one of them. And all of them presented different stakeholders. Steve Ruhlberg actually made a great patient. And he talked about his experience with these kind of patients, because he specifically interviewed a couple of them. And what I found really interesting is that, um,

36:48
He figured out all these patients were always talking in conditional terms. If I take the treatment, then what do I expect? So they were kind of very, very much looking into the estimate framework from this conditional point of view, which I thought was really, really interesting because I personally was thinking very, very differently.

37:18
I took actually the viewpoint of the HTA person, given my kind of background where I’m working. And because of that, I of course was very vocal about the treatment policy. And of course, I’m not an, you know, I’m not employed by an HTA buddy. So it’s, you know, just…

37:48
imagination of what could happen, and also, of course, to have a little bit of controversy there. I also challenge people a little bit to think about safety and efficacy and having similar estimates there, or maybe even the same estimates there, from a kind of intercurrent

38:14
treatment strategy perspective, from a population perspective, of course not from a variable and from a measure point of view perspective, but these kind of estimates need to be somehow consistent across all the different endpoints that I want to make a kind of summary statement across.

38:44
compound versus another active compound? What are all the negative things of this compound versus compound I have already in the market? So if I sum that up, I need to somehow have a consistent estimate approach because otherwise I think it’s really difficult to kind of come up with a conclusion statement in terms of the benefit-risk assessment or the added benefit.

39:14
this new drug will have in the marketplace. Of course, there was no agreement between all the different stakeholders, not surprisingly. And the regulatory point of view was more kind of on treatment estimate. And there was also some discussions about, OK,

39:43
what should be your primary estimate, for the secondary estimate, so the primary one being for the regulators. And you can still have other estimates specified, pre-specified as secondary analysis, which most HDA bodies would be completely fine with. Just have them pre-specified. And as Muna actually let.

40:11
this session and shared this session, I also interviewed her. I’m standing here with Muna and Muna has led lots of activities around the estimates topics and also at this conference she was quite prominently in sharing sessions on this. So Muna, what are your key takeaways from these estimates sessions?

40:37
Thank you Alexander for the kind introduction. So I guess my main takeaways are…

40:43
Firstly, that the topic is still very hot. So we had actually quite a few sessions around estimates. One was specifically about case studies. So that’s something that we heard at last year’s conference that people wanted to see more tangible examples that they can relate to. So we saw very nice discussions in the context of Alzheimer’s disease, oncology, but also in HTA discussions. So it sort of broadened the view, I would say, at this conference compared to last year’s conference

41:13
but also conferences that I’ve seen outside. Secondly, I think one point that was, that repeatedly came up, first of all, in the case study session, but also in a session on non-proportional hazards, was that there is still a lot of discussion and thinking, I think, needed for estimates in the time tree event setting. Okay. So some points that were raised were around.

41:39
the value of the hazard ratio, for example, as a summary measure. That is, I think, a hot topic and I think it will become more and more of a discussion topic for us but also for, I assume, regulatory agencies. So what would be the alternative? Well, I mean, there are quite a few alternatives out there, right? So the hazard ratio is just one.

42:05
way of summarising the treatment difference between two survival curves. Alternatives were published, for example the restricted mean survival, or you could look at the difference of survival curves at a certain time point. And some of these alternative summary measures may be more flexible, let’s say. They may not rely as, or they do not rely on the proportion of hazardous assumption as the hazard ratio actually does. And…

42:34
In fact, there were a couple of publications which are not that recent. So there’s a paper called The Hazard of Hazard Ratios by Hennan, for example, and then Arlen and Cook have published a paper on this as well. And so this notion that the hazard ratio may not have a cause and interpretation itself starts to keep coming up in the literature, and I think we in the pharmaceutical industry.

43:01
will sort of have to face those challenges and at least discuss it. If there’s no issue at the end and we can still use the hazard ratio, that’s great. If you will have to look for alternatives, then I think this conference was a good starting point for that because we have actually connected these two topics a bit at this conference. And this also sort of brings me to the third new aspect, or let’s say one aspect that is related to STMens, which I think is very nice. So we had a pre-conference course on causal inference. We also had a whole session on causal inference.

43:30
causal inference yesterday, so on the Monday, the first day of the conference. And I think what’s nice about this is that while causal inference or the word causal itself was not mentioned in the ICHN 9 addendum, this idea of actually causal inference beyond ITT and randomization is something that I think the ICHN 9 addendum has brought a bit more to our world in the pharmaceutical industry.

43:59
And it’s very nice to see that the PSI is actually picking this up and including training courses but also relevant sessions in this conference so that most of us who are not that familiar with the topic can actually have a glimpse at what was already done maybe at other companies, what type of estimates people look at and also how maybe we can expand our toolkit but partly also to see that some of the tools that we already know we can in fact use for some of this.

44:28
What for me was kind of a difference here was this causal inference in the estimates framework, because for me, working on phase three, phase four for quite some time, having doing observational studies, all these kind of propensity scoring and inverse probability weighting was something that I was used to, but to the…

44:55
observational study setting and not to the randomized clinical trial setting. And I think that’s quite new. That’s a very good point and I think a lot of these discussions actually come through the notion of this intercurrent event and the appreciation that these are not complications, they are not a nuisance, they are sort of part of the picture. And I think, I mean there’s a paper by Hernan and I keep quoting him, but I think there’s a very nice paper by him which is essentially saying that…

45:24
After randomization, as soon as people start dropping out for lack of efficacy or adverse events or start taking another medication, randomization is sort of lost for certain.

45:38
So I think the paper’s entitled something along the lines of randomized studies analyzed as observational studies. Yeah, I was just thinking that way. And I think there are a lot of connections there. And yeah, I mean I think a lot of the methods and thinking that has been, let’s say, maybe more standard in observational studies, late stage studies, real world evidence, may become something that we want to look at as well.

46:07
But over time, the more dropouts you have, the more your study becomes kind of observational. In a sense, of course not in a sense from a kind of legal perspective, but just from a kind of how the data looks and feels. So why do we do randomization? We do randomization to get balanced groups such that at the end of the day…

46:33
if I’m only changing treatment in both groups, if something changes at the end, I know it must be because of the treatment. But now, as you said, when you have dropout and all these other intercurrent events happening, you have a lot of other selection processes going on. And whatever you now see at the end, you’re not as certain anymore as it’s because of the treatment that’s actually taken. So yeah, it’s creating a whole lot of new, let’s say, complications but also food for thought.

47:02
It kind of helps me to get a completely new viewpoint on things. Thanks a lot for this short interview, Gunnar. The next session I attended was on the other half of my favorite topic, which is visualization. And I’ve actually chaired that session. And the first speaker here is someone that you already had here on, heard here on the podcast as well.

47:31
Zach Skrivenek and he made a very very nice presentation about the power of visualization and the future of visualization and what we as a function can do about it. We also had two great case studies by colleagues from Bayer presented. They also emphasized very much what you can do in the safety space.

48:00
visualized data, and they even had some audio features in it, which I found really, really interesting. So when you see how the AEs come and go over time, they had audio features in it when these AEs actually occur, so that you don’t miss out if something changes in your data. So.

48:29
very, very kind of sophisticated graphs and they implemented them in R and these are actually available. So just check out the conference program at psiweb.org. There you will find a link to the conference and the science, scientific program of it. If you click on these different sessions you will see…

48:57
who presented there as well as the abstracts, so you know who to contact there. The last session of the conference is always a regulatory session, the regulatory question and answer session. So throughout the conference and before the conference, you were able to submit questions. And so we had a couple of different people from regulatory there on the podium.

49:24
And it was a really, really interesting discussion. One of which was, of course, about estimates, which is a pretty hot topic, of course. And I found it really interesting. There was one thing mentioned about cardiovascular studies where sometimes it’s not completely clear whether it’s a safety event or whether it’s a efficacy event

49:56
these things go into each other. And of course, that maybe helps you think about similarity between estimates for benefit and estimates for risks, not in two different ways, but more think about them linking them to each other.

50:26
They also mentioned that in terms of estimates, it’s really important to have an open discussion. And what I found particularly interesting, it was a remark that the estimate doesn’t need to always fit into the five existing categories. And if you want to deviate from it, you just need to explain it very, very good. Another.

50:55
topic was about consistency across studies. So for the same indication, we should strive to establish something like consistency across the different studies. I think that would also from a commercialization point of view, from a kind of medical understanding point of view, would help a lot.

51:24
If we could establish for certain kind of indications, we could establish some kind of standard approaches, how to analyze these things and how to interpret different estimates. Another topic, pretty big topic, was about basket umbrella platform studies.

51:51
And it was a couple of times repeated that these terms are not necessarily clearly defined. So don’t focus too much on these terms. Focus more on the features of the studies. And one quote from the regulatory session, which was really a highlight for me, was, remember, p-value is not the holy grail.

52:22
It’s not the end of the story. So p-value is not the holy grail. I think that is really, really important, because we as statisticians sometimes overestimate that. And maybe our medical colleagues do that even.

52:39
There were some discussions about Bayesian analysis in phase 3, and the regulators made it pretty clear that they don’t see a need for that. As the phase 3 study needs to stand on its own, it’s a confirmatory trial. So all the frequentist approaches that we currently have are completely sufficient for that.

53:05
Another topic was about subgroup analysis. Describe why you do subgroup analysis. Explain if there’s kind of some standard expectations on certain subgroups for a specific disease, because they are analyzed always. And also state whether there’s kind of specific prior knowledge about it, a prior belief.

53:34
based on some data that you have seen. So that is also very, very important. Okay, and with that, I wanna end. It was a great, great conference. And if you have missed it this year, make sure to prepare in time for next year’s conference. If you wanna give a presentation, if you wanna present a poster, think about the topics. Maybe you can already, you know,

54:04
work about them, finalize them, so that you are able to submit these in the second half of this year. So it makes it easier for you to attend next year’s conference. Thanks and goodbye. See you next week. We thank PSI for sponsoring this show. Thanks for listening.

54:27
Please visit thee to find the show notes and learn more about our podcast to boost your career as a statistician in the health sector. If you enjoyed the show, please tell your colleagues about it.

Join The Effective Statistician LinkedIn group

I want to help the community of statisticians, data scientists, programmers and other quantitative scientists to be more influential, innovative, and effective. I believe that as a community we can help our research, our regulatory and payer systems, and ultimately physicians and patients take better decisions based on better evidence.

I work to achieve a future in which everyone can access the right evidence in the right format at the right time to make sound decisions.

When my kids are sick, I want to have good evidence to discuss with the physician about the different therapy choices.

When my mother is sick, I want her to understand the evidence and being able to understand it.

When I get sick, I want to find evidence that I can trust and that helps me to have meaningful discussions with my healthcare professionals.

I want to live in a world, where the media reports correctly about medical evidence and in which society distinguishes between fake evidence and real evidence.

Let’s work together to achieve this.