We’re all making decisions to optimize our studies. Optimizing project plans across different studies presents further challenges. But how to increase the overall portfolio of a company given its limitations leads yet another set of challenges. Knowing how decisions are taken at this top level helps to understand the bigger picture and why this sometimes implies sub-optimal decisions on a study or compound level.
Another big learning from these complex interactions are the influencing skills needed to work with senior and diverse teams.
Only a few statisticians in the industry work in these areas and Andreas Sashegyi is one of these.
In this episode, he will share his learnings and we cover these topics:
- What is decision science?
- What is the role of a statistician in decision science?
- In which way differs the role of a statistician in decision science from a role in clinical development?
- Which capabilities do you need as a statistician to succeed in your role in decision science?
- Which barriers did you need to overcome in decision science and how did you do this?
- Now being back in clinical development, how is your time in decision sciences influencing your work ?
- Given your learnings from decision science, what would you encourage people to learn to be more impactful in their day-to-day job as statistician?
Andreas Sashegyi
Andreas Sashegyi obtained a PhD in Biostatistics from the University of Waterloo in Canada in 1998, and joined Eli Lilly and Company later that year. Over his two decades with Lilly he has held various positions across numerous therapeutic areas, focusing on late-stage development of compounds within the endocrine, cardiovascular, auto-immune and oncology spaces.
He also spent six years as an advisor of Lilly’s Decision Sciences group, consulting within R&D on major investment and portfolio management decisions.
Dr. Sashegyi is currently senior research advisor and therapeutic area lead for statistics, for Lilly Oncology.
Featured courses
Click on the button to see our Teachble Inc. cources.
Transcript
How to make the right portfolio decisions – Interview with Andreas Sashegyi
00:00
You are listening to the Effective Statistician Podcast, episode number 33, How to make the right portfolio decision, an interview with Andreas Saschegi. Welcome to the Effective Statistician with Alexander Schacht and Benjamin Pisker, the weekly podcast for statisticians in the health sector, designed to improve your leadership skills, widen your business acumen and enhance your efficiency.
00:29
And here are some exciting news I have for you. I’m creating an online course together with a colleague from the US that you will learn about in two weeks in an episode where we will talk about leadership skills. And this online course will also be about improving your leadership skills as a statistician, even if you have no direct reports.
00:55
If you want to learn more about it and keep up to date, register your interest on our homepage that we have created exactly for that at thee slash course. So just leave your email address there and then you will be notified when there’s any updates regarding this course. We will create the course and it will start about early next year.
01:24
and you will learn more about it in the upcoming episodes. In today’s episode, we have also something that has a little bit to do with leadership. It’s about making the right portfolio decisions. And for that, I have a very, very dear friend, Ander Reyes-Saschegi, who is an expert in this area and has worked as a statistician on these kind of decisions.
01:52
And I think that will be quite interesting for you to hear about, even if you’re not working on these kind of things, because there’s lots of learnings for other situations as well. This podcast is created in association with PSI, a global member organization dedicated to leading and promoting best practice and industry initiatives.
02:14
So join PSI today to further develop your statistical capabilities with access to the special interest groups, the video on demand content library, free registration to all PSI webinars and much much more. Just visit the PSI website now at www.psiweb.org and become a PSI member. Just one last note before we get started with the episode.
02:44
so we used some alternative tool to record the episode. The audio quality is not that great, but I can promise that the content quality is really, really good. So please stay tuned.
03:03
Hi, this is another episode of the Effective Stat Station, this time with Alexander Schacht only without my co-host Benjamin Piske, who is on his well-deserved vacation. And today I’m with a good friend and colleague, Andreas Saschegi. Hi Andreas, how are you doing? Hello Alexander, very well and yourself? I’m very, very good. So today we use a little bit of a different technology.
03:33
isn’t causing too much problems with the audio. But anyway, I’m pretty sure that the content will be amazing. So stay tuned. So Andreas has a very, very long history working at Illy Lillie and he worked in kind of the different classical areas where clinical statistician in the track development
04:03
but today we’ll not talk so much about this, but more about a special kind of career steps that he made some time ago. But before we go into this kind of episode, maybe Andreas, you can introduce yourself a little bit. Sure, thanks, Alexander. So I am Andreas Ascheg, a senior research scientist in…
04:28
Statistics at Eli Lillian Company. I’ve been with the company for over 20 years, spending most of that time in clinical drug development, specifically late phase development, but also spending six years of that time in our decision sciences organization. Yep, and this is exactly what we will talk about today. So decision science.
04:57
What is that actually? I would define decision sciences as the collection of methodological as well as psychological principles involved in rational decision making. So that’s two components there then. On the one hand, it involves technical and I will say often statistical elements in terms of the synthesis and logical interpretation of information that’s relevant to a decision.
05:26
And further to that, I will say that the proper accounting for uncertainty, for instance, is a key component in that process. But beyond the technical aspects, decision sciences also recognizes the impact of psychological factors in decision making, such as the fundamental motivations of the decision maker, conscious and unconscious biases, cognitive traps, and so forth. And it accounts for these in analyses to support a decision.
05:55
and in developing recommendations for a decision maker. So if we talk about decisions, what kind of decisions are we actually then talking about?
06:13
We are typically talking about decisions regarding a clinical development plan. That’s one example. So, for instance, teams have to decide when preparing for the next stage of development what kind of trial to run, what questions they will answer, what alternative options they have. And the tradeoffs that they consider in decisions like that are typically related to the kind
06:43
to timelines as well as costs, probability of success, and the value of those decisions. So for instance, should you run a smaller or larger trial, which has of course cost and time implications, what will be the value of the data generated from such a study? Is the added benefit of a larger, more robust trial worth the cost of the trial?
07:10
the trade-off required in terms of additional time and expense. So around clinical development plans, there are decisions like, there are, you know, typical, the typical decisions that are required require thoughtful analyses in terms of the trade-offs on these various metrics that I’ve described. The second kind of decision at a portfolio level would involve, for instance, investment decisions.
07:39
in one compound versus another if balancing really a portfolio of assets when there are opportunities for multiple compounds to be developed further but limited research and development funds dictate that only a limited number of such opportunities can be supported. And the trade-off considerations also at the portfolio level are then again similar as at the compound.
08:08
Yeah, so then it would be more kind of evaluating different strategies on a portfolio level rather than on an individual compound level. That’s correct. And the fundamental considerations, however, are largely similar. And where do you get all the data for all these kind of different things? What are the data sources that would typically be used?
08:37
The data sources are varied. So they come, some of the, some of the information, the data that’s needed to form decisions come from other functions like project management, which can inform for instance, the costs associated with a particular development plan, the timelines associated with a development plan. Previous data generated on the compound in terms of efficacy and safety can inform
09:07
things like quantitative measures of the likelihood of success of a particular trial or clinical development plan. And then finally, finance can weigh in on the projected value that a particular data package should a particular trial be positive can deliver. So various functions come together to deliver the…
09:34
the information needed for a comprehensive decision analysis. Some of it admittedly involves educated guesses. There’s a lot of uncertainty that one has to deal with and that too is very much a central element of decision analysis. And I think it’s pretty easy for a clinical statistician to come up with a
10:03
standard error or standard deviation around an estimate for the efficacy if you already have some kind of phase two data or something like this or if you have some maybe comparator data. How do you come up with something like this for kind of the predicted value of a drug or something like this?
10:30
arguably a more challenging calculation. It involves assumptions, it involves a lot of market research. Not all of that research is quantitative. There are financial models, forecasting models, that are leveraged to establish quantitative values of measure and those models admittedly have been criticized for being in many cases inaccurate.
11:00
of all the metrics that we’ve discussed, those being again probability of technical success, timelines, cost and financial return. The financial return is arguably the most difficult to assess accurately and for that reason can be taken into account in the decision analysis in
11:24
you know, in more of a directional or a discrete manner as opposed to a specific quantitative value. We just have to recognize the limitations that we have in estimates that our other functional experts provide and appropriately reflect the uncertainty in those estimates. As an example, perhaps less so at a compound level, but if trying to balance
11:53
let’s say cost and value across a number of compounds in an entire portfolio, one might rather prefer to categorize value into three discrete groups, high, moderate, and low, versus using specific numerical estimates of net present value for each. Okay. Okay. So that I think describes very nicely kind of the
12:20
place of data science. Now you have been working more on the clinical side before you joined that group. So what was kind of your thinking when you entered that group? Yeah, my thinking was certainly that the technical and methodological skill set that’s needed to succeed in the decision sciences role.
12:47
I thought that that skill set comes naturally to me as a statistician. In many ways, that was true. But the so-called softer skills, for example, how to lead a team through a decision analysis, how to effectively interact with a challenging decision maker, how to influence a decision board who may have significant biases to overcome. Those were the kinds of challenges that were much more difficult for me to acquire.
13:17
Beyond that, stepping in to consult with a team facing a tough decision I found to be much more difficult than I thought. From the perspective of the volume of background information that’s needed in order to understand the problem well enough to be able to consult effectively, in working with the team and in…
13:45
presenting a recommendation, for instance, to a governance committee, too, my impression was that it’s very difficult to establish credibility because you’re viewed as an outsider. You’re viewed as a consultant coming in to analyze a situation and to provide a recommendation. And there is, I will say, in some cases even a certain antagonism or at least a certain suspicion that’s brought towards one as a consultant.
14:13
and to establish credibility in that context is harder than I thought it would be. Okay, and so can you give an example of what kind of misperceptions people had about your biases? I don’t know necessarily that there were preconceived notions about…
14:43
about my biases, I think there was more the sense that…
14:53
that it was not so easy to accept the recommendations that I was sometimes making, again because of this perception that I really didn’t understand the problem well enough or understand the context well enough. And part of that was also, however, driven, I would say, by the individual motivations of the committee being presented to. So as an example. Okay.
15:23
When presenting on the current status of a portfolio of drug compounds, there are going to be some features that are attractive and some features where the recommendation will be to take some kind of intervention because of some imbalance or some sub-optimality that exists within that portfolio. The compounds that are affected by…
15:49
you know, the compounds where a recommendation is being made to make a certain change are represented, you know, in the decision committee by representatives, vice presidents responsible for those compounds. When it’s a difficult message to accept when somebody makes a recommendation for a change that you as the vice president for that compound in question may not agree with, then it’s a very easy and natural reaction to say,
16:18
this person doesn’t really know what he or she is talking about because there are other, there is additional context about this compound that the individual doesn’t understand because the individual isn’t the expert on the compound. Oh yeah. Yeah. So that kind of dynamic one has to be aware that those kinds of biases that perhaps sense of antagonism is naturally present in an interaction like that. One of the best pieces of advice I received from a senior executive.
16:47
within my organization is that when I present to a committee, to a group of stakeholders about a portfolio, always remember that there are winners and losers in that group given the data that you’re presenting and you have to be sensitive to them all. And the message has to be delivered in a way that people can accept it, even those for whom it will be challenging to hear.
17:16
Sensitive to all. So I can understand that it’s kind of you need to be sensitive to those that represent molecules that might be cut and where you know, maybe even if you know decision might be to stop the development altogether, but in which way sensitive to to the winners so to say Well, the winners are the ones that are that are of course easy easy to
17:46
to deal with. I think portfolio management is always a question of balance. What is, and in a portfolio by nature is always in flux. The individuals who may be winning in a certain point in time because portfolio decisions or I will say,
18:15
The movement in the portfolio has played out in such a way that particular therapeutic areas are looking very good. Those individuals responsible for those areas can’t then just sit back and relax and say, our job is done. We have contributed to a healthy portfolio balance and don’t need to do anymore. It requires active management on the part of all stakeholders throughout.
18:40
And equally so those responsible for parts of the portfolio where some critical intervention is required in order to optimize the portfolio again are also simply doing their job to achieve that optimal status. Ultimately, all stakeholders are working towards that. But in any given point in time, I think that sensitivity particularly to those…
19:07
to whom challenging messages are directed is clearly more important. So you overcome these kind of challenges that you described earlier in kind of managing these groups, presenting it well.
19:24
What were kind of the most helpful techniques that you used to manage that? There certainly are techniques around presentation skills and knowing your audience. Knowing, one of the most important things is knowing who is in the room when you’re making a presentation, making a recommendation, understanding.
19:52
who is in the room, what their roles are, and what their individual motivations and needs are. That has been one of the most helpful recognitions for me. Understanding what everyone’s needs are, and as much as possible, again, being sensitive to those needs within the bounds that are possible. That’s one thing. The other thing that is really helpful is, and requires a lot of effort, is understanding the context
20:22
very well. If consulting with the team and providing a recommendation to team leadership about a particular development, understanding the context of the molecule in question very well is critically important. When presenting to a governance committee on a portfolio question, really knowing that portfolio very well and even knowing details about individual compounds within that portfolio is critically important.
20:50
I have found, made the profound experience that the degree of nervousness I have getting up in front of a committee is very directly correlated to how well I understand the subject matter, how well I understand not only what I’m talking about but also the broader context within which I’m making the presentation so that if somebody asks me about some detail that I don’t have to say, I’m sorry, I don’t know that.
21:19
So a depth of understanding coupled with an understanding of the needs of the listeners is I think are the two pieces that are most critical. So Did you rehearse these kind of things beforehand? So if you had a really high-stake presentation? I did rehearse presentations, but that can also be a trap I found.
21:45
Initially, early in my career in decision sciences, I rehearsed a great deal to the point where I had a very polished speech almost that I could deliver that sounded very good, that sounded logical and flawless. But if I was asked a question that was a little bit off topic.
22:08
uh… i a i felt uh… uneasy and uncomfortable and often unable to answer adequately very quickly so i found that in in terms of preparing for an important stakeholder meeting or governance presentation is actually more important to spend time not so much on rehearsing the actual presentation and trying to almost memorize the words it was much more important know the context know the content as well as the context very well
22:39
the specific words on the day will come and the confidence will come as well. So, later on in more recent times, I’ve spent much less time on trying to focus on specific details of the delivery and more time on simply focusing on the actual content in the message. Yeah, and preparing really, having the audience in mind and maybe for…
23:08
foreseeing questions that would come from them. Yeah, yeah, yeah, yeah. So given these kind of experiences, I see a lot of kind of similarities actually with, you know, the decision in clinical development. Maybe sometimes the stakes are not as high as at the portfolio level. But…
23:37
Where would you see the biggest differences in terms of your role as a decision and decision science rather than in clinical development? I would say that clinical development is much more focused on trial design and for instance statistical inference for the purpose of treatment effect estimation.
24:01
and less so on comprehensive analysis of a set of data to drive a logical choice. Some of the broad tools and techniques required for both kinds of activities are similar, but the objectives are different and the specific applications are different. Nonetheless, an important area of overlap lies in the development of clinical plans.
24:32
when teams face the choice of which of, let’s say, two or more potential plans to move forward with. In making that choice, the clinical development statistician is very much a central figure, and in that instance, is in fact helping the team make an important decision. So that would be an area of overlap. Yeah. So when you actually then went back…
24:59
from decision science back into clinical development now. How has your experience at decision science kind of shaped how you do your job now? That’s a good question. When working as part of a team now, I am much more aware of the motivations and also the biases, whether conscious or unconscious, that team members have toward their compounds.
25:28
Now, while the kind of passion and commitment that I see most team members have to the project they’re working on is very important to a degree, it can also negatively impact objectivity. So as an example, I try to remind team members when needed that when preparing for a governance visit to request funding for a particular project, let’s say…
25:56
The primary objective of that governance committee is to make the decision that is best for the portfolio, which may not necessarily be best for the team’s compound. And I have been in a situation, just to give an example, where the medical leadership of a compound has strongly advocated for a particular development plan that really had a very weak business case with respect to opportunities on other compounds
26:25
that the governance committee was interested in supporting. It was clear to me that from the compounds perspective, starting the clinical trial that was being proposed would have made sense. It would have offered the opportunity for generating positive data for this compound, even though the risks were high. From a portfolio perspective, however, that clinical development plan made absolutely no sense. And
26:53
And yet there was a very strong push on the part of medical leadership of the team to move forward with this proposal and to bring it to the governance committee. So that that is a challenging situation and I have been I’ve had opportunities to. To try to impart a little bit more objectivity into teams and to you know, to provide that reminder that that we can we have to be very objective in the way we.
27:22
present our plans in the data that supports those plans, but also realize quickly when it becomes apparent that relative to other portfolio opportunities, those plans really don’t make sense. Yeah. Yeah, I think it’s, I can clearly understand that bias. So if you work for, often for many years on a compound, it’s very easy to say this compound becomes your baby.
27:52
and that you want to kind of protect it from everything that is bad for it and you want to kind of, you know, have it grow and don’t let anybody say anything worse about it or bad about it. Yes, and it’s a necessary tension.
28:17
And that what you just described, Alexander, is probably something that we should all, as statisticians, be very keenly aware of. Because the paradox is, in drug development, you need that kind of tension. You need team members who are passionate advocates for their compounds, because we know that in drug development, often there are difficult choices, and sometimes even successful compounds go through periods in their development.
28:45
when the data really doesn’t look very encouraging, and yet there is a legitimate reason to move on. And that requires the passion of teams. But that has to be balanced with passion that goes beyond what is justifiably rational. And that tipping point is not always entirely clear. And so we talk of something that we call dispassionate passion. And achieving that balance though is very difficult.
29:15
Yeah, yeah. Yeah, I once heard a quote from someone, someone, most senior researchers told me, never fall in love with your compound. Yes, that’s right. Just because it makes you blind and, and, yeah, maybe too, too, too passionate and too, too biased. Yeah, yeah. So.
29:48
Given all your learnings from now decision science, what would you encourage people to learn to be more impactful in that day-to-day job as a statistician? What would be kind of key recommendations that you would make?
30:11
A few things to that question. First and most fundamentally, and I think this would go not only for statisticians, but really across functions. It is so important and valuable to understand our team members and our team leadership. Get to know your colleagues very well. Understand their motivations. Understand their needs. And…
30:41
in the context in which you conduct your work.
30:49
on, drug development is fundamentally a collaborative effort and to the extent that we can, and we rely on each other and each other’s expertise to drive a project forward. So to the extent that we can understand each other better and again be very cognizant of our motivations, it can make working together a lot more efficient and transparent and easier.
31:15
And then as I mentioned, there is this broader context. Really understand that context. Be aware of the external landscape and the business priorities that your assigned tasks have. We don’t always work on the highest priority projects within a given organization. And that’s fine.
31:44
One can have a very rewarding career working at times on things that are very important and at other times things that are not so important, but understanding the business priority can help guide the specific efforts that are most impactful in any given situation. So I think that’s very critical. All of that, the broader context, the business priority, the external landscape,
32:09
can be boiled down to what I would just call this general sense of business acumen. If we can conduct our particular assigned tasks within a context of really understanding what the business is trying to achieve and how the business operates will make us most effective. In that case, then we can look at our project and we don’t have to wait for somebody to tell us what is the next step that we need to do.
32:38
we understand what is that next step because we understand the development path, the factors that impact the development path and what the business requires.
32:52
Very good. That was a very, very nice ending. So thanks so much for this very, very nice interview. I think it highlighted that there is much more to a statistician to be effective than just doing stats. And that I think the people side of things is a huge driver and…
33:22
Yeah, also I think the learnings about kind of how to present to a high level committee is very, very important and knowing all these kind of different things. I’m just thinking about it’s kind of probably similar as if you present at an advisory board or something like this that you know.
33:50
what are the different people involved there? Some of these principles are very broadly applicable, you’re right. And then, you know, they are principles that really go beyond the discussion of a particular role within decision sciences or clinical statistics. As you know, a good colleague of mine and former supervisor
34:17
likes to say, and I’ve used this expression so many times, I really ought to give him some royalties, that we become over time less a statistician and more an excellent drug developer with a statistical toolkit. And that difference may sound subtle, but I think it’s very important. Yep. Completely agree to that. Okay. Thanks a lot. Thank you, Alexander. And it was good to talk to you, as always.
34:48
This show was created in association with PSI. Thanks for listening. Please visit thee to find the show notes and learn more about our podcast to boost your career as a statistician in the health sector. If you enjoyed the show, please tell your colleagues about it. And if you’re interested in leadership, don’t forget to leave your email address at thee slash comps.
Join The Effective Statistician LinkedIn group
This group was set up to help each other to become more effective statisticians. We’ll run challenges in this group, e.g. around writing abstracts for conferences or other projects. I’ll also post into this group further content.
I want to help the community of statisticians, data scientists, programmers and other quantitative scientists to be more influential, innovative, and effective. I believe that as a community we can help our research, our regulatory and payer systems, and ultimately physicians and patients take better decisions based on better evidence.
I work to achieve a future in which everyone can access the right evidence in the right format at the right time to make sound decisions.
When my kids are sick, I want to have good evidence to discuss with the physician about the different therapy choices.
When my mother is sick, I want her to understand the evidence and being able to understand it.
When I get sick, I want to find evidence that I can trust and that helps me to have meaningful discussions with my healthcare professionals.
I want to live in a world, where the media reports correctly about medical evidence and in which society distinguishes between fake evidence and real evidence.
Let’s work together to achieve this.
