Problems for statisticians to take on

Interview with Stephen Pyke

Many statisticians stay in the statistics department for their entire career. But not Stephen Pyke. After climbing the corporate ladder in statistics very high at GSK, his career brought him to various parts of the pharma world – and his new job is a combination of lots of aspects – including statistics and data. 

Stephen shared some important learnings from his career during the PSI strategy day in 2019. In today’s episode, Stephen and I discuss the main points of his presentation and the take-away learnings for every statistician – even if you don’t work in a large pharma company. We specifically cover following questions:

• What are the unique skills of statisticians – beyond just knowing about stats – compared to many other functions in the pharma world?
• Which problems do statisticians need to take care of beyond the current regular work?
• How can we make time to tackle these problems?
• What skills do statisticians need to improve on to successfully tackle these problems?

Through these questions we touch on the following topics:

• how a statistician can have a career outside of statistics
• how people respond to incentives
• The main job in clinical development: minimize value destruction
• Addressing the challenges of low productivity
• Problems that needs more attention
  • Time – Study enrollment time is often an important source of uncertainty in project plans
  • Quality – Detecting site quality issues quickly is critical
  • Cost – Trial costs are more likely to increase than reduce
• Statistics at the interface – Partnering with other disciplines
  

Stephen Pyke

SVP, Development DDA
R&D Site Lead, Stockley Park

Steve is SVP, Development DDA (Digital, Data & Analytics), a newly created role with accountability to deliver enhanced productivity and efficiency across Development by leveraging technology, data and advanced analytics.  Steve trained as a statistician, first joining the pharmaceutical industry in 1996, since when he has held various leadership roles, mainly in Clinical Development.  Steve began his career in academia (NIMR, LSHTM) where he held research and teaching positions.  He has served in a number of honorary professional positions, including: Chair, PSI (2009-11); Vice President, RSS (2013-2016); Chair, CDISC (2018); Advisory Board, Mathematics in Education & Industry (2017-present).  Steve studied Mathematics and Statistics at, respectively, University of York and Imperial College, London.

Listen to this episode, and share it with others who might learn from it!

Transcript:

Alexander: You’re listening to The Effective Statistician podcast, a weekly podcast with Alexander Schacht and Benjamin Piske. Designed to help you reach your potential, lead great signs and serve patients without becoming overwhelmed by work. Today. I’m talking with Steve Pyke about problems for Statisticians to take on.

Okay, this was a really, really nice interview and I hope you will enjoy it as much as I did. It’s based on a presentation that he gave at a PSI strategy day in 2019 and that really set a really good tone for the strategy day and for the board meeting that happened the day after. And this was also the origin, so to say, as a new vision for PSI. 

So if you don’t know about Steve Pyke, then you will learn about him in just a couple of minutes because he’s actually quite a successful statistician I can say. And probably most people will agree with that, and they listen to his amazing career and how he as a statistician got inside until many other areas, and can help you to understand what are all the other tasks that you can have an impact in that you may not think about.

So, please enjoy the podcast, and if you do, please tell your colleagues about it. That would be really, really wonderful. Because we need more people to benefit from this podcast and we need more people to grow their leadership skills, learn more about statistics, learn more about the environment and be more effective at work. So the podcast is produced in association with PSI, a community dedicated to leading, promoting the use of Statistics in the healthcare industry for the benefit of patients. Join PSI today to further develop your statistical capabilities with access to the video on demand, Content Library, free registration to all PSI webinars, all of them and much much more. So, the reduced rate is just 20 pounds per year for non high-income countries and just 95 pounds for high-income countries. Just go to the psiweb.org and there you see all the activities, and you can also join.

Welcome to another episode of the effective statistician. And today, I’m really proud to have a very prominent guest here, who will basically speak about a presentation Steve gave at the PSI strategy day in 2019 and for the very thought-provoking and triggered a lot of discussions, and setups the attendees of the strategy day to think more broadly, more future-oriented and then how to best set up PSI for the next year’s. And that was also the basis for the new PSI purpose and then the Strategic objectives that we recently also shared more broadly. So I’m really happy to have Steve here on the call. Hi, Steve, how are you doing? 

Steve: Good morning Alexander, very well, thank you. It’s a great pleasure to have the chance to talk with you today. And you reminded me of that session I had with the board not so very long ago. And nice to be reconnected with PSI after a little period of absence because obviously, as you discovered I’ve been doing some other things for the last couple of years.

Alexander: Yeah, maybe we can dive a little bit into that because as a Statistician, you have a pretty broad career looking into lots of lots of different things you started. In Academia and then moved into Pharma, working in Stats and Data Management. Such a kind of typical thing. And You also went into a couple of other areas into which you have overseen, much more than just regular statisticians’ side. 

Steve: Yeah. I mean, I think I’ve been very fortunate in my career to have managers who were willing to take a chance on me. I trained as a statistician, there was nothing in my training that suggested I could do, to have a go at. But I guess some people see things in you and they think you can take on new challenges and that’s why I have the chance to do so. Yeah, I went initially to a more sort of broader scientific role where I was overseeing not only statistics but also some other of the scientific discipline. So Pharmacology, Computational Biology, Genetics, which was a real stretch for me, but incredibly exciting. Epidemiology, which was a little closer to home. So I had this sort of cluster of sort of related but very distinct scientific disciplines, which I was overseeing. And it was a real education and of course what it gave me a huge opportunity to do was get much more exposure into the sort of research and Discovery end of the business, through some of those groups. And then more recently and I suppose what I focused on particularly when I met with the board, clinical operations. So the last couple of years, when I’ve spent my time doing, is running a sort of fairly typical clinical operations group. So clinical trials, phases 1 – 4, data management, project management, clinical in-country staff, the CRAs and so on. So again, not obviously closely related to my statistics training, but actually surprisingly there were some things that really were helpful for me in terms of what was done previously. So, yeah, I had a really enjoyable career. 

Alexander: So, when you were leading all these different quantitative functions, like Epidemiology, and Genetics and so on. What’s this now all called Data Scientists in a new world?

Steve: This is sort of an interesting and somewhat provocative question. We are all data scientists, statisticians included these days. I think that’s true, actually not only of the disciplines which would probably self-describe as Quantitative Scientists, but also probably of the clinical operations side. That data now has to be seen as integral to the way that we think about doing our job. And the way that we can and should draw on data in order to make sensible choices to make the decisions more often is obviously key. I think the statisticians, you know, the sort of discussion here is about whether this is anything more than just, what’s in a name, I mean, is there any substantive difference? I think there has been. I think the gap is closing and I do think, you know, when I think about my time  in Pharma and you know more recently than me Alexander. But my sense was, it took a little bit of time for the Statisticians in Pharma to catch up to the idea that things have moved on. And that’s where the data scientist, I think wants some challenge to say actually these much bigger datasets are new ways of tackling problems. The Pharma Statisticians need to sort of take note. 

So that was quite what was behind your question. But it would be something that I would say. In terms of this whole issue of data science, really anything different or no. 

Alexander: Well there’s no right or wrong. It’s just interesting to see all the different perspectives on it. So just recently, You have taken on a new challenge within your organization, and maybe you can talk a little bit about that role. 

Steve: Yeah, just just actually since the holidays, I’ve come back into a role as head of Digital Data and Analytics for the Development Organization. So again another evolution in my role and is somewhat related to what, you know, the previous question You asked me this sort of notion today. If Pharma has any desire to be successful, it absolutely has to embrace the sort of wave of technological innovation that’s coming through the use of very large data assets, which you don’t necessarily have inside your own company’s walls. You need to license from other data generators and data providers. And then the analytics, the visual tools and many of these will be very visual but the sort of analytic tools, which go with them, whether that’s in the form of ways to monitor, oversee dashboard type tools or whether it’s tools that allow us to measure in ways that we couldn’t measure previously. And I know the audience will be very wearables familiar with apps and all the rest of it. 

So, I think what my boss has seen is if we leave this to individual enthusiasts or even the functional groups, what you end up with is a little bit of a disconnected, non sort of coherent series of Investments and in a world where financial resources are as constrained as any other. I think my role is to help the business make the priority calls and when it makes those calls to ensure that there’s a degree of cohesion in the Collective investment plan that we bring together. So, yeah. I must say I’m very excited and, you know, I am less than two weeks into that role and really trying to integrate in my own mind. Exactly what I need to do to help the business be successful. 

Alexander: So in your title is that you mentioned digital, and depending on the people, you talk to that has very different connotations. So for example, when you talk to people that’s in marketing, digital is all about, let’s say digital interactions with customers like emails and things like that, but also storing all this information centrally and then, being able to analyze it, visualize it and and improve on your marketing and sales efforts. Is digital in the development organization something similar? 

Steve: These words, like the data science label, are sort of used in ways which have different meanings depending on the context. I think, for me, it’s this idea that the world around us has changed in ways which are, when you’re looking back over the last five years, incredibly dramatic, in our personal lives. Ubiquity of smartphones and all the apps that come through those. And the things they are now allowing us to do, the desktops that we have. And my endeavors to become paper free and truly digital. You know I had lessons, advanced lessons in OneNote the other day and the way that allows us to integrate different aspects of the way we work in the office. Incredibly valuable, when you think about the ways that we interact then, beyond the office desk, and into the outside world with patients, with Healthcare Providers, with audiences, beyond the digital, in the sense, becomes the medium by which we are able to do all of that using different platforms that are in their root.

In some sense, an integration of capabilities in a way that I don’t think we’ve ever seen historically. So for most of my time, for all of my time in the industry. Digital, if it’s meant anything has been, is a tool. And here’s another tool and here’s a device, but honestly, they’re not connected in any way. so when i think about digital is that word integration actually comes forward more than any other. And the idea that what I’m doing can be integrated with what other people are doing, but we can share that we can communicate and all of that can happen in a very facile way. And actually yeah, I think the challenge for those particularly those of us of a certain age who have grown up with that very sort of fractionated way of working, and a very sort of paper-based way of working is to say that that can no longer be considered an appropriate way to go. We’ve got to engage, we’ve got to embrace, and we’ve got to understand that all of this stuff already exists and we’ve got to get out there and use it to figure out a good way to use it. 

Alexander: So it’s very much about having good standards for how we capture and how we store data. Having a kind of finance for how we avoid all these isolated initiative silos within everything. For example, I’m just thinking about things like resource planning on something like this. Where I sometimes see that this is done differently in every department, maybe even every group, but basically the challenge is always the same. You have some people in your group and you want to plan, is that enough for the next year? Is that too much? Or what will that look like? And based on the projects that you have anticipated there are gaps there.

Steve: It’s an interesting example you gave, that you’re sort of two things which might be connected. Resource planning is always a challenging area and I’m not sure that’s wholey solved by digital, perfectly honest for sure. If we are smart about the way we build our tools and dashboards and if we do it in a way, which is where there is genuinely one source of the truth and everything is integrated and we’re all looking at the same data at the same time, whatever our role wherever we are in the business and it’s sort of real time, then that allows you to do things, that historically have been more difficult. And to some degree, we can think about how to act to optimize our investment in resources and so on.

However, I think when it comes to resource planning in particular, you know, what has to accept, there’s a little bit of art to this as much as there is data in science in the sense that, if I ask you how many statisticians does it take to design and deliver a protocol? You’ll say to me. “Well, it sort of depends, show me the protocol and I’ll maybe be able to give you an answer”. And it depends, which I think is always going to be with us, the complexity of the protocol, the specificity, the newness of the disease area and so on and so on. So, you know, I think what digital can do is help ensure that all that can be known is known. And all that can be integrated and surfaced in front can be surfaced. There will always, in my view, be the need for management judgment and some sense of experience being brought to bear. But yeah, I think that’s the issue for me on resourcing. I think you started by asking a question that’s in order which now, I forgot what it was.

Alexander: That’s fine, I think we talked a little bit about what digital means and then the clinical development world.

Steve: Yeah. I remember what you’ve mentioned standards and I will just say a couple of words about that because I do think that, you know, it’s sometimes lost on certain audiences. That if you integrate data in a way, which is sometimes haphazard, then what you get out is haphazard and one has to be one of two things and I think both are perfectly reasonable to do. Either you need to have really high quality metadata and really good understanding of where the linkages are between your different data sets and invest time to build that and in a certain use, that’s critical, that’s essential. In other contexts, I’m thinking particularly about electronic medical records using data sets which are owned by other providers which we might wish to link. There are some pretty sophisticated statistical methods now, which allow you in a sort of sense, to make good guesses about where those linkages can come and in some sense link things where we don’t have perfect information, but where we can say something about the likelihood of error and make some sensible choices about how best to link things. So, you know, I think one has to be mindful that might still garbage in garbage out. You need to know when you’re putting sick data into a system. What’s his pedigree? What is known? What is not known? And therefore would have wanted to do as a consequence. 

Alexander: Yeah, I think it’s similar to clinical data in a sense. So if you give all the clinical data that you have to a Physician to make a treatment decision that will not automatically, give okay thats yes or no just based on the data always additional art to it to see the whole patient and and the other things that you can’t measure with the data. And I think that’s always the case also, when we look into the business data, but I think the point that you’re making is that at least we should have all the data in front of us, that is pretty easy to take to make good data-driven decisions, but obviously for nearly all decisions that will be some art to it and not just the signs.

Steve: And you remind me of one of my sort of themes when I met with the board. Which I mean, just to say a couple of words about it. It’s one thing to have data and evidence. It’s another thing to convince the people who are the decision-makers to act on it. I mean, we see this in our everyday lives all the time and I shared some Book, I’d read over the holidays. But I think this whole notion that if only we had the data, then we would act differently, we would act more sensibly that we’ve made better decisions. It’s not true. And I think if you know, one has to recognize that there is a change element here that there is an influencing element here in some sense. Particularly, if you know if we’re talking to an audience of Statisticians, you will know yourself, you’ve got to persuade the organization that what you see and therefore the action that you advise is the way to go. And so understanding, how to explain what the data is telling you? How to show that audience what it means? And then to ensure that the decisions that flow from it, are in some sense, the decisions, optimal or whatever. You know, what? Don’t let’s not overlook that aspect of what we’re talking about here. It is about evidence, it is about visualizing and servicing, it is about integration, but it’s about some other things too, human behaviour, culture. 

Alexander: Yeah. I think it’s just not enough to be right. That doesn’t mean you get to the conclusion, it is the decision. If you think about these different skills that the Statisticians need to be successful in that regard. We’ll talk a little bit later about the different problems with that, the bigger problem that Statisticians should tackle, but let’s pause for a minute and think about what other skills that Statisticians really need to improve on, to influence decisions in the right way and also have a have a seat at the table when the decisions are made and not just being the report providers at the same time excluded from all the discussions.

Steve: Yeah, it’s a really important question, it seems to me and I take it as given that technically statisticians are up to the job. They know what they are doing. I mean, I’m not even going to talk about that because I know that there’s some very good people, doing very nice work and that If you join the industry as a statistician you get incredibly strong training and so on. So I think the technical pieces are given, although, you know, it would be interesting to ask whether, and to what extent all of the people listening in today are embracing some of the new methods.

But anyway, let’s park that and leave that for others to reflect on, for me then, I’ve had the real privilege to sit on a number of senior governance boards in my organization and therefore what I’ve had routinely sometimes or multiple times a week is the opportunity to hear from Project Teams, including the Statisticians about their projects, about what they’re proposing about the investments that are going to be necessary to deliberate and the choices they’re making and why? And so on. And some of those have been outstanding and have carried through the governance board in a very compelling way, and they’ve got exactly what they asked for when others are left out. So, I think what it requires are probably two things. One is the ability to articulate in a way which is understood by us or a non-technical audience. I mean, scientifically they’re familiar with clinical protocols. They understand clinical development, but they’re not statisticians.

So the ability to communicate sometimes quite complex concepts, sometimes quite complex statistical analysis of data to be able to articulate and communicate and explain what that’s telling me. In ways which are graphs by the audience and then import that data to the argument that’s being made. I mean, it’s incredibly important. And by the way, there’s a sort of, sort of an understanding I hope so, you know, for a Statistician goes to governance board meetings, they’re not there just to sit and listen, they speak. So you do it? Perhaps I should have begun there, of course that they will contribute, not just listen. But they will contribute in a way which is grasped, and powerfully understood by the audience. So that’s key I think. 

But I think the other piece for me is, for them to go in with an absolute understanding of the criticality of the role that they play in the decision making process. And the idea that things like the probability of success, for example, a very sort of simple idea but incredibly important when we’re making investment decisions, if you’re being asked as a governance board to spend a hundred million pounds on a program, you want to be confident that the end of there is some chance that you understand at the beginning that you will be successful. And there is some chance that you will have data which will allow you to make a clear decision and not end up still struggling to understand what the next step should be. So this whole concept of all right. Well, how does this next investment fit into the overall scheme of things? How does it advance the base of knowledge? And how does it allow us then to be in a position where we can with greater certainty and greater confidence? Decide to progress or not progress in light of the data that’s in our hand. In other words, you know, dealing with uncertainty which is absolutely stock in trade for statisticians, how to help the organization. We feel comfortable that we are dealing with that uncertainty in the most efficient way. So, you know, there’s a sort of I mean you could argue that sort of a technical part of what statisticians are doing, but it’s a very particularly important piece of what statisticians need to be able to do in a governance board setting when investments are being discussed. 

Alexander: So it’s basically first coming into the meeting with a mindset of being an owner, not just a consultant.

Steve: Absolutely. 

Alexander: And it’s coming into the meeting knowing what the audience needs to make a decision. Knowing what I’ll see, kind of a business environment. What are their goals? What are the constraints that people work on? What are the most important priorities and then also be able to listen to that and then be able to communicate back in a way that addresses these concerns and sees these priorities, these simple plain language not a technical language. 

Steve: It’s exactly right. And I think the head of our company now puts it really nicely, he talks about smart risk taking. And he has this sort of concept of being willing to take risks and fundamentally drug development is all about taking risks. He goes on to point out and I think it’s rather well done. That we could choose never to invest and that would be the decision most of the time. But of course, it would cause us to fail. We are, you know, and we sort of understand this and yet we often overlook it that we understand that we have to take risks. And we understand that most of it will fail. And that’s okay, that’s the nature of what we do. But that’s why the role of the Statistician is so critical because they are in position to help us understand the nature of those risks and how those risks are then discharged over time by new pieces of evidence which we bring through our clinical studies. I mean the more you think about it, the more powerfully important that sort of observation really is. Simple observation is smart, risk-taking requires a really clear understanding of where the risks arise and how they get this job. 

Alexander: Yeah. Okay, as we are talking now about, these problems you mentioned are a couple of problems that the Statistician should focus on in the Pharma world beyond the typical things that we mostly focus on, like yeah making studies and making maybe also project plans for complete compounds, and then how we develop compounds or you know also doing faith beyond or before the clinical development, but I think there’s lot of other things where statisticians can have a bigger impact and especially with your background in clinical operations. You talked about a couple of these. What are these from your point of view? 

Steve: Yeah, I mean, I’ll start with one which actually my new new role is still very much on my mind and it’s around enrollment of patients into our clinical trials. If you think about how long it takes to conduct a clinical trial, you know, there’s the protocol writing phase, there’s the initiation phase where we’re getting the center set up, later on there’s the treatment phase and then there’s the closeout phase, where we’re cleaning the data and then reporting trays in the medical writing and to some extent, most of that you can plan it pretty well, but if you look at the data, what you find is that the phase, which is least reliable, in terms of its duration is patient enrollment phase.

And in particular, many of our trials, not all of them. Some of them go very much to plan. But too many of them don’t if we’re off plan, which is very rare because we recruit much faster than we expected. It’s always, always be recursive because we recruit much slower than we’d anticipated. And moreover, even when we’re recruiting to plan. And this is the sort of digital piece and the connection here, you know, I was looking at some data quite recently, in a diseased area, the data that we were projecting for our trial was not terribly different from what we saw in the competitive landscape. So I assumed it was reasonably representative and you’re talking about one to two patients per site per year. And of course, for any significantly sized study, that either means hundreds and hundreds of sites, or at least a couple of years of recruiting patients. And then, of course, you’ve got the treatment phase after you’ve gotten them recruited into the study.

It seems to me there’s quite a bottleneck here and I’m real opportunity to be addressed. And then you say, okay. So what’s the connection to statisticians? Well, of course, the connection it seems to me is that if things are going to go off plan as they often do, how do we know whether we’re on plan or off plan? You know, you may have seen, as I’ve seen recruitment curves. Okay, by this day, here’s where we’ll be able to randomize this manual screen. This many will have this, many centers open and so on. You’ve seen many of them. I’m sure I’ve seen many of them, but too many of them don’t have confidence in terms around them. Now. This is not new. It’s not a new idea, and sometimes you do see them presented and there are tools out there, which will give you these sorts of ranges around the plan. And then you add another layer on to that, which is when should we intervene? When do we conclude that this is off track? And therefore, we need to do something different, and what represents optimality in some sense, however we define it? Well, when is the moment to intervene? Early? Late? And watch it and see if it’s different for every study. But isn’t that exactly the sort of thing that statistical thinking can help us with? 

And moreover, if a lot of thinking has already been done and I think there is stuff out there which you can see in the literature and which you can see, indeed in some tools why is there still a situation where too often that’s not routinely used? So, you know, I do wonder whether without doing anything mind-blowing, cutting edge, or groundbreaking, actually there are some simple things that statisticians could be talking to their organization about, say maybe we can help. Maybe we can help you with planning enrollment, and as you know, go back to where I began, you know, if enrollment was the plan then there’s a real benefit to the wider organization because too often it’s not a plan. And of course if it goes over that’s a lot of extra money and monitoring costs and project management costs and so on. So the sums involved here can be very large.

Alexander: I think that is a general theme that I see very often about data that’s in a wider organization. That mostly people just report point estimates. It’s the mean proportion or the mean time or whatever. But it’s very rare that you get a sense of how much variability is there. And just also from a belief point of view, you know, you could ask 10 different experienced Project Managers for, okay, how many patients were your recruit per site, per year? And you will probably get 10 different answers. But what it is only taking is the average. Or the one that has most in general or whatever.

Steve: And I think this partly here is sort of sense that if I try to tell the whole story and all the uncertainty, it gets too complex to deal with, but actually again that’s where statisticians can really be helpful. Because you know, your job, our jobs when I used to do it. It’s about making sense of uncertainty as about making sense of variability and it’s about pointing out that uncertainty comes with certain patterns, you know, and when there is uncertainty, it’s actually quite predictable, what the uncertainty will look like in many settings. Therefore, the ability to put a sort of reference ranges confidence bars. However, we want to describe it around the plan. It’s not hard to understand once you see it there and then particularly if you couple it with and we’re going to make some interventions, if it starts to go below this line and or above it, indeed. And these are what those interventions might be, and it’s really helping the Wider Organization understand how what appears at first to be unmanageable complexity, actually can be very manageable and can be understood in a very sort of straightforward way. 

Alexander: It also helps to avoid lots of unnecessary discussions about taking actions when actually it’s just a little bit of random variation happening here. Just because the recruitment line is a little bit below the plant time, doesn’t mean that there’s a real deviation as well.

Steve: That’s true. Although I would say that’s less of a problem in the sense that when I think I’ve seen more often, this team says yeah, we know we’re tracking behind, but it will pick up, don’t worry. I’ve been here before, I know what happens. We never hit the plan, we always catch up or words to that effect and maybe, you know, I don’t say that experience counts for nothing. It clearly is very relevant. Although I’m bound to say, why didn’t we put that into a flap? Anyway, human nature being what it is. We think things will sort themselves out until the end.

Alexander: And that’s the other interesting piece and knowing about all these cognitive biases. Yeah, for example, nicely described in Kahneman’s book “Thinking Fast and Slow” we’re lots of these like what you just mentioned. We tend to overestimate what we can do in a short period of time, and that is true for individuals and true for teams.

So now we talked a little bit about recruitment. What other problems do you see?

Steve: Yeah, without giving away any of the confidential details, you’ll immediately understand that this relates to experiences I had in my role and it was around the other big one for me was around quality. So the quality of the data coming back from the site, the quality of the study conduct and for in operations organization, we live or die by the quality of the data we deliver. It’s not great to come in late, but it’s absolutely impermissible to come in with call quality study conducts. So that, when the inspectors go visit the site, they don’t have confidence in the data that’s been ultimately delivered. And we have a range in the operational space, we have a range of tools which have been used. But historically, we’ve relied mostly on a large army of clinical research associates, so-called study monitors, who go to the investigator side to watch over what the investigator staff, and site staff are doing and check and train. Indeed those Staffing protocols and they do a terrific job by and large. I must say, but what we still see is the quality issues do arise and, you know, to some extent, some of that is just part of what we understand, you know, it’s a bit like the noise in this. Your site from time to time and if it’s one site in 500, probably it doesn’t seriously challenge our understanding of the data, our sensors believe in the Integrity of the study. 

But clearly, you know, there’s going to come a point where the amount of poor data or the amount of questions around the quality of the conductor study becomes problematic. And you then say, so how do we know? We have some very good monitors. We have some who are less experienced. And so to monitor making sure the monitors know what they’re doing is kind of rather obvious.

Alexander: Let’s go back to the point about the monitor. So yes, having good monitors is for sure important, but where do the Statisticians come into play?

Steve: Yeah, so that’s where I was sort of getting some data itself tells us a lot, the information that we were gathering around the sites as the study is underway, should tell us a lot. So for example one can imagine that by looking at enrollment patterns we can infer whether the site needs more intensive monitoring or less intensive monitoring. 

If the site hasn’t recruited patients for quite a long time, you’re bound to ask why? And you might want to go visit on the other hand. If they recruited 50 patients in a short period, you might worry that the site will be overwhelmed by all of the activities associated with that. 

There are tools that have been available for us and many organizations in line with the regulatory agency expectations, have moved to what’s called a risk-based approach. So by looking at the data and you can pre-specify these are the things that would cause us to want to take a closer look, you can specify criteria in advance. I mentioned once that there are. You can start to build decision rules around those. 

Alexander: Yeah. Actually, I talked with Tim Rolf from JFK about risk based monitoring in an earlier episode. 

Steve: Sorry, I didn’t hear you there.

Alexander: I actually talked with Tim Rolf in an earlier episode about risk based monitoring.

Steve: And there you go. Yeah, so clearly there is some stuff out there and this is advancing all the time and we’re moving into a world of centralized monitoring now and there are again some nice tools. But there are limitations to what it could do. Those limitations, you know, are around short duration studies, there around smaller studies, and you might say, well we need those tools less there. I guess, the central point I wanted to come to nonetheless was despite these tools by the fact that they are available. We still see quality issues, and I had a situation not so very long ago. Very large, big investment, very large study, some very significant issues. And now, you know, the analysis of quite how those arose is still I think subject of some debate. But there’s that combination again of data interpretation, decision-making, and acting. It seems to me that the availability of tools, statistical in nature, has helped but it hasn’t solved the problem. And I’m left with sort of 

two questions. Really one. Do we need some more statistical insight to develop those tools further? To make them more generally applicable, to make them more easily understood, to make sure that we’re making better choices more often. And then secondly, in the interpretation of the data, those tools are using are we acting on it in the way? And again, their sense of intervention and decision-making and the sort of statistical thinking that helps us understand what we’re being told by these tools could be really powerful. 

So, you know, this is an example of an area which is incredibly important for our industry. High quality strategies are absolutely the foundation of what we do. There are tools but they haven’t solved the problem is really what I’m saying. And it seems to me, there’s more opportunity for more statistical intervention, more statistical guidance to develop the tools and then to use them appropriately. 

Alexander: So it’s not necessarily that these Statisticians need to be at the table when these tools are used but they need to look at the tools and prove them and train the users better on using some. 

Steve: It might be both. But for sure, so long as quality issues persist. I think there is going to be a need for help from the Statisticians and for sure, you know, the delivery of these tools and availability of these tools will not fix the problem. So, you know, we have to ask, what’s the missing piece here? And I think one of the missing is really thoughtful statistical input both into the further development of the tools, but also into the interrogation the tool is telling us. 

Alexander: Okay, what would it require for a Statistician to be bad acting in that area? 

Steve: Well, one thing you need to do is take notice of it. 

Alexander: Talk to your clinical operations partner and the studies. 

Steve: Yeah. I mean I can remember when I joined the industry. I knew nothing about clinical operations. And even when I stopped being a statistician, I knew almost nothing about clinical operations. I think it’s one of those areas that many Statisticians, It’s just not visible to them, they are focused on their patch and it’s actually it’s something which is quite true of the industry. As a whole, we’ve become hyper specialists, you know, we know the piece that we’re accountable for incredibly well, incredibly deeply. I wonder how many of us take a moment to step back and look, more broadly across, you know, everything that goes on in clinical development less still, what goes on in research and discovery and all the rest of it. 

Now, I understand why because we’re all incredibly busy. And what we’ve been asked to do, is challenging and time-consuming and so on, but I do think functional groups. I hope what they’re doing and I would encourage them to do it more is to look around and say there are opportunities elsewhere beyond reporting the primary and secondary efficacy outcomes of a clinical study.

Alexander: Yeah, sometimes it’s actually quite easy. You could just have, you know, a regular lunch with different people, from different departments and just ask them, what’s on your mind? What’s your biggest problem? What keeps you up at night? And these types of questions and learn through that approach much more about the bigger business and how you fit in, and by that understand where maybe much bigger problems. So you can help with rather than kind of maybe improving some footnotes in your tables. 

Steve: Yeah, I agree with you and I think it’s a particular responsibility for managers, statisticians to do exactly. Is to have a broader sort of listening ear and broader kind of overview of what’s going on in their business. I mean, just just not clinical operations, but the thing I did previous to that, you know, I spend a little bit of time and I would never claim myself to be expert, but I spend a little bit of time getting to understand clinical pharmacology and through some really helpful clinical pharmacologist, who taught me a little bit. You learn about some of the methods they’re using Okay, pk/pd models, but how does model informed drug development come to life? And the FDA is now talking about this in sort of very clear terms.

In my previous company, we’re doing this many years ago and the interface between statistics and clinical pharmacology should be a really dynamic one because there are many crossover problems. 

Alexander: Yeah.

Steve: We’re both dealing with data. It should be an integrated partnership because many of the problems need both disciplines to solve them and I’m really delighted that modeling from drug development is something which is beginning to take hold. But for me, it’s absolutely about that interface, it exemplifies perfectly what you were just saying which is, you know, have you sat down with lunch with your nearest clinical pharmacologist to understand a little bit about what they’re doing.

Alexander: In our leadership program, we had one student that triggered such discussions, went to a safety physician he was working with, and where he was regularly sending listings to and asked him, what do you do with these listings? And he said, “Basically, I look at them for a day all parking what’s happening. And then they make a decision whether we increase the dose or not.” So you’re looking into these long Excel spreadsheets with all the different values and within two hours, he programmed a nice visualization tool that he gave to the physician and the physician was super happy because he could now do this job instead of a day in just 15 minutes. 

But I think it requires stepping out of your comfort zone, sitting down with others, asking what their problems are, getting more overall business acumen. 

Steve: I think it’s a nice example you give and I’ve seen exactly that myself. There’s a sort of not quite as bad but sort of related problem, which is because we live in the silos and too many companies. We’re quite siloed in the way that we do things. You find multiple reinventions of the same solution or a similar solution to a very closely related problem. And although you might say, well, that’s a lesser problem because okay, let’s say someone in safety had generated a visual tool to look at these to replace looking at long Excel, spreadsheets and listings, it sort of solves a problem but it goes no way to address in the lack of efficiency. And so, this is where I think, you know, it’s sort of layers of problems and layers of concern but it’s almost a design problem with the way many of our companies are constructed that we’re going to have pockets of expertise. And if only they knew what the problems are with the wider business, where we solve them in a heartbeat and it’s just not happening. And Statisticians, it turns out, can solve many of the problems that are really instrumental for us.

Alexander: Yeah. So, we talk now about a couple of problems, actually at the board. We also talked some further, but this one important question that we also had at the board discussed, and you just mentioned that you know, all of us are so busy and we have many more plates that we can get to enthuse. There was a question about that organization. How can we make time to address these problems?

Steve: It’s not easy, but if you step back, I mean, I don’t know how many statisticians there are in your organization. Tell me, Ten? A Hundred? I don’t know.

Alexander: Less than a Hundred. Much smaller than JFK.

Steve: Not more than 100, okay.

Alexander: But in any organization, in an early most organization, you will have a reasonable number of Statisticians. 

Steve: Yeah. So in my company, there are many. I think you have to ask yourself. What’s the priority of the function? What’s it there to do? Now, of course, you know, if you work in a CRO the priorities are different than if you work in a Pharma company, and I totally understand that. And so the ability to make choices is to some extent constrained by the environment that you operate in. If you’re a team of one, your freedom to operate is much less than if you’re a team of hundred or with a team of two hundred. I totally get that, but if I work on the basis that, many of the statisticians in PSI work from at least medium sized organizations. If not, very large ones, and they are part of a body of a good number of statisticians and indeed related disciplines programmers, and so on. Is it really impossible to carve out a small number, or a small percentage of the effort within that organization? To ask these questions in a very deliberate way. I don’t think so. I don’t think it’s unreasonable to ask. 

We’ve just got to be determined to do it. If there’s a will to do it, a way will be found, in my view. And I think if it’s not happening and putting aside the very small enterprises to where there really is no opportunity. But if it’s not happening, I ask myself, is that because there’s really no will in reality to tackle some of these broader problems? 

Alexander: Yeah. 

Steve: So yeah, it’s not easy. Don’t get me wrong, I’m not saying it’s easy, but if you want to do it enough, you’ll find a way. 

Alexander: And I would even challenge people from an individual perspective. So even if you have a never-ending to-do list, I think that will never go away. You could work 24 hours and you’ll not get to the end of your to-do list. 

Steve: And in the end, isn’t it just about priorities. 

Alexander: Exactly. 

Steve: How do you prioritize? What you’ve got to do? We’ve all got more than we can do. We make choices. And this is where we come back to managers. Again, in the end the managers make choices that impact all the people they report, who report to them. So they multiplicative are sort of in the impact they have on the organization around them. So, it really comes down to managing your choices. How are they using the resources at their disposal? What’s important enough to allocate time to? 

Alexander: And you can always have a discussion with your manager and say, “Here I have had a discussion with the clinical operations person. And I would really like to spend two days working on that and see where we get to. And, you know, maybe delay reviewing this draft by two days and later work on it.” Maybe the manager will say yes.

Steve:  Doesn’t this take us back to some extent to where you began, which is this whole issue of, you know, the zeitgeist at the moment is Digital Data Analytics everyone’s talking about. Every chief executive worth their salt is saying, Are we investing in? Are we doing enough? Have we got enough data scientists? You know, if the statisticians can grab some of that air time, some of that conversation, there’s probably never been a better time to argue for the powerful impact. The sort of thinking you bring to bear can have on the organization around you. 

Alexander: Yeah. I completely agree. And if you can deliver on something and set an example, I think the next step is also to speak about it. Leverage, basically, your business partner that they also highlight your role in solving these problems. So that the more senior people in the organization can see, okay statisticians are not just clinical data processors, but they are critical thinkers and analytical thinkers and can bring these skills beyond clinical study work and these kinds of things. So, but I think it takes a step to have a broader business understanding like we talked about, takes a step to speak up, to go out and ask, to listen, to really understand all the different problems. But then also to sell your achievements. 

Steve: Yeah, and when we spoke a little earlier I said, look, I’m going to park the technical issue, I’m going to take it as a given and I hesitate to say anything about technical statistics these days because I’m a little rusty. But I’ve seen what statisticians do and I regularly today see what statisticians are doing in my organization. And what’s really striking is that some of the things they’re doing are quite different from the things that I was doing when I was a junior or middle seniority Statistician. There have been technical advances. Now, I won’t tell you what they are. 

You know better than me Alexander, but there have been technical advances in the Pharma industry and what’s acceptable to regulators but more broadly in terms of understanding how different statistical approaches, many of them Bayesian. How would those can bring really useful insights? And in the area that we’re talking about today, which is you know, helping statisticians and statistics have impact and influence across the Wider Organization, helping other functions do their jobs more effectively. It’s not the old stuff that’s going to really make a difference. Some of it will, but I think good new Statisticians who understand the way the industry is going the way the, the new methods, the new approaches the new tools, these really have to be embraced in order to facilitate the kind of impacts that we are talking about here. 

Alexander: Yeah. That was a really, really nice summary to summarize our discussion. Thanks so much Steve for the time. It was a pleasure to talk to you, and it was a pleasure to hear you at the PSI strategy day. 

Steve: Well, thank you Alexander for the invitation. I’ve very much enjoyed our conversation today. And yeah, I hope the listeners do too.

Alexander: Thanks so much. 

Steve: All the best, bye bye.

Alexander: This show was created in association with PSI and thanks to Reine who helped so much with the show in the background. Think I would be really worried if I didn’t have her. Thank you for listening and just go to the effectivestatistician.com, to find the show notes and learn more about other episodes that will help you to boost your career as a statistician in the health sector. And also, if you enjoyed the show, please tell your colleagues about it. Just share a link, like it on LinkedIn or whatever Social Media platform you are. And just, know, tell your colleagues about it. Some people have actually posted that in internal social media on internal news channels. That would be awesome. So, reach your potential, lead great signs and serve patients, just be an Effective Statistician.