Linear mixed models – a refresher and introduction

Linear Mixed Models – A Refresher And Introduction

[00:00:00] Alexander: Welcome to a new episode, and this is again with Paolo. Hi Paolo, how are you doing? 

[00:00:08] Paolo: I’m very good, Alexander. How are you [00:00:10] doing? 

[00:00:10] Alexander: Very good. It is actually the second episode that we recorded today, but you of course don’t know about these things because you get these [00:00:20] episodes in a different, different way but of course we record all these episodes quite ahead before we publish them.

[00:00:29] Alexander: [00:00:30] Today we want to talk about linear mixed models and this will be the first episode in a series of episodes [00:00:40] where we talk about linear models. So let’s start first a little bit with why [00:00:50] actually linear models in the first place. So when have you first get in touch with linear models?

[00:00:58] Paolo: I think linear models [00:01:00] in general size of the inclusion of random effects model, started when I was. You know at the university from my first degree because it’s kind of [00:01:10] a standard approach for the first experience with the model. But it’s simple is nice is insightful, and you can learn a lot [00:01:20] and then build on, on this.

[00:01:22] Paolo: Specifically modeled and expanded the tool to incorporate other features of the data. 

[00:01:28] Alexander: Yeah, yeah. [00:01:30] And now, when we talk about linear models, the important thing to have in mind, it is linear in the [00:01:40] coefficients that we talk about, yeah? So linear in the sense that there’s there, So, the dependent variable y [00:01:50] equals to beta times x, and x being the independent variable of variables, and its [00:02:00] linear in the beta.

[00:02:01] Alexander: Yeah. So that is, that is important to understand. It has nothing to do with whether the axes are linear. They can [00:02:10] be, you can have quadratic terms in there or logarithmic terms or interactions or all kinds of different things. The important thing, it is linear in the [00:02:20] better. Yeah. That’s just a little bit of a background.

[00:02:25] Alexander: Very often people think like, Whoa, but that curve is [00:02:30] obviously not a line. That has nothing to do with being easy. Yeah so you can have y [00:02:40] equals beta one x plus beta two x squared to beta three logarithmic of x. Yeah. This is still a linear [00:02:50] model because it’s linear and the better one, two and three.

[00:02:53] Paolo: Yeah, of course. 

[00:02:54] Alexander: So just a little bit of a side step to that. Okay. So a [00:03:00] little bit of history, where are these models coming from? 

[00:03:04] Paolo: So when we speak about linear mixed effects models I think that [00:03:10] everything started in the early 80, we, the paper in the 82 from the well known, Lerder Rubin.[00:03:20] 

[00:03:20] Paolo: And then there have been few improvements. And then everything started to grow with more [00:03:30] refinement of fine tuning more expansions on the tool. And we have a, a lot of literature that specific area? Yeah, so I think [00:03:40] that it’s quite important to note that, still you know, we, we are using these missiles from about 40 years.

[00:03:48] Paolo: But they seem relatively [00:03:50] new to a lot of people and probably also we have a lot of. People using them without knowing all the assumptions and [00:04:00] the implications there, because There are quite a lot of intricacies.

[00:04:06] Alexander: Yeah. Yeah. So, and that is the that’s the [00:04:10] first interesting thing that you talk about is this mixed models. So, what I just talked about is y equals beta x. [00:04:20] Yeah, that is the, the original model. And well, basically plus epsilon at the end, always. Yeah. And so [00:04:30] epsilon gets you the the normal variability. Yeah. So the you always condition on the X and you assume the X to be [00:04:40] constant and then the So, and then the epsilon, that is somehow disreputed.

[00:04:47] Alexander: Usually we just assume it to be [00:04:50] normally disreputed. And that is the error. Now, in reality, of course, the x, is [00:05:00] usually not completely fixed. Yeah. So the X can of course be things like a treatment group. Yeah. Then that is [00:05:10] fixed and you can actually change it and things like that. But very often these can be other factors like baseline disease severity.

[00:05:18] Alexander: And [00:05:20] this is of course also something that comes with measurement error. and all kind of other things. Yeah. So you condition on [00:05:30] it and by doing that, you, you take it as fixed, but that is also a big assumption that leads [00:05:40] to all kind of different things. So for example, if you measure something very imprecisely, that will have an [00:05:50] impact on the better and always.

[00:05:52] Alexander: Push the better closer to zero rather than away. The other thing is said of course, there’s assumptions [00:06:00] about independence here. Yeah. So we usually assume that all the different patients. are independent from each other. That is the [00:06:10] first thing. Now we can lessen this assumption by having this mixed effect.

[00:06:18] Alexander: Yeah. So we [00:06:20] basically add another gamma times z or something like this with the z being random variables. Yeah. [00:06:30] And through that, we can send model, for example, said certain patients may come from, let’s say, the same center [00:06:40] and therefore have maybe especially if you have these cluster randomized studies they come, you know, and have a co variability is a, say, they are [00:06:50] not independent of each other.

[00:06:52] Alexander: Yeah. And so there’s a, there’s a covariance matrix that is not just diagonal or you have [00:07:00] repeated measures. Yeah. So you said you have multiple measures and they all come from the same patient and then you assume some kind of you know, correlation structures [00:07:10] as well. 

[00:07:11] Paolo: Yeah. For example, when you have repeated measurement, of course the repeated measurement within each individuals are not [00:07:20] independent and you’re not interested in learning for each and every patient how much These observations are related [00:07:30] or the difference between these two observations.

[00:07:34] Paolo: So instead of modeling this as as [00:07:40] a fixed term, we, we can think about some random variability. Modeling the errors [00:07:50] within the patient, but of course, without imposing a single effect per patient, but maybe thinking of more general, [00:08:00] random effect. In which you have, a single realization per patient and you can effectively model the the [00:08:10] data respecting the nature and also estimating what’s important for you, like the treatment effect or the effect of time.

[00:08:19] Alexander: Yeah. [00:08:20] So you see, in this random effect, yeah basically see, if you think about So this is beta x plus gamma z, yeah [00:08:30] the z’s are normally distributed variables, yeah, and they all center around zero. Yeah. [00:08:40] So they don’t add any, any main effects but just add variability and you’re interested in [00:08:50] understanding that now, you know, you can have all kinds of different assumptions about that.

[00:08:55] Alexander: Yeah. So for example you can assume like, you don’t [00:09:00] know anything. About the variability. Yeah. And then you have these covariance metrics where you just assume them to be, [00:09:10] you know, covariance metrics in, in, in that sense, it’s a symmetrical and these kinds of things, but basically see degrees of freedom is here [00:09:20] said, you know, every cell has its own value and said there’s no restriction on that.

[00:09:27] Alexander: In other circumstances, of course, it [00:09:30] could make sense to use different covariance metrics, yeah? So this, and lots of, in SAS I think it’s called unstructured covariance [00:09:40] metrics. 

[00:09:40] Paolo: Yeah, the more flexible is the unstructured covariance metrics. 

[00:09:45] Alexander: That is the maximum flexibility. That, of course, you also need to [00:09:50] estimate. All these parameters. Yeah. And estimating all these parameters leads to lesser degrees of freedom. Yeah. And so more [00:10:00] instability. If you have lots of patients, well, that’s not so much of a problem. If you have less patience, then yes, that can become a problem. In [00:10:10] these cases, you can also assume other Covance metrics.

[00:10:16] Paolo: Could be something more [00:10:20] parsimonious, like assuming an autoregressive structure, for example, that you have the same covariance for, for the same lag, if, if you’re thinking about [00:10:30] repeated measurement or you can impose different constraint for example if you’re using, Unequally spaced Time points, you can assume that [00:10:40] time points equally spaced have the same covariance measure, and it’s a different if the space between the time points is different.

[00:10:49] Alexander: [00:10:50] Yeah. Another things that you might have is this compound symmetry, yeah, where all the covariances are all the same, yeah, [00:11:00] because you have this kind of symmetric approach that for example could happen if you have, if you have certain kind of biologic experiments, [00:11:10] yeah, and they all come from the same plate or something like this, yeah, so they all share the same background noise or something like this.

[00:11:19] Alexander: [00:11:20] So, these, it’s sometimes really helpful to have a look into these, to understand especially for smaller [00:11:30] sample sizes, can I make that model more effective? Yeah, by being, yeah, more restricted and not adding [00:11:40] too many degrees of freedom.