Generalised Additive Modelling

In the previous section, we discovered two main issues to address: Independence and homogeneity. Different options can be used to solve the dependence problem, namely (i) extend the linear regression model in Eq. 8.7 with more interaction terms, (ii) add more explanatory variables, or (iii) apply a transformation to linearise the relationships. We already discussed arguments against applying transformations. And all possible interactions and explanatory variables were already used in Section 8.5, however no improvements could be seen. Therefore, we move from linear regression models to smoothing techniques.

Smoothing models allow for non-linear relationships and belong to the family of generalised additive models (GAM). A detailed explanation on GAMs can be found in Hastie and Tibshirani (1990), Ruppert et al. (2003), Wood (2006), Zuur et al. (2007), Keele (2008), among others. A GAM example in forensic entomology can be found in Tarone and Foran (2008). Smoothing is a scientific field on itself, and there are many different ways of smoothing (Wood 2006). The easiest smoothing method is the moving average smoother, which calculates the average at a target value using all the observations that are in a window around this target value, and then this window is moved along the gradient, and each time an average is calculated. The other two phrases frequently seen in the smoothing literature are LOESS smoother and smoothing splines. The LOESS smoother is just an extension of the moving average in the sense that weighted linear regression is applied on all the observations in the window around the target value. The smoothing splines are a bit more difficult to explain. Basically the covariate gradient is split up in multiple bins, and on each bin a cubic polynomial is fitted. These are then glued together at the intersection points (also called knots). First order and second order derivatives are used to ensure smooth connections. From here onwards, it becomes rather technical; the smoothing splines are estimated by using a penalised sum of squares criterion which consists of a measure of fit and a penalty for the amount of non-smoothness or wiggliness. Full details can be found in Wood (2006).

Smoothing techniques have one major problem; whichever method is chosen, somehow the software needs to know the amount of smoothing. For the moving average and LOESS smoother, this means the size of the window. This is a value that you have to choose, and its effect can be between a straight line, or a curve that connects each point. Although this may sound rather subjective and scary, in practise it is not that difficult, and various tools exist to guide you, for example:

1. Use the AIC to choose the optimal amount of smoothing.

2. Decrease the amount of smoothing if there are patterns in the residuals.

3. Apply automatic smoothing selection using a tool called cross-validation. In here, observations are omitted in turn, the smoother is applied on the remaining data, the omitted data points are predicted, and prediction residuals are calculated. The value of the amount of smoothing that produces the lowest sum of squared prediction residuals is deemed as the optimal amount of smoothing, and is expressed as a number larger or equal than 0. We also called it the effective degrees of freedom (edf). A value of 1 means a straight line and 4 gives a fit similar to a third order polynomial and an edf of 10 is a highly non-linear curve.

The main advantage of a smoother is that it captures non-linear patterns in the data. So far, we have not mentioned anything about software. Although many packages can do linear regression, only a few can do GAMs. Without no doubt, S-PLUS ( and R (R Development Core Team 2008) are the best software options for GAM. We used R, which is a free statistical software package that can be downloaded from Within R, various packages (a collection of functions) exists for GAMs, and we used the mgcv package (Wood 2004, 2006). The problem with R is that it has a steep learning curve and the analyses carried in this chapter can also be applied via an graphical user interface like for example Brodgar (

The GAM applied on our data has the form:

x Amino_FZ¡ x Ethanolí + b6x Propofol¡ x Ethanol¡

+ b7x Amino_FZi x Propofol¡ x Ethanoli + f (Seriesi)+ si (8.8)

The only difference with the linear regression model in Eq. 8.7 is that the explanatory variable Larval stage is replaced by Series, and a smoothing function is used to model its effect. The explanatory variable Series has more unique values than Larval stage, and is therefore better able to capture the non-linear time effects. The smoother was estimated as a smoothing spline, and to estimate the optimal amount of smoothing, cross-validation was used.

The smoother represents the Length - Series relationship, alias the growth pattern. The GAM in Eq 8.8 contains one smoother for the time effect. This means that it assumes that all observations have the same shape for the growth pattern over time, what ever the drug treatment is. However, it may well be that a certain drug treatment, or a combination of drug treatment result in different growth rates. It is relatively easy to extend the model in (8.8) to allow for a different growth pattern for each of the eight drug treatment combinations (this would give eight smoothers, one for each treatment). Alternatively, perhaps the maggots that received the ethanol treatment have a different growth pattern compared to those who did not receive Ethanol. In this case, we should consider a GAM with two smoothers. This is the interaction equivalent in a GAM.

Hence, in this context, each treatment combination is allowed to have a different length - series relationship. Obviously, we raise the question whether we should indeed use one series smoother for each treatment combination, or whether we can replace them by one overall smoother, or try different combinations. We used the AIC to find the optimal model.

As judged by the AIC, the model with eight series smoothers (one per drug treatment combination) was not better than the model with one overall smoother, nor did any other combination of smoothers give a lower (better) AIC. Hence, drug treatment does not change the shape of growth rate, just the absolute values.

0 0

Post a comment