Fast Bayesian Inference with INLA

Update: As of March 2014 I am no longer a Research Fellow at NTNU, see here.

I am currently a research fellow and 4th year PhD candidate within the INLA group.  If you deal with Bayesian models and have never heard about INLA, I sincerely think you should spend a small portion of your time to at least know what it is. If you have heard about it before, you know how nice it is and I would like to ask you to help us spread the word. After all, this can really help some applied researches that work with time-consuming modeling tasks, such as those that involve spatial and spatio-temporal modeling.

INLA is a deterministic method and provides a faster and more accurate alternative to simulation-based MCMC schemes within the class of latent Gaussian models. I am not talking about a slightly improvement over commonly used methods. I am talking about reducing your computational time by orders of magnitude. It is not uncommon to see cases where the computational time got reduced from weeks to hours or from days to minutes.

inla_EPIL_exampleJust to give an example, the Figure above shows the comparison between the posterior marginals approximated by INLA (solid blue line) and MCMC (histograms) for some parameters of the EPIL example. I know the quality of the Figure is not great but those pictures have been computed quite a while ago and I just didn’t have the time to generate those histograms again (remember, I am a 4th year PhD candidate, thesis to finish and all, rs). The approximations returned by INLA are basically instantaneous, while MCMC only match INLA’s accuracy after 16 minutes of computation time (using JAGS). You can find the (very simple) INLA R code for this example in the INLA website.

As I said before, for large datasets and/or complex models, INLA can be orders of magnitude faster than MCMC. At this point, if you are eager to try INLA I suggest you to download and install the R package INLA and to take a look at the worked out examples in the INLA website. If you stay with me for the weeks to come I plan to write more details and useful information about INLA and its R package. If you are not familiar with the concept of latent Gaussian models, I would like to point out that the following models belong to this broad class:

  • Dynamic linear models
  • Stochastic volatility
  • Generalized linear (mixed) models
  • Generalized additive (mixed) models
  • Spline smoothing 
  • Semi-parametric regression
  • Space-varying (semi-parametric) regression models
  • Disease mapping
  • Log-Gaussian Cox-processes
  • Model-based geostatistics
  • Spatio-temporal models
  • Survival analysis
  • +++

If you have any doubts regarding INLA there is also a very helpful INLA discussion forum.

Advertisements

7 thoughts on “Fast Bayesian Inference with INLA

  1. Big fan of INLA for computing models which fit into the common examples given on the INLA site but I’m finding it difficult to reproduce some of the things that are quite easy to do in MCMC schemes such as autocorrelated errors.

    • Hi Sam,

      You are right, INLA is great as long as your model fit its restrictions. I don’t know which particular model you are referring to but sometimes there are models that INLA can fit that does a similar job. For example, sometimes you can include the autocorrelation through the latent field instead of the likelihood. One example is the use of dynamic models to fit time-series data.

  2. Yes, sometimes the model has to be ‘rewritten’ to cast in into the inla-form, like Thiago says. A small example may be helpful; let
    y = intercept + beta* x + v
    where v is ‘autocorrelated errors’, We can rewrite this as
    y = intercept + beta*x + v + w
    where w are iid errors with high fixed precision. In this new representation, ‘v’ is moved into the latent field, so the formula could be
    formula = y ~ 1 + x + f(v, model=”ar1″)
    and
    r = inla(formula,
    data=…,
    family = “gaussian”,
    control.family = list(
    hyper = list(
    prec = list(
    initial = 10,
    fixed = TRUE))))

    which fix the precision of ‘w’ to exp(10).

  3. Do you know of any astronomers/cosmologists using INLA yet? There’s a huge amount of work done in cosmology to map large scale structure and the cosmic microwave background (via latent Gaussian models) into order to recover the covariance matrix hyperparameters of these fields. But everything I’ve seen so far is MCMC or a physics-y version of variational Bayes. Perhaps you could send an emissary to the kingdom of astrostatistics to spread the word at next year’s IAUS306 ( http://sccc21.sim.ul.pt )?

    • That is a good question Ewan. I don’t know about a specific astronomers/cosmologists using INLA. I was recently at a conference where an astronomer was solving his big latent Gaussian model with MCMC (took ages to run) and after his talk I pointed out that he could use INLA to speed up his computations.

      However, I am not sure if he actually took my advice. My experience is that a lot of people feel comfortable right now with MCMC and have a significant resistance to try anything else that deviates from that. I will take your suggestion about IAUS306 to my group and see what happens.

    • Hi Mahasen,

      I am not sure what exactly do you mean by semi-parametric model count data, but this example

      # with smooth covariate
      formula3 = Y ~ f(region.struct,model="besag",graph.file=g) +
      f(region,model="iid") + f(x, model="rw2")
      result3 = inla(formula3,family="poisson",data=Germany,E=E)

      that can be found in http://www.r-inla.org/examples/volume-1/code-for-bym-example illustrates a count model with an independent effect, a spatial effect as well as a non-linear effect (semi-parametric?) on the covariate x.

      Does it help you?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s