# Automatic Differentiation in R

09 Jan 2017Automatic Differentiation (AD) specifically reverse-mode AD takes a function, defined by a computer program and computes the gradient exactly (up to machine precision) and at the same time complexity as the original program.

Here I will show how one can use the Reverse-Mode Automatic Differentiation Library provided by Stan Math Library in R.

#### Setting Up

The hardest part is making sure that dependencies are resolved during compilation. The relevant headers are in the `Rcpp`

, `RcppEigen`

and `StanHeaders`

packages, so first these packages must be installed:

The headers must be included during compilation, to this end we set `PKG_CXXFLAGS`

system variable.

Where `YOUR_LIB_PATH`

should be replaced with the directory where R packages are installed on your system, which can be found by callind `.libPaths()`

. Note that if you are developing a package it is enough to add the following line to the `DESCRIPTION`

file:

#### Reverse-Mode AD

Now we are ready to see AD in action. The following function computes the log density function of a normal distrubted variable , and its derivative with respect to and .

The function is now callable from R.

Compare this with the gradient found using numerical differentiation:

The `C++`

code is mostly straightforward: Variables are declared as `stan::math::var`

all other variables are treated as constants. To compute the actual value of a variable we call the member-function `val()`

, to compute the gradient of a variable `y`

with respect to some variables `x1,x2`

we call `y`

’s member function `grad()`

, and the partial derivative can be obtained by calling `x1`

’s member function `adj()`

. Finally it should be noted that `stan::math`

allocates memory for variables on a global stack, that means memory is not automatically freed up when the variables goes out of scope. The function `recover_memory()`

clears this stack and `ChainableStack::memalloc_.free_all()`

frees the memory associated with it, both should be called after finish computation.