Automatic Differentiation in R09 Jan 2017
Automatic Differentiation (AD) specifically reverse-mode AD takes a function, defined by a computer program and computes the gradient exactly (up to machine precision) and at the same time complexity as the original program.
Here I will show how one can use the Reverse-Mode Automatic Differentiation Library provided by Stan Math Library in R.
The hardest part is making sure that dependencies are resolved during compilation. The relevant headers are in the
StanHeaders packages, so first these packages must be installed:
The headers must be included during compilation, to this end we set
PKG_CXXFLAGS system variable.
YOUR_LIB_PATH should be replaced with the directory where R packages are installed on your system, which can be found by callind
.libPaths(). Note that if you are developing a package it is enough to add the following line to the
Now we are ready to see AD in action. The following function computes the log density function of a normal distrubted variable , and its derivative with respect to and .
The function is now callable from R.
Compare this with the gradient found using numerical differentiation:
C++ code is mostly straightforward: Variables are declared as
stan::math::var all other variables are treated as constants. To compute the actual value of a variable we call the member-function
val(), to compute the gradient of a variable
y with respect to some variables
x1,x2 we call
y’s member function
grad(), and the partial derivative can be obtained by calling
x1’s member function
adj(). Finally it should be noted that
stan::math allocates memory for variables on a global stack, that means memory is not automatically freed up when the variables goes out of scope. The function
recover_memory() clears this stack and
ChainableStack::memalloc_.free_all() frees the memory associated with it, both should be called after finish computation.