Solved – Calculating gradient of a function for optimization

gradientjavaoptimizationr

I need to optimize a function. This function is a likelihood function which takes a set of parameters (to be optimized) and calculates the likelihood (to be optimized) as a result. 

b{} //utility function parameters (input)
d //data (input)
// calculate likelihood (operation)
exp(d*b)...
...
ll // likelihood (output)

I first wrote my likelihood function in R language and used optim as optimizer. This optim function takes an objective function and inputs but doesn't require a gradient function, it calls a C library in back-end but i couldn't find it's source code to explore how it works. (Probably it gives different values iteratively as input; not sure…)

Now i need to develop same function in java. For a similar optimization calculation, i decided to use Weka optimization library (or Joptimizer which is so similar). These libraries optimization functions need "objective function"(my likelihood function) and "gradient function" (gradient of my likelihood function).

class MyOpt extends Optimization {
   // Provide the objective function **i have this**
   protected double objectiveFunction(double[] x) {
     // How to calculate your objective function...
     // ...
   }

   // Provide the first derivatives **i need this**
   protected double[] evaluateGradient(double[] x) {
     // How to calculate the gradient of the objective function...
     // ...
   }

   // If possible, provide the indexˆ{th} row of the Hessian matrix
   protected double[] evaluateHessian(double[] x, int index) {
     // How to calculate the indexˆth variable's second derivative
     // ...
   }
 }

I couldn't turn my objective function's (likelihood) code to a simple mathematical function to differentiate so i can't provide a gradient function. I would like to learn how to calculate derivative of a function which cannot be formulated mathematically.

Best Answer

Well, you may try to compute a derivative numerically, but that will drastically increase computation time. You can read about gradient checking here www.holehouse.org/mlclass

Related Question