Solved – Resilient Propagation: How to choose between RPROP+, RPROP-, iRPROP+, and iPROP-

backpropagationgradient descentperceptron

I am using ENCOG to implement a Perceptron network. One of the easiest back-propagation (gradient descent) algorithms to use is the Resilient Propagation algorithm.

There are four variants for this algorithm:

  1. RPROP+
  2. RPROP-
  3. iRPROP+
  4. iRPROP-

My question is, how would one choose between these variants, and what exactly is the difference between these calculation methods?

I understand that Resilient Propagation keeps a map of weight differences which the Resilient Back Propagation algorithm uses to calculate the weight changes on the next iteration (instead of updating all the weights with the same value), but I would like to know how these different variants differ in updating these weights, and any advice (anecdotal or with backup research) that one would use to choose between these variants.

Best Answer

According to the 2003 paper Empirical Evaluation of the Improved Rprop Learning Algorithms, iRPROP+ has the best performance.