An updater is a generalization of a gradient optimization step. Several updaters do perform gradient optimization, but there are others that do something different. e.g. multiplicative updates are used in NMF, and models based on quotients of accumulated data are updated by recomputing the quotient on moving averages. Updating is broken out from the generation of the gradient (or quotient pair) which happens in the model's update method. Updaters support a few methods:

init: to initialize any params and working storage for the updater.
update: the basic update called by the learner on each minibatch. 
updateM: an update called at the end of a pass over the dataset. 
clear: clears working storage, called at the beginning of a dataset pass.

Used for batch mode inference in LDA, NMF and Gibbs sampling. Accumulates update data into numerator and denominator "accumulator" matrices during a pass over the dataset. At the end of the pass, it updates the model by computing the ratio of the accumulators.

The incremental version of the Batch Norm updater. Both numerator and denominator are updated using moving averages.

<math>V^{t+1}</math>