Hello everybody,

today I want to describe in simple words some training algos of Encog.

Before I'll continue, I want to show general block schema of training algorithms:

Init NN can look like this:

public BasicNetwork CreateNetwork()
{
   var network = new BasicNetwork();
   network.AddLayer(new BasicLayer(WindowSize));
   network.AddLayer(new BasicLayer(10));
   network.AddLayer(new BasicLayer(1));
   network.Structure.FinalizeStructure();
   network.Reset();
   return network;
}

Steps "NN error < acceptable error" -> "Update weights according to learning algorithim" can look like this:

public void Train(BasicNetwork network, IMLDataSet training)
{
    ITrain train = new ResilientPropagation(network, training);
    int epoch = 1;
    do{
            train.Iteration();
            Console.WriteLine(@"Epoch #" + epoch + @" Error:" + train.Error);
            epoch++;
         } while (train.Error > acceptableError);
    }

The first in my decription goes Backpropogation algorithm. Few characteristics of it: 

1. probably the oldest training algorithm

2. As name says error are propogated back

3.  usage: 

var train = new Backpropogation(network, trainingSet, learningRate, momentum);

The next learning algorithm goes Manhattan update rule. Few characteristics of it:

1. use only sign of the gradient

2. Updates weiht by constant value ( depending from the sign it will be added or subtracted from the weithg )

3.  usage: 

var train = new ManhattanPropogation(network, trainingSet, constantValue);

as usually constantValue is very low ( around 0.00001 )

Other one is Quck propogation algorithm. Features of this method:

1. Newton's method of error minimization

2. Use learning trate which is higher value ( I used values starting from 2 )

3. Usage: 

var train = new QuickPropogation(network, trainingSet, learningRate);

Yet one more training algorithm is Resilent propogation algorithm. Some details of this method:

1. One of the fastest algorithm in Encog.

2. Easiest usage ( you don't care about rate, momentum or constant value )

3. It uses sign of gradient value.

4. It tries to compute delta value for each weight and use it for calculated weight.

5.  Example of usage:

var train = new ResilentPropogation(network, trainingSet);

6. It has four variants of resilent propogation: RPROP+, RPROP-, iRPROP+, iRPROP- . The best is considered iRPROP+

One more detail goes Scaled conjugate gradient. Features of it:

1. Use conjugate gradient method

2. Not applicable to all dataset

3. Doesn't require any learning paramethers

4. Example of usage:

var train = new ScaledConjugateGradient(network, trainingSet);

And the last one propogation algorithm is Levenberg Marquardt Algorithm ( aka LMA ). This is my favorite method. Features of it:

1. Something in between Gauss-Newton Algorithm ( GNA ) & Gradient Method

2. Easy to use ( no learning parameters )

3. Example of creating:

var train = new LevenbergMarquardtTraining(network, trainingSet);

4. Sometime it can be very efficient.

Ready to take your Acumatica experience to the next level? Just as Encog offers a variety of training algorithms to optimize neural networks, our team specializes in tailoring Acumatica to fit your unique business needs. Whether it's custom workflows, integrations, or advanced reporting, we’re here to help you achieve peak efficiency. Leave a customization request today and let’s build a solution that works as hard as you do!