today I want to write few words about different regularization techniques. I will compare L1 regularization, L2 regularization and adding noise to the weights during learning process.
L2 regularization penalizes high weight values.
L1 regularization penalizes values that do not equal to zero.
Adding noise to weights during learning ensures that the learned hidden representation take extreme values.