Contents tagged with L2 regularization

  • Notes about generalization improvement

    Hello everybody,

    today I want to write few words about different regularization techniques. I will compare L1 regularization, L2 regularization and adding noise to the weights during learning process.

    L2 regularization penalizes high weight values.

    L1 regularization penalizes values that do not equal to zero.

    Adding noise to weights during learning ensures that the learned hidden representation take extreme values.