Regularizer¶
Regularization prevent overfitting and introduce additional information (prior knowledge) to solve an ill-posed problem.
Regularizers implement the following main methods:
- value(r::Regularizer)¶
Compute the value of the regularizer.
- gradient(r::Regularizer)¶
Compute the gradient of the regularizer.
The following regularizers are implemented:
- L2reg(w::Vector, λ::Float64)¶
Implements an \(L^2\)-norm regularization of the weight vector w of the decision function:
\[\begin{split}\Omega({\bf w})&=\frac{1}{2\lambda}\|{\bf w}\|^2,\end{split}\]where the regularization parameter \(\lambda\) controls the influence of the regularizer.
Note
The \(L^2\)-norm regularization corresponds to Gaussian prior assumption of \({\bf w}\sim\mathcal{N}({\bf 0},\lambda{\bf I})\).