The last century has seen tremendous innovation within the field of arithmetic. New theories are postulated and ancient theorems are created sturdy by persistent mathematicians. And that we are still reaping the advantages of their thorough endeavors to make intelligent machines.
Here is a list of theorems normal machine learning models:
1. The Gauss-Markov Theorem:
The first a part of this theorem was given by Carl Friedrich Gauss within the year 1821 and by Andrey Markov in 1900. The trendy notation of this theorem was given by FA Graybill in 1976.
Statement: Once the error chance distribution is unknown in an exceedingly linear model, then, amongst all of the linear unbiased estimators for the parameters of the linear model, the figurer obtained victimization the strategy of the statistical procedure is that the one that minimizes the variance. The mathematical expectation of every error is assumed to be zero, and every one of them has the identical (unknown) variance.
Application: linear regression models
2. Universal Approximation Theorem:
Statement: A feed-forward network with one hidden layer containing a finite range of neurons will approximate continuous functions on compact subsets of R^n, underneath gentle assumptions on the activation perform.
Application: Artificial neural networks
3. Singular Value Decomposition:
It will be used for eigen decomposition of a symmetric matrix with positive eigenvalues to any m x n matrix by polar decomposition.
Statement: Suppose M may be m √? n matrix whose entries come back from the sector K, that is either the sector of real numbers or the sector of advanced numbers. Then there exists a resolution, referred to as a ‚??singular worth decomposition‚?? of M, of the shape.
U is an m √? m unitary matrix over K, (unitary matrices are orthogonal matrices),
ő£ may be a diagonal m √? n matrix with non-negative real numbers on the diagonal,
V is an n √? n unitary matrix over K, and V‚?? is that the conjugate transpose of V.
Application: Principal part Analysis
4. Mercer‚??s Theorem:
Postulated by Mercer in 1909, this theorem represents regular positive functions on a square because of the add of convergence of product functions.
Statement: Suppose K may be a continuous regular non-negative definite kernel. Then there's Associate in Nursing orthonormal basis I of L2[a, b] consisting of eigen functions of K specified the corresponding sequence of eigenvalues i is non-negative. The eigen functions admire non-zero eigenvalues are continuous on [a, b] and K has the representation
Application: Support Vector Machines.
5. Representer Theorem:
Statement: Among all functions, that admit Associate in Nursing infinite representation in terms of eigen functions due to Mercer‚??s theorem, the one that minimizes the regular risk invariably contains a finite illustration within the basis shaped by the kernel evaluated at the ‚??n‚?? coaching points.
Where H is that the Hilbert space and k is that the reproducing kernel.
Application: Kernel tricks (class of algorithms for pattern analysis, Support Vector Machines)