# 统计学习理论

## 形式定义

${\displaystyle X}$为所有可能的输入组成的向量空间， ${\displaystyle Y}$为所有可能的输出组成的向量空间。统计学习理论认为，积空间${\displaystyle Z=X\times Y}$上存在某个未知的概率分布${\displaystyle p(z)=p({\vec {x}},y)}$。训练集由这个概率分布中的${\displaystyle n}$个样例构成，并用${\displaystyle S=\{({\vec {x}}_{1},y_{1}),\dots ,({\vec {x}}_{n},y_{n})\}=\{{\vec {z}}_{1},\dots ,{\vec {z}}_{n}\}}$表示。每个${\displaystyle {\vec {x}}_{i}}$都是训练数据的一个输入向量， 而${\displaystyle y_{i}}$则是对应的输出向量。

## 损失函数

### 回归问题

${\displaystyle V(f({\vec {x}}),y)=(y-f({\vec {x}}))^{2}}$

${\displaystyle V(f({\vec {x}}),y)=|y-f({\vec {x}})|}$

### 分类问题

${\displaystyle V(f({\vec {x}}),y)=\theta (-yf({\vec {x}}))}$

### 正则化

${\displaystyle {\frac {1}{n}}\displaystyle \sum _{i=1}^{n}V(f({\vec {x}}_{i}),y_{i})+\gamma \|f\|_{\mathcal {H}}^{2}}$

1. ^ Rosasco, L., Vito, E.D., Caponnetto, A., Fiana, M., and Verri A. 2004. Neural computation Vol 16, pp 1063-1076
2. ^ Vapnik, V.N. and Chervonenkis, A.Y. 1971. On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and its Applications Vol 16, pp 264-280.
3. ^ Mukherjee, S., Niyogi, P. Poggio, T., and Rifkin, R. 2006. Learning theory: stability is sufficient for generalization and necessary and sufficient for consistency of empirical risk minimization. Advances in Computational Mathematics. Vol 25, pp 161-193.
4. ^ Tomaso Poggio, Lorenzo Rosasco, et al. Statistical Learning Theory and Applications, 2012, Class 2页面存档备份，存于互联网档案馆