# 自動微分

• 對一個函數的表示式做符號上的微分，並且計算其在某一點上的值。
• 使用差分.

## 复合函数求导法则，前向和反向積累

$\frac{df}{dx} = \frac{dg}{dh} \frac{dh}{dx}$

### 前向積累

$w_1 = x_1$ $w'_1 = 1$ (種子)
$w_2 = x_2$ $w'_2 = 0$ (種子)
$w_3 = w_1 w_2$ $w'_3 = w'_1 w_2 + w_1 w'_2 = 1 x_2 + x_1 0 = x_2$
$w_4 = \sin(w_1)$ $w'_4 = \cos(w_1)w'_1 = \cos(x_1) 1$
$w_5 = w_3 + w_4$ $w'_5 = w'_3 + w'_4 = x_2 + \cos(x_1)$

## 前向式積累自動微分的由來與二元數

$(x + x'\varepsilon) + (y + y'\varepsilon) = x + y + (x' + y')\varepsilon$
$(x + x'\varepsilon) \cdot (y + y'\varepsilon) = xy + xy'\varepsilon + yx'\varepsilon + x'y'\varepsilon^2 = xy + (x y' + yx')\varepsilon$

 $P(x + x'\varepsilon)$ $=\,$ $p_0 + p_1(x + x'\varepsilon) + \cdots + p_n (x + x'\varepsilon)^n$ $=\,$ $p_0 + p_1 x + \cdots + p_n x^n$ $\, {} + p_1x'\varepsilon + 2p_2xx'\varepsilon + \cdots + np_n x^{n-1} x'\varepsilon$ $=\,$ $P(x) + P^{(1)}(x)x'\varepsilon$

$\langle u,u'\rangle +\langle v,v'\rangle = \langle u+v, u'+v' \rangle$
$\langle u,u'\rangle -\langle v,v'\rangle = \langle u-v, u'-v' \rangle$
$\langle u,u'\rangle *\langle v,v'\rangle = \langle u v, u'v+uv' \rangle$
$\langle u,u'\rangle /\langle v,v'\rangle = \left\langle \frac{u}{v}, \frac{u'v-uv'}{v^2} \right\rangle \quad ( v\ne 0)$
$\sin\langle u,u'\rangle = \langle \sin(u) , u' \cos(u) \rangle$
$\cos\langle u,u'\rangle = \langle \cos(u) , -u' \sin(u) \rangle$
$\exp\langle u,u'\rangle = \langle \exp u , u' \exp u \rangle$
$\log\langle u,u'\rangle = \langle \log(u) , u'/u \rangle \quad (u>0)$
$\langle u,u'\rangle^k = \langle u^k , k u^{k-1} u' \rangle \quad (u \ne 0)$
$\left| \langle u,u'\rangle \right| = \langle \left| u \right| , u' \mbox{sign} u \rangle \quad (u \ne 0)$

$g(\langle u,u' \rangle , \langle v,v' \rangle ) = \langle g(u,v) , g^{(1)}(u,v) u' + g^{(2)}(u,v) v' \rangle$

## 實作

### 程式碼轉換

Figure 4: Example of how source code transformation could work

## Literature

• Rall, Louis B. Automatic Differentiation: Techniques and Applications. Lecture Notes in Computer Science 120. Springer. 1981. ISBN 0-540-10861-0 请检查|isbn=值 (帮助).
• Griewank, Andreas. Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. Frontiers in Applied Mathematics 19. SIAM. 2000. ISBN 0-89871-451-6.