# 自動微分

• 對一個函數的表示式做符號上的微分，並且計算其在某一點上的值。
• 使用差分.

## 复合函数求导法则，前向和反向積累

${\displaystyle {\frac {df}{dx}}={\frac {dg}{dh}}{\frac {dh}{dx}}}$

### 前向積累

${\displaystyle w_{1}=x_{1}}$ ${\displaystyle w'_{1}=1}$ (種子)
${\displaystyle w_{2}=x_{2}}$ ${\displaystyle w'_{2}=0}$ (種子)
${\displaystyle w_{3}=w_{1}w_{2}}$ ${\displaystyle w'_{3}=w'_{1}w_{2}+w_{1}w'_{2}=1x_{2}+x_{1}0=x_{2}}$
${\displaystyle w_{4}=\sin(w_{1})}$ ${\displaystyle w'_{4}=\cos(w_{1})w'_{1}=\cos(x_{1})1}$
${\displaystyle w_{5}=w_{3}+w_{4}}$ ${\displaystyle w'_{5}=w'_{3}+w'_{4}=x_{2}+\cos(x_{1})}$

## 前向式積累自動微分的由來與二元數

${\displaystyle (x+x'\varepsilon )+(y+y'\varepsilon )=x+y+(x'+y')\varepsilon }$
${\displaystyle (x+x'\varepsilon )\cdot (y+y'\varepsilon )=xy+xy'\varepsilon +yx'\varepsilon +x'y'\varepsilon ^{2}=xy+(xy'+yx')\varepsilon }$

 ${\displaystyle P(x+x'\varepsilon )}$ ${\displaystyle =\,}$ ${\displaystyle p_{0}+p_{1}(x+x'\varepsilon )+\cdots +p_{n}(x+x'\varepsilon )^{n}}$ ${\displaystyle =\,}$ ${\displaystyle p_{0}+p_{1}x+\cdots +p_{n}x^{n}}$ ${\displaystyle \,{}+p_{1}x'\varepsilon +2p_{2}xx'\varepsilon +\cdots +np_{n}x^{n-1}x'\varepsilon }$ ${\displaystyle =\,}$ ${\displaystyle P(x)+P^{(1)}(x)x'\varepsilon }$

${\displaystyle \langle u,u'\rangle +\langle v,v'\rangle =\langle u+v,u'+v'\rangle }$
${\displaystyle \langle u,u'\rangle -\langle v,v'\rangle =\langle u-v,u'-v'\rangle }$
${\displaystyle \langle u,u'\rangle *\langle v,v'\rangle =\langle uv,u'v+uv'\rangle }$
${\displaystyle \langle u,u'\rangle /\langle v,v'\rangle =\left\langle {\frac {u}{v}},{\frac {u'v-uv'}{v^{2}}}\right\rangle \quad (v\neq 0)}$
${\displaystyle \sin \langle u,u'\rangle =\langle \sin(u),u'\cos(u)\rangle }$
${\displaystyle \cos \langle u,u'\rangle =\langle \cos(u),-u'\sin(u)\rangle }$
${\displaystyle \exp \langle u,u'\rangle =\langle \exp u,u'\exp u\rangle }$
${\displaystyle \log \langle u,u'\rangle =\langle \log(u),u'/u\rangle \quad (u>0)}$
${\displaystyle \langle u,u'\rangle ^{k}=\langle u^{k},ku^{k-1}u'\rangle \quad (u\neq 0)}$
${\displaystyle \left|\langle u,u'\rangle \right|=\langle \left|u\right|,u'{\mbox{sign}}u\rangle \quad (u\neq 0)}$

${\displaystyle g(\langle u,u'\rangle ,\langle v,v'\rangle )=\langle g(u,v),g^{(1)}(u,v)u'+g^{(2)}(u,v)v'\rangle }$

## 實作

### 程式碼轉換

Figure 4: Example of how source code transformation could work

## Literature

• Rall, Louis B. Automatic Differentiation: Techniques and Applications. Lecture Notes in Computer Science 120. Springer. 1981. ISBN 0-540-10861-0 请检查|isbn=值 (帮助).
• Griewank, Andreas. Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation. Frontiers in Applied Mathematics 19. SIAM. 2000. ISBN 0-89871-451-6.