common.title

量子コンピュータの基本 - 線形代数の公式~スカラー変数のベクトル微分、行列微分

Tetsuro Tabata a year ago

#量子コンピュータ #量子計算 #線形代数 #ベクトル微分

量子コンピュータの基本 - 線形代数の公式~スカラー変数のベクトル微分、行列微分

§ この記事の目的

量子コンピュータのみに限らず、機械学習や深層学習の理論でもよく出てくる線形代数の計算や式展開のうち、ベクトルや行列を用いた微分の計算の公式とその導出方法について確認します。

§ 微分公式のまとめ

まずは公式の一覧を示します。

1. ベクトル微分の公式

ここでは、x\mathbf{x}a\mathbf{a}を列ベクトル、AAを行列とします。

xTax=aTxx=a(1)xTAxx=(A+AT)x(2)tr(xaT)x=tr(axT)x=a(3)x(aAx)T(aAx)=2AT(aAx)(4)\frac{\partial \mathbf{x}^T\mathbf{a}}{\partial\mathbf{x}}= \frac{\partial \mathbf{a}^T\mathbf{x}}{\partial\mathbf{x}}=\mathbf{a}\quad\quad(式1)\\ \frac{\partial \mathbf{x}^TA\mathbf{x}}{\partial \mathbf{x}}=(A + A^T)\mathbf{x}\quad\quad(式2)\\ \frac{\partial tr(\mathbf{xa}^T)}{\partial \mathbf{x}} = \frac{\partial tr(\mathbf{ax}^T)}{\partial \mathbf{x}} = \mathbf{a}\quad\quad(式3)\\ \frac{\partial}{\partial \mathbf{x}}(\mathbf{a}-A\mathbf{x})^T(\mathbf{a}-A\mathbf{x})=-2A^T(\mathbf{a}-A\mathbf{x})\quad\quad(式4)

2. 行列微分の公式

ここでは、x\mathbf{x}y\mathbf{y}を列ベクトル、XXAAを行列とします。
また、A|A|は行列AAの行列式を意味します。

xTXyX=xyT(5)xTX1yX=X1xyTX1(6)logXX=(X1)T(7)tr(X)X=I(8)tr(XA)X=AT(9)tr(XTA)X=A(10)tr(XAAT)X=X(A+AT)(10)xlogX=tr(X1Xx)(11)\frac{\partial \mathbf{x}^TX\mathbf{y}}{\partial X}=\mathbf{x}\mathbf{y}^T\quad\quad(式5)\\ \frac{\partial \mathbf{x}^TX^{-1}\mathbf{y}}{\partial X}=-X^{-1}\mathbf{x}\mathbf{y}^TX^{-1}\quad\quad(式6)\\ \frac{\partial \log |X|}{\partial X}=(X^{-1})^T\quad\quad(式7)\\ \frac{\partial tr(X)}{\partial X}=I\quad\quad(式8)\\ \frac{\partial tr(XA)}{\partial X}=A^T\quad\quad(式9)\\ \frac{\partial tr(X^TA)}{\partial X}=A\quad\quad(式10)\\ \frac{\partial tr(XAA^T)}{\partial X}=X(A + A^T)\quad\quad(式10)\\ \frac{\partial}{\partial x}\log|X|=tr\bigg(X^{-1}\frac{\partial X}{\partial x}\bigg)\quad\quad(式11)

スカラーyyが行列XXの関数y=f(X)y=f(X)で表される場合、yyの関数g(y)g(y)の行列XXでの微分は、

g(y)X=g(y)yf(X)X(12)\frac{\partial g(y)}{\partial X} = \frac{\partial g(y)}{\partial y}\frac{\partial f(X)}{\partial X}\quad\quad(式12)

§ 公式の導出

(式1)~(式4)までの導出方法を確認します。
厳密な証明でなく、あくまで一例ですのでご了承下さい。
(式5)以降は紙面の都合上、導出を割愛します。

1. (式1)の導出

例として3次元を取り使いますが、どの次元でも結果は同じとなります。
列ベクトルx,a\mathbf{x},\mathbf{a}をそれぞれ以下とします。

x=(x1x2x3),a=(a1a2a3)\mathbf{x}=\begin{pmatrix}x_1 \\ x_2 \\ x_3\end{pmatrix}, \mathbf{a}=\begin{pmatrix}a_1 \\ a_2 \\ a_3\end{pmatrix}

また、ベクトル微分x\frac{\partial }{\partial \mathbf{x}}は以下のように作用するものとします。

x=(x1x2x3)\frac{\partial}{\partial \mathbf{x}} = \begin{pmatrix}\frac{\partial }{\partial x_1} \\ \frac{\partial }{\partial x_2} \\ \frac{\partial }{\partial x_3}\end{pmatrix}

以下、導出例です。

xTax=x{(x1  x2  x3)(a1a2a3)}=x(a1x1+a2x2+0a3x3)=(x1x2x3)(a1x1+a2x2+a3x3)=(x1(a1x1+a2x2+a3x3)x2(a1x1+a2x2+a3x3)x3(a1x1+a2x2+a3x3))=(a1a2a3)=a\begin{align} \frac{\partial \mathbf{x}^T\mathbf{a}}{\partial \mathbf{x}} &= \frac{\partial}{\partial \mathbf{x}} \left\{(x_1\;x_2\;x_3)\begin{pmatrix}a_1 \\ a_2 \\ a_3\end{pmatrix} \right\}\\ &= \frac{\partial}{\partial \mathbf{x}}(a_1x_1+a_2x_2+0a_3x_3)\\ &=\begin{pmatrix}\frac{\partial }{\partial x_1} \\ \frac{\partial }{\partial x_2} \\ \frac{\partial }{\partial x_3}\end{pmatrix}(a_1x_1+a_2x_2+a_3x_3)\\ &= \begin{pmatrix} \frac{\partial}{\partial x_1}(a_1x_1+a_2x_2+a_3x_3) \\ \frac{\partial}{\partial x_2}(a_1x_1+a_2x_2+a_3x_3) \\ \frac{\partial}{\partial x_3}(a_1x_1+a_2x_2+a_3x_3) \end{pmatrix}\\ &=\begin{pmatrix}a_1 \\ a_2 \\ a_3\end{pmatrix} \\ &=\mathbf{a} \end{align}

また、

aTxx=x{(a1  a2  a3)(x1x2x3)}=x(a1x1+a2x2+0a3x3)=(x1x2x3)(a1x1+a2x2+a3x3)=(a1a2a3)=a\begin{align} \frac{\partial \mathbf{a}^T\mathbf{x}}{\partial \mathbf{x}} &= \frac{\partial}{\partial \mathbf{x}} \left\{(a_1\;a_2\;a_3)\begin{pmatrix}x_1 \\ x_2 \\ x_3\end{pmatrix} \right\}\\ &= \frac{\partial}{\partial \mathbf{x}}(a_1x_1+a_2x_2+0a_3x_3)\\ &=\begin{pmatrix}\frac{\partial }{\partial x_1} \\ \frac{\partial }{\partial x_2} \\ \frac{\partial }{\partial x_3}\end{pmatrix}(a_1x_1+a_2x_2+a_3x_3)\\ &=\begin{pmatrix}a_1 \\ a_2 \\ a_3\end{pmatrix} \\ &=\mathbf{a} \end{align}

2. (式2)の導出

行列AAを以下とします。

A=(A11A12A13A21A22A23A31A32A33)A=\begin{pmatrix}A_{11} & A_{12} & A_{13}\\A_{21} & A_{22} & A_{23}\\A_{31} & A_{32} & A_{33}\end{pmatrix}

以下、導出例です。

xTAxx=x{(x1  x2  x3)(A11A12A13A21A22A23A31A32A33)(x1x2x3)}=x{(A11x1+A21x2+A31x3A12x1+A22x2+A32x3A13x1+A23x2+A33x3)(x1x2x3)}=x(A11x1x1+A21x1x2+A31x1x3+A12x1x2+A22x2x2+A32x2x3+A13x1x3+A23x2x3+A33x3x3)=(2A11x1+A21x2+A31x3+A12x2+A13x3A21x1+A12x1+2A22x2+A32x3+A23x3A31x1+A32x2+A13x1+A23x2+2A33x3)=(A11x1+A21x2+A31x3A12x1+A22x2+A32x3A13x1+A23x2+A33x3)+(A11x1+A12x2+A13x3A21x1+A22x2+A23x3A31x1+A32x2+A33x3)=(A11A21A31A12A22A32A13A23A33)(x1x2x3)+(A11A12A13A21A22A23A31A32A33)(x1x2x3)=(AT+A)x=(A+AT)x\begin{align} \frac{\partial \mathbf{x}^TA\mathbf{x}}{\partial \mathbf{x}} &=\frac{\partial}{\partial \mathbf{x}} \left\{(x_1\;x_2\;x_3)\begin{pmatrix}A_{11} & A_{12} & A_{13}\\A_{21} & A_{22} & A_{23}\\A_{31} & A_{32} & A_{33}\end{pmatrix}\begin{pmatrix}x_1 \\ x_2 \\ x_3\end{pmatrix}\right\}\\ &=\frac{\partial}{\partial \mathbf{x}} \left\{(A_{11}x_1+A_{21}x_2+A_{31}x_3\quad A_{12}x_1+A_{22}x_2+A_{32}x_3\quad A_{13}x_1+A_{23}x_2+A_{33}x_3)\begin{pmatrix}x_1 \\ x_2 \\ x_3\end{pmatrix}\right\}\\ &=\frac{\partial}{\partial \mathbf{x}}(A_{11}x_1x_1+A_{21}x_1x_2+A_{31}x_1x_3+A_{12}x_1x_2+A_{22}x_2x_2+A_{32}x_2x_3+A_{13}x_1x_3+A_{23}x_2x_3+A_{33}x_3x_3)\\ &=\begin{pmatrix}2A_{11}x_1+A_{21}x_2+A_{31}x_3+A_{12}x_2+A_{13}x_3 \\ A_{21}x_1+A_{12}x_1+2A_{22}x_2+A_{32}x_3+A_{23}x_3 \\ A_{31}x_1+A_{32}x_2+A_{13}x_1+A_{23}x_2+2A_{33}x_3 \end{pmatrix}\\ &=\begin{pmatrix}A_{11}x_1+A_{21}x_2+A_{31}x_3 \\ A_{12}x_1+A_{22}x_2+A_{32}x_3 \\ A_{13}x_1+A_{23}x_2+A_{33}x_3\end{pmatrix}+ \begin{pmatrix}A_{11}x_1+A_{12}x_2+A_{13}x_3 \\ A_{21}x_1+A_{22}x_2+A_{23}x_3 \\ A_{31}x_1+A_{32}x_2+A_{33}x_3\end{pmatrix}\\ &=\begin{pmatrix}A_{11} & A_{21} & A_{31} \\ A_{12} & A_{22} & A_{32} \\ A_{13} & A_{23} & A_{33}\end{pmatrix}\begin{pmatrix}x_1 \\ x_2 \\ x_3\end{pmatrix}+ \begin{pmatrix}A_{11} & A_{12} & A_{13} \\ A_{21} & A_{22} & A_{23} \\ A_{31} & A_{32} & A_{33}\end{pmatrix}\begin{pmatrix}x_1 \\ x_2 \\ x_3\end{pmatrix}\\ &=(A^T + A)\mathbf{x}\\ &=(A + A^T)\mathbf{x} \end{align}

3. (式3)の導出

tr(xaT)x\begin{align} \frac{\partial tr(\mathbf{x}\mathbf{a}^T)}{\partial \mathbf{x}} \end{align}

ここで、

xaT=(x1x2x3)(a1  a2  a3)=(a1x1a2x1a3x1a1x2a2x2a3x2a1x3a2x3a3x3)\begin{align} \mathbf{x}\mathbf{a}^T&=\begin{pmatrix}x_1 \\ x_2 \\ x_3\end{pmatrix}(a_1\;a_2\;a_3)\\ &=\begin{pmatrix} a_1x_1 & a_2x_1 & a_3x_1 \\ a_1x_2 & a_2x_2 & a_3x_2 \\ a_1x_3 & a_2x_3 & a_3x_3 \end{pmatrix} \end{align}

トレースを取ると、

tr(xaT)=a1x1+a2x2+a3x3tr(\mathbf{x}\mathbf{a}^T)=a_1x_1 + a_2x_2 + a_3x_3

よって、

tr(xaT)x=x(a1x1+a2x2+a3x3)=(x1(a1x1+a2x2+a3x3)x2(a1x1+a2x2+a3x3)x3(a1x1+a2x2+a3x3))=(a1a2a3)=a\begin{align} \frac{\partial tr(\mathbf{x}\mathbf{a}^T)}{\partial \mathbf{x}}&= \frac{\partial}{\partial \mathbf{x}}(a_1x_1 + a_2x_2 + a_3x_3)\\ &=\begin{pmatrix} \frac{\partial}{\partial x_1}(a_1x_1 + a_2x_2 + a_3x_3)\\ \frac{\partial}{\partial x_2}(a_1x_1 + a_2x_2 + a_3x_3)\\ \frac{\partial}{\partial x_3}(a_1x_1 + a_2x_2 + a_3x_3) \end{pmatrix}\\ &=\begin{pmatrix}a_1 \\ a_2 \\ a_3\end{pmatrix}\\ &=\mathbf{a} \end{align}

また、

tr(axT)x\begin{align} \frac{\partial tr(\mathbf{a}\mathbf{x}^T)}{\partial \mathbf{x}} \end{align}

ここで、

axT=(a1a2a3)(x1  x2  x3)=(a1x1a1x2a1x3a2x1a2x2a2x3a3x1a3x2a3x3)\begin{align} \mathbf{a}\mathbf{x}^T&=\begin{pmatrix}a_1 \\ a_2 \\ a_3\end{pmatrix}(x_1\;x_2\;x_3)\\ &=\begin{pmatrix} a_1x_1 & a_1x_2 & a_1x_3 \\ a_2x_1 & a_2x_2 & a_2x_3 \\ a_3x_1 & a_3x_2 & a_3x_3 \end{pmatrix} \end{align}

トレースを取ると、

tr(axT)=a1x1+a2x2+a3x3tr(\mathbf{a}\mathbf{x}^T)=a_1x_1 + a_2x_2 + a_3x_3

これ以降は同じ導出のため省略します。

4. (式4)の導出

x(aAx)T(aAx)=x(aTxTAT)(aAx)=x(aTaaTAxxTATa+xTATAx)\begin{align} \frac{\partial}{\partial \mathbf{x}}(\mathbf{a}-A\mathbf{x})^T(\mathbf{a}-A\mathbf{x})&=\frac{\partial}{\partial \mathbf{x}}(\mathbf{a}^T-\mathbf{x}^TA^T)(\mathbf{a}-A\mathbf{x})\\ &=\frac{\partial}{\partial \mathbf{x}}(\mathbf{a}^T\mathbf{a}-\mathbf{a}^TA\mathbf{x}-\mathbf{x}^TA^T\mathbf{a}+\mathbf{x}^TA^TA\mathbf{x}) \end{align}

ここで第一項aTa\mathbf{a}^T\mathbf{a}xxに関わらないことから微分すると消える項のため除外します。

上式=x(aTAxxTATa+xTATAx)=x{(a1  a2  a3)(A11A12A13A21A22A23A31A32A33)(x1x2x3)(x1  x2  x3)(A11A21A31A12A22A32A13A23A33)(a1a2a3)+(x1  x2  x3)(A11A21A31A12A22A32A13A23A33)(A11A12A13A21A22A23A31A32A33)(x1x2x3)}=x{(a1A11+a2A21+a3A31a1A12+a2A22+a3A32a1A13+a2A23+a3A33)(x1x2x3)(a1A11+a2A12+a3A13a1A21+a2A22+a3A23a1A31+a2A32+a3A33)(a1a2a3)+(x1  x2  x3)(B11B12B13B21B22B23B31B32B33)(x1x2x3)}\begin{align} 上式&=\frac{\partial}{\partial \mathbf{x}}(-\mathbf{a}^TA\mathbf{x}-\mathbf{x}^TA^T\mathbf{a}+\mathbf{x}^TA^TA\mathbf{x})\\ &=\frac{\partial}{\partial \mathbf{x}}\left\{-(a_1\;a_2\;a_3)\begin{pmatrix}A_{11} & A_{12} & A_{13}\\A_{21} & A_{22} & A_{23}\\A_{31} & A_{32} & A_{33}\end{pmatrix}\begin{pmatrix}x_1\\x_2\\x_3\end{pmatrix} -(x_1\;x_2\;x_3)\begin{pmatrix}A_{11} & A_{21} & A_{31}\\A_{12} & A_{22} & A_{32}\\A_{13} & A_{23} & A_{33}\end{pmatrix}\begin{pmatrix}a_1\\a_2\\a_3\end{pmatrix} +(x_1\;x_2\;x_3)\begin{pmatrix}A_{11} & A_{21} & A_{31}\\A_{12} & A_{22} & A_{32}\\A_{13} & A_{23} & A_{33}\end{pmatrix}\begin{pmatrix}A_{11} & A_{12} & A_{13}\\A_{21} & A_{22} & A_{23}\\A_{31} & A_{32} & A_{33}\end{pmatrix}\begin{pmatrix}x _1\\x_2\\x_3\end{pmatrix}\right\}\\ &=\frac{\partial}{\partial \mathbf{x}}\left\{ -(a_1A_{11}+a_2A_{21}+a_3A_{31}\quad a_1A_{12}+a_2A_{22}+a_3A_{32}\quad a_1A_{13}+a_2A_{23}+a_3A_{33})\begin{pmatrix}x_1 \\ x_2 \\ x_3\end{pmatrix}-(a_1A_{11}+a_2A_{12}+a_3A_{13}\quad a_1A_{21}+a_2A_{22}+a_3A_{23}\quad a_1A_{31}+a_2A_{32}+a_3A_{33})\begin{pmatrix}a_1 \\ a_2 \\ a_3\end{pmatrix} +(x_1\;x_2\;x_3)\begin{pmatrix}B_{11} & B_{12} & B_{13} \\B_{21} & B_{22} & B_{23} \\B_{31} & B_{32} & B_{33}\end{pmatrix}\begin{pmatrix}x_1 \\ x_2 \\ x_3\end{pmatrix}\right\}\end{align}

ここで、行列BBを以下のように仮置きしました。

B=(B11B12B13B21B22B23B31B32B33)=(A11A11+A21A21+A31A31A11A12+A21A22+A31A32A11A13+A21A23+A31A33A12A11+A22A21+A32A31A12A12+A22A22+A32A32A12A13+A22A23+A32A33A13A11+A23A21+A33A31A13A12+A23A22+A33A32A13A13+A23A23+A33A33)\begin{align} B&=\begin{pmatrix}B_{11} & B_{12} & B_{13} \\ B_{21} & B_{22} & B_{23} \\ B_{31} & B_{32} & B_{33}\end{pmatrix}\\ &=\begin{pmatrix}A_{11}A_{11}+A_{21}A_{21}+A_{31}A_{31} & A_{11}A_{12}+A_{21}A_{22}+A_{31}A_{32} & A_{11}A_{13}+A_{21}A_{23}+A_{31}A_{33}\\ A_{12}A_{11}+A_{22}A_{21}+A_{32}A_{31} & A_{12}A_{12}+A_{22}A_{22}+A_{32}A_{32} & A_{12}A_{13}+A_{22}A_{23}+A_{32}A_{33}\\ A_{13}A_{11}+A_{23}A_{21}+A_{33}A_{31} & A_{13}A_{12}+A_{23}A_{22}+A_{33}A_{32} & A_{13}A_{13}+A_{23}A_{23}+A_{33}A_{33}\end{pmatrix} \end{align}

これより、

上式=x{(x1(a1A11+a2A21+a3A31)+x2(a1A12+a2A22+a3A32)+x3(a1A13+a2A23+a3A33))(a1(x1A11+x2A12+x3A13)+a2(x1A21+x2A22+x3A23)+a3(x1A31+x2A32+x3A33))+(x1B11+x2B21+x3B31x1B12+x2B22+x3B32x1B13+x2B23+x3B33)(x1x2x3)}=x{x1(a1A11+a2A21+a3A31)x2(a1A12+a2A22+a3A32)x3(a1A13+a2A23+a3A33)x1(a1A11+a2A21+a3A31)x2(a1A12+a2A21+a3A32)x3(a1A13+a2A23+a3A33)+x12B11+x1x2B21+x1x3B31+x1x2B12+x22B22+x2x3B32+x1x3B13+x2x3B23+x32B33}\begin{align} 上式&=\frac{\partial}{\partial \mathbf{x}}\left\{ -\bigg(x_1(a_1A_{11}+a_2A_{21}+a_3A_{31})+x_2(a_1A_{12}+a_2A_{22}+a_3A_{32})+x_3(a_1A_{13}+a_2A_{23}+a_3A_{33})\bigg) -\bigg(a_1(x_1A_{11}+x_2A_{12}+x_3A_{13})+a_2(x_1A_{21}+x_2A_{22}+x_3A_{23})+a_3(x_1A_{31}+x_2A_{32}+x_3A_{33})\bigg) +(x_1B_{11}+x_2B_{21}+x_3B_{31}\quad x_1B_{12}+x_2B_{22}+x_3B_{32}\quad x_1B_{13}+x_2B_{23}+x_3B_{33})\begin{pmatrix}x_1 \\ x_2 \\ x_3\end{pmatrix}\right\}\\ &=\frac{\partial}{\partial \mathbf{x}}\left\{ -x_1(a_1A_{11}+a_2A_{21}+a_3A_{31})-x_2(a_1A_{12}+a_2A_{22}+a_3A_{32})-x_3(a_1A_{13}+a_2A_{23}+a_3A_{33})-x_1(a_1A_{11}+a_2A_{21}+a_3A_{31})-x_2(a_1A_{12}+a_2A_{21}+a_3A_{32})-x_3(a_1A_{13}+a_2A_{23}+a_3A_{33}) +x_1^2B_{11}+x_1x_2B_{21}+x_1x_3B_{31}+x_1x_2B_{12}+x_2^2B_{22}+x_2x_3B_{32}+x_1x_3B_{13}+x_2x_3B_{23}+x_3^2B_{33}\right\} \end{align}

ベクトル微分x\frac{\partial}{\partial \mathbf{x}}を作用させると、

上式=((a1A11+a2A21+a3A31)(a1A11+a2A21+a3A31)+2x1B11+x2B21+x3B31+x2B12+x3B13(a1A12+a2A22+a3A32)(a1A12+a2A22+a3A32)+x1B21+x1B12+2x2B22+x3B32+x3B33(a1A13+a2A23+a3A33)(a1A13+a2A23+a3A33)+x1B31+x2B32+x1B13+x2B23+2x3B33)\begin{align} 上式&= \begin{pmatrix} -(a_1A_{11} + a_2A_{21}+a_3A_{31})-(a_1A_{11} + a_2A_{21}+a_3A_{31})+2x_1B_{11}+x_2B_{21}+x_3B_{31}+x_2B_{12}+x_3B_{13}\\ -(a_1A_{12} + a_2A_{22}+a_3A_{32})-(a_1A_{12} + a_2A_{22}+a_3A_{32})+x_1B_{21}+x_1B_{12}+2x_2B_{22}+x_3B_{32}+x_3B_{33}\\ -(a_1A_{13} + a_2A_{23}+a_3A_{33})-(a_1A_{13} + a_2A_{23}+a_3A_{33})+x_1B_{31}+x_2B_{32}+x_1B_{13}+x_2B_{23}+2x_3B_{33} \end{pmatrix} \end{align}

BBAAで戻すと、

上式=(2(a1A11+a2A21+a3A31)+2x1(A11A11+A21A21+A31A31)+x2(A12A11+A22A21+A32A31)+x3(A13A11+A23A21+A33A31)+x2(A11A12+A21A22+A31A32)+x3(A11A13+A21A23+A31A33)2(a1A12+a2A22+a3A32)+x1(A12A11+A22A21+A32A31)+x1(A11A12+A21A22+A31A32)+2x2(A12A12+A22A22+A32A32)+x3(A13A12+A23A22+A33A32)+x3(A12A13+A22A23+A32A33)2(a1A13+a2A23+a3A33)+x1(A13A11+A23A21+A33A31)+x2(A13A12+A23A22+A33A32)+x1(A11A13+A21A23+A31A33)+x2(A12A13+A22A23+A32A33)+2x3(A13A13+A23A23+A33A33))=(2(a1A11+a2A21+a3A31)+2x1(A11A11+A21A21+A31A31)+2x2(A11A12+A21A22+A31A32)+2x3(A11A13+A21A23+A31A33)2(a1A12+a2A22+a3A32)+2x1(A11A12+A21A22+A31A32)+2x2(A12A12+A22A22+A32A32)+2x3(A12A13+A22A23+A32A33)2(a1A13+a2A23+a3A33)+2x1(A11A13+A21A23+A31A33)+2x2(A12A13+A22A23+A32A33)+2x3(A13A13+A23A23+A33A33))=2(a1A11+a2A21+a3A31a1A12+a2A22+a3A32a1A13+a2A23+a3A33)+2(A11A11+A21A21+A31A31A11A12+A21A22+A31A32A11A13+A21A23+A31A33A11A12+A21A22+A31A32A12A12+A22A22+A32A32A12A13+A22A23+A32A33A11A13+A21A23+A31A33A12A13+A22A23+A32A33A13A13+A23A23+A33A33)(x1x2x3)=2{(A11A21A31A12A22A32A13A23A33)(a1a2a3)(A11A21A31A12A22A32A13A23A33)(A11A12A13A21A22A23A31A32A33)(x1x2x3)}=2(ATaATAx)=2AT(aAx)\begin{align} 上式&=\begin{pmatrix} -2(a_1A_{11}+a_2A_{21}+a_3A_{31}) +2x_1(A_{11}A_{11}+A_{21}A_{21}+A_{31}A_{31}) +x_2(A_{12}A_{11}+A_{22}A_{21}+A_{32}A_{31}) +x_3(A_{13}A_{11}+A_{23}A_{21}+A_{33}A_{31}) +x_2(A_{11}A_{12}+A_{21}A_{22}+A_{31}A_{32}) +x_3(A_{11}A_{13}+A_{21}A_{23}+A_{31}A_{33})\\ \\ -2(a_1A_{12}+a_2A_{22}+a_3A_{32}) +x_1(A_{12}A_{11}+A_{22}A_{21}+A_{32}A_{31}) +x_1(A_{11}A_{12}+A_{21}A_{22}+A_{31}A_{32}) +2x_2(A_{12}A_{12}+A_{22}A_{22}+A_{32}A_{32}) +x_3(A_{13}A_{12}+A_{23}A_{22}+A_{33}A_{32}) +x_3(A_{12}A_{13}+A_{22}A_{23}+A_{32}A_{33})\\ \\ -2(a_1A_{13}+a_2A_{23}+a_3A_{33}) +x_1(A_{13}A_{11}+A_{23}A_{21}+A_{33}A_{31}) +x_2(A_{13}A_{12}+A_{23}A_{22}+A_{33}A_{32}) +x_1(A_{11}A_{13}+A_{21}A_{23}+A_{31}A_{33}) +x_2(A_{12}A_{13}+A_{22}A_{23}+A_{32}A_{33}) +2x_3(A_{13}A_{13}+A_{23}A_{23}+A_{33}A_{33}) \end{pmatrix}\\ &=\begin{pmatrix} -2(a_1A_{11}+a_2A_{21}+a_3A_{31}) +2x_1(A_{11}A_{11}+A_{21}A_{21}+A_{31}A_{31}) +2x_2(A_{11}A_{12}+A_{21}A_{22}+A_{31}A_{32}) +2x_3(A_{11}A_{13}+A_{21}A_{23}+A_{31}A_{33})\\\\ -2(a_1A_{12}+a_2A_{22}+a_3A_{32}) +2x_1(A_{11}A_{12}+A_{21}A_{22}+A_{31}A_{32}) +2x_2(A_{12}A_{12}+A_{22}A_{22}+A_{32}A_{32}) +2x_3(A_{12}A_{13}+A_{22}A_{23}+A_{32}A_{33})\\\\ -2(a_1A_{13}+a_2A_{23}+a_3A_{33}) +2x_1(A_{11}A_{13}+A_{21}A_{23}+A_{31}A_{33}) +2x_2(A_{12}A_{13}+A_{22}A_{23}+A_{32}A_{33}) +2x_3(A_{13}A_{13}+A_{23}A_{23}+A_{33}A_{33}) \end{pmatrix}\\ &=-2\begin{pmatrix} a_1A_{11}+a_2A_{21}+a_3A_{31}\\ a_1A_{12}+a_2A_{22}+a_3A_{32}\\ a_1A_{13}+a_2A_{23}+a_3A_{33} \end{pmatrix} +2\begin{pmatrix} A_{11}A_{11}+A_{21}A_{21}+A_{31}A_{31} & A_{11}A_{12}+A_{21}A_{22}+A_{31}A_{32} & A_{11}A_{13}+A_{21}A_{23}+A_{31}A_{33}\\ A_{11}A_{12}+A_{21}A_{22}+A_{31}A_{32} & A_{12}A_{12}+A_{22}A_{22}+A_{32}A_{32} & A_{12}A_{13}+A_{22}A_{23}+A_{32}A_{33}\\ A_{11}A_{13}+A_{21}A_{23}+A_{31}A_{33} & A_{12}A_{13}+A_{22}A_{23}+A_{32}A_{33} & A_{13}A_{13}+A_{23}A_{23}+A_{33}A_{33} \end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix}\\ &=-2\left\{ \begin{pmatrix} A_{11} & A_{21} & A_{31} \\ A_{12} & A_{22} & A_{32} \\ A_{13} & A_{23} & A_{33} \end{pmatrix} \begin{pmatrix} a_1 \\ a_2 \\ a_3 \end{pmatrix} -\begin{pmatrix} A_{11} & A_{21} & A_{31} \\ A_{12} & A_{22} & A_{32} \\ A_{13} & A_{23} & A_{33} \end{pmatrix} \begin{pmatrix} A_{11} & A_{12} & A_{13} \\ A_{21} & A_{22} & A_{23} \\ A_{31} & A_{32} & A_{33} \end{pmatrix} \begin{pmatrix} x_1 \\ x_2 \\ x_3 \end{pmatrix} \right\}\\ &=-2(A^T\mathbf{a}-A^TA\mathbf{x})\\ &=-2A^T(\mathbf{a}-A\mathbf{x}) \end{align}