$\require{cancel} \newcommand{\Ket}[1]{\left|{#1}\right\rangle} \newcommand{\Bra}[1]{\left\langle{#1}\right|} \newcommand{\Braket}[1]{\left\langle{#1}\right\rangle} \newcommand{\Rsr}[1]{\frac{1}{\sqrt{#1}}} \newcommand{\RSR}[1]{1/\sqrt{#1}} \newcommand{\Verti}{\rvert} \newcommand{\HAT}[1]{\hat{\,#1~}} \DeclareMathOperator{\Tr}{Tr}$

Eigenvalues

^{First created in February 2019}

Ref:

• https://www.adelaide.edu.au/mathslearning/play/seminars/evalue-magic-tricks-handout.pdf

Finding Nemo

For a matrix $A$, if there exists a scalar $\lambda$ and a vector $\mathbf{v}\neq 0$, such that $\boxed{A\mathbf{v}=\lambda\mathbf{v}}~$, we call $\lambda$ an eigenvalue and $\mathbf{v}$ an eigenvector of $A$.

It follows that $\lambda\mathbf{v}-A\mathbf{v}=(\lambda I-A)\mathbf{v}=0.~~ \therefore\boxed{\det(\lambda I-A)=0.}$

It means that applying $A$ to $\mathbf{v}$ does not change its direction but scales it by a factor of the eigenvalue $\lambda$.

The function $f(x)=\det(xI-A)$ is called the characteristic polynomial of matrix $A$, which we will discuss more later.

---

Not all matrices have eigenvalues (and therefore eigenvectors). When one does, the collection of all its eigenvectors plus the zero vector is called the eigenspace.

e.g. The eigenspace of a flip matrix is the line or plane it flips over. If it does not shear in other directions, the axis it flips along is an eigenspace as well, with a negative eigenvalue. There are no eigenvalues for a rotation matrix as no vectors point to the same direction afterwards. (Product of two matrices with eigenvalues do not necessarily have eigenvalues.)

---

For diagonal matrices, the eigenvalues are on the diagonal. If all eigenvalues are distinct, the eigenspace is the axes.

e.g. A $3\times 3$ diagonal matrix $D= \begin{bmatrix} \lambda_1 & 0 & 0 \\ 0 & \lambda_2 & 0 \\ 0 & 0 & \lambda_3 \end{bmatrix}$ has eigenvalues $\lambda_1, \lambda_2$ and $\lambda_3$,

so $D\mathbf{i}=\lambda_1\mathbf{i},~~D\mathbf{j}=\lambda_2\mathbf{j},$ and $D\mathbf{k}=\lambda_3\mathbf{k}.$

That means the eigenspace is the union of $a_x\mathbf{i}, a_y\mathbf{j}, \text{and}~a_z\mathbf{k}$, where $a_x, a_y, a_z\in\mathbb{R}$ (zero included). If there are identical eigenvalues, e.g. $\lambda_1=\lambda_2$, any vectors on the $xy$-plane are eigenvectors as they are sheared equally on both axises. Generally, if there are $r$ identical eigenvalues, their corresponding eigenspace will be of $r$-dimension. If all eigenvalues are identical, the matrix becomes $\lambda I$ and its eigenspace will be $\mathbb{R}^n$.

---

Similarly, for a triangular matrix $A$, $\det(\lambda I-A)=\prod_{k=1}^n(\lambda-a_{kk})=0$, so all eigenvalues are on the diagonal as well. The eigenspace are not the axises, but it is still true that $r$ identical eigenvalues correspond to an eigenspace of $r$-dimension. (The above analysis implies that an $n\times n$ matrix has at most $n$ distinct eigenvalues.)

Row and column operations do not change the eigenvalues (even though the eigenvectors would change), one can reduce a matrix into a triangular one and find the eigenvalues, then substitute each eigenvalues into $\lambda I-A$ to solve for the eigenvectors.

Characteristic Polynomial

Let us explore the characteristic polynomail mentioned before:

$f(\lambda)= \det(\lambda I-A)= \begin{vmatrix} \lambda - a_{11} & a_{12} & \ldots & a_{1n} \\ a_{21} & \lambda - a_{22} & \ldots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{n1} & a_{n2} & \ldots & \lambda - a_{nn} \end{vmatrix} =(\lambda - a_{11})(\lambda - a_{12})\ldots(\lambda - a_{nn})+\ldots .$

---

From the definition of the determinant (odd and even permutations), the degree of the trailing terms (the dots) are in at least 1 and at most $n-2$.

This means that $c_{n-1}\lambda^{n-1}$ and $c_0\lambda^0$ must be in $(\lambda - a_{11})(\lambda - a_{22})\ldots(\lambda - a_{nn})$ (the leading term above). It follows that

$c_{n-1}=-\sum_{k=1}^na_{kk}$ and $c_0=\prod_{k=1}^na_{kk}$.

---

For an eigenvalue $\lambda_k,~f(\lambda_k)=0$. i.e. Eigenvalues are zeros of the characteristic polynomial $f(\lambda)$.

$f(\lambda) =\prod_{k=1}^n(\lambda-\lambda_k) =\sum_{k=1}^nc_k\lambda^k,~$ where $c_n=1$ and $c_{n-1}=-\sum_{k=1}\lambda_k$.

Since $c_{n-1}=-\sum_{r=1}^na_{kk}=-\Tr(A),~$ the trace of $A$ $\boxed{\Tr(A)=\sum_{k=1}^n\lambda_k.}$

---

Since $f(0) =\prod_{k=1}^n(-\lambda_k) =(-1)^n\prod_{k=1}^n\lambda_k~$ and $~f(0)=\det(0~I-A)=(-1)^n\det(A),~\boxed{\det(A)=\prod_{k=1}^n\lambda_k.}$

---

Let the $n$ independent eigenvectors of $A$ be $\mathbf{v}_k,~k=1,2,\ldots,n.$ (It is always possible if all eigenvalues are distinct. In case there are repeating eigenvalues, the dimension of the eigenspace is the multiplicity of the corresponding eigenvalue, so you can always find enough eigenvectors for that eigenvalue.)

$\because A\mathbf{v}_r=\lambda_r\mathbf{v}_r,~~ A\big[\mathbf{v}_1\mathbf{v}_2\ldots\mathbf{v}_n\big] =\big[A\mathbf{v}_1~A\mathbf{v}_2\ldots A\mathbf{v}_n\big] =\big[(\lambda_1\mathbf{v}_1)(\lambda_2\mathbf{v}_2)\ldots(\lambda_n\mathbf{v}_n)\big] =\big[\mathbf{v}_1\mathbf{v}_2\ldots\mathbf{v}_n\big]D ,$

where $D= \begin{bmatrix} \lambda_1 & 0 & \ldots & 0 \\ 0 & \lambda_2 & \ldots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \ldots & \lambda_n \end{bmatrix} .$

$\therefore~AP=PD,$\quad where $P=\big[\mathbf{v}_1\mathbf{v}_2\ldots\mathbf{v}_n\big].$\quad As $\mathbf{v}_k$ are distinct, $P$ is invertable, so $A=PDP^{-1}$. (See Similarity.)

---

Since $\det(AP)=\det(PD),~~\det(A)\det(P)=\det(P)\det(D),~~ \det(A)=\det(D)=\prod_{k=1}^n\lambda_k,~$ which we have derived earlier.

If $A$ is invertible, so is $D$.

\begin{align*} \text{Also, }\boxed{A^n=PD^nP^{-1}}\text{ as } A^n& =PDP^{-1}\cdot PDP^{-1}\ldots\cdot PDP^{-1}\\ &=PD\cdot(P^{-1}P)D\ldots(P^{-1}P)DP^{-1}\\ &=PD^nP^{-1}. \end{align*}

---

For any polynomial $\displaystyle g(A)=\sum_{k=0}^n c_k A^k=\sum_{k=0}^n c_k(PD^kP^{-1}) =P\left(\sum_{k=0}^n c_k D^k\right)P^{-1}=P~g(D)~P^{-1} .$

i.e. $\boxed{g(A)=P~g(D)~P^{-1}}.~g(A)$ is similar to $g(D)$.

---

$D^k= \begin{bmatrix} \lambda_1^k & 0 & \ldots & 0 \\ 0 & \lambda_2^k & \ldots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \ldots & \lambda_n^k \end{bmatrix} ,$

$g(D)=\sum_{k=0}^n c_k D^k= \begin{bmatrix} \sum_k c_k \lambda_1^k & 0 & \ldots & 0 \\ 0 & \sum_k c_k \lambda_2^k & \ldots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \ldots & \sum_k c_k \lambda_n^k \end{bmatrix} =\begin{bmatrix} g(\lambda_1) & 0 & \ldots & 0 \\ 0 & g(\lambda_2) & \ldots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \ldots & g(\lambda_n) \end{bmatrix} .$

So the eigenvalues of $g(D)$ (including multiplicity) are $g(\lambda_r),$ where $r=1,2,\ldots,n$.

---

As $f(A)=\det(AI-A)=0,~f(A)=P~f(D)~P^{-1}=0.$

$~\boxed{f(A)=0,~\text{where }f(\lambda)=\det(\lambda I-A).}$ (Cayley-Hamilton Theorem)

The function $f(x)=\det(xI-A)$ is called the characteristic polynomial of matrix $A$. The eigenvalues of $A$ are the zeros of its characteristic polynomial $f(x)$. In other words, given a transformation matrix, one may find its eigenvalues by solving the characteristic polynomial. Like determinants provide a "connection" between matrices and scalars, eigenvalues connect matrices and polynomials.

Note that this idea of "connections" is not a mathematical concept but helps you think. e.g. the fact that $\det(AB)=\det(A)\cdot\det(B)\quad\text{and}\quad\det(A^{-1}) =\frac{1}{\det(A)}$ would lead to $A$ having no inverse if $\det(A)=0$.

For example,

Matrix	Eigenvalues
$A$	$\lambda_k$
$cA$	$c\lambda_k$
$A^n$	$\lambda_k^n$
$f(A)$	$0$