Inverse Matrix

This section focus on how to compute the inverse of a matrix. Later we’ll see it from the point of view of an inverse map.

Only full rank square matrices have inverses. From a calculation’s perspective, what we know is that the inverse of \(A\) is the unique solution \(A^{-1}\) of the equation:

\[ AA^{-1} = I = A^{-1}A \]

How do we solve this equation?

How to compute an inverse?

Consider the matrix

\[ A=\left(\begin{matrix}1 & 2 \\3 & 5 \end{matrix}\right) \]

The goal is to solve:

\[ \left(\begin{matrix}1 & 2 \\3 & 5 \end{matrix}\right)\left(\begin{matrix}a & c \\b & d \end{matrix}\right)=\left(\begin{matrix}1 & 0 \\0 & 1 \end{matrix}\right) \]

Close inspection of the product between the matrices shows us that we can break this problem into two independent problems:

\[ \begin{align} &\left(\begin{matrix}1 & 2 \\3 & 5 \end{matrix}\right)\left(\begin{matrix}a\\b \end{matrix}\right) = \left(\begin{matrix}1\\0 \end{matrix}\right) \\ &\left(\begin{matrix}1 & 2 \\3 & 5 \end{matrix}\right)\left(\begin{matrix}c\\d \end{matrix}\right) = \left(\begin{matrix}0\\1 \end{matrix}\right) \end{align} \]

With elimination we find:

\[ \begin{align} &\left(\begin{matrix}1 & 2 \\3 & 5 \end{matrix}\;\middle|\;\begin{matrix}1\\0 \end{matrix}\right) \overset{l_2'=l_2-3l_1}{\longrightarrow} \left(\begin{matrix}1 & 2 \\0 & -1 \end{matrix}\;\middle|\;\begin{matrix}1\\-3 \end{matrix}\right) \overset{l_2'=-l_2}{\longrightarrow} \left(\begin{matrix}1 & 2 \\0 & 1 \end{matrix}\;\middle|\;\begin{matrix}1\\3 \end{matrix}\right) \overset{l_1'=l_1-2l_2}{\longrightarrow} \left(\begin{matrix}1 & 0 \\0 & 1 \end{matrix}\;\middle|\;\begin{matrix}-5\\3 \end{matrix}\right)\\ &\left(\begin{matrix}1 & 2 \\3 & 5 \end{matrix}\;\middle|\;\begin{matrix}0\\1 \end{matrix}\right) \overset{l_2'=l_2-3l_1}{\longrightarrow} \left(\begin{matrix}1 & 2 \\0 & -1 \end{matrix}\;\middle|\;\begin{matrix}0\\1 \end{matrix}\right) \overset{l_2'=-l_2}{\longrightarrow} \left(\begin{matrix}1 & 2 \\0 & 1 \end{matrix}\;\middle|\;\begin{matrix}0\\-1 \end{matrix}\right) \overset{l_1'=l_1-2l_2}{\longrightarrow} \left(\begin{matrix}1 & 0 \\0 & 1 \end{matrix}\;\middle|\;\begin{matrix}2\\-1 \end{matrix}\right) \end{align} \tag{1}\]

From the rhs of Equation 1 we conclude that the inverse matrix is:

\[ A^{-1}=\left(\begin{matrix}-5 & 3 \\2 & -1 \end{matrix}\right) \]

Checking the result:

\[ AA^{-1}=\left(\begin{matrix}1 & 2 \\3 & 5 \end{matrix}\right)\left(\begin{matrix}-5 & 2 \\3 & -1 \end{matrix}\right)=\left(\begin{matrix}1 & 0 \\0 & 1 \end{matrix}\right) \]

It works!

Observations:

  • Observing the calculations Equation 1 we notice the arrow operators are the same for both problems, and since in this case we are dealing with a full rank matrix, the end point simplification must be an identity matrix, just like we observe on the rhs of Equation 1.
Recall

In case it is not clear why the columns \((-5,3)\) and \((2,-1)\) tell us the columns of \(A^{-1}\), remember the meaning of the extended matrix notation in Equation 1:

\[ \begin{align} &\left(\begin{matrix}1 & 0 \\0 & 1 \end{matrix}\;\middle|\;\begin{matrix}-5\\3 \end{matrix}\right)\leftrightsquigarrow\left(\begin{matrix}1 & 0 \\0 & 1 \end{matrix}\right)\left(\begin{matrix}a\\b \end{matrix}\right)=\left(\begin{matrix}-5\\3\end{matrix}\right) \implies \left(\begin{matrix}a\\b \end{matrix}\right)=\left(\begin{matrix}-5\\3\end{matrix}\right)\\ &\left(\begin{matrix}1 & 0 \\0 & 1 \end{matrix}\;\middle|\;\begin{matrix}2\\-1 \end{matrix}\right)\leftrightsquigarrow\left(\begin{matrix}1 & 0 \\0 & 1 \end{matrix}\right)\left(\begin{matrix}c\\d \end{matrix}\right)=\left(\begin{matrix}2\\-1\end{matrix}\right) \implies \left(\begin{matrix}c\\d \end{matrix}\right)=\left(\begin{matrix}2\\-1\end{matrix}\right) \end{align} \]

  • Since the calculations operations in Equation 1 are the same, the main difference being the right hand side vector on which they are applied, rather than doing the same thing twice, we do it one shot adopting the following notation for Equation 1:

    \[ \left(\begin{matrix}1 & 2 \\3 & 5 \end{matrix}\;\middle|\;\begin{matrix}1 & 0\\0 & 1\end{matrix}\right) \overset{l_2'=l_2-3l_1}{\longrightarrow} \left(\begin{matrix}1 & 2 \\0 & -1 \end{matrix}\;\middle|\;\begin{matrix}1 & 0\\-3 &1 \end{matrix}\right) \overset{l_2'=-l_2}{\longrightarrow} \left(\begin{matrix}1 & 2 \\0 & 1 \end{matrix}\;\middle|\;\begin{matrix}1 & 0\\3 & -1\end{matrix}\right) \overset{l_1'=l_1-2l_2}{\longrightarrow} \left(\begin{matrix}1 & 0 \\0 & 1 \end{matrix}\;\middle|\;\begin{matrix}-5 & 2\\3 & -1 \end{matrix}\right)\\ \tag{2}\]

  • Lets describe what is going on in Equation 2 using a different notation. To each arrow operator corresponds a matrix. There are three eliminations steps and thus three elimination matrices \(E_1\), \(E_2\) and \(E_3\). Acting with them on \(A\) we reduced it to an \(I\):

    \[ \overbrace{E_3 E_2 E_1}^\text{arrows operators} A =I \]

    In turn acting with them on \(I\) returns \(A^{-1}\):

    \[ E_3 E_2 E_1 I = A^{-1} \]

    From this we conclude that the inverse is in fact the product of these elimination matrices!

    \[ A^{-1} = E_3 E_2 E_1 \]

  • Here is the recipe to compute the inverse of a given \(A\):

    • Step 1: Write the extended matrix \([A|I]\)

    • Step 2: Apply elimination operations on all entries bellow and above the diagonal so that the block \(A\) ends becoming an \(I\), in the process apply the same operation on the extended \(I\) block.

    • Step 3: The final result has the form \([I| A^{-1}]\), read the inverse matrix entries from this notation and check your results by computing \(AA^{-1}=I\).

    • Step 4: Congratulations!

Another way to compute inverses (determinant way)

Remember the formula that contained all ways to compute determinants:

\[ \left(\begin{matrix} A_{11} & A_{12} & A_{13}\\ A_{21} & A_{22} & A_{23}\\ A_{31} & A_{32} & A_{33} \end{matrix}\right) \left(\begin{matrix} C_{11} & C_{21} & C_{31}\\ C_{12} & C_{22} & C_{32}\\ C_{13} & C_{23} & C_{33} \end{matrix}\right)=\det A\left(\begin{matrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1 \end{matrix}\right) \tag{3}\]

If an inverse of \(A\) exists, then multiplying this formula by \(A^{-1}\) and using \(A^{-1}A=I\) and the fact that \(A^{-1}I=A^{-1}\) gives us:

\[ \left(\begin{matrix} A_{11} & A_{12} & A_{13}\\ A_{21} & A_{22} & A_{23}\\ A_{31} & A_{32} & A_{33} \end{matrix}\right)^{-1}=\frac{1}{\det A}\left(\begin{matrix} C_{11} & C_{21} & C_{31}\\ C_{12} & C_{22} & C_{32}\\ C_{13} & C_{23} & C_{33} \end{matrix}\right) \tag{4}\]

As an example: consider the simple \(2\times2\) matrix, then:

\[ \left(\begin{matrix} a & b\\ c & d \end{matrix}\right)^{-1} = \frac{1}{ad-bc}\left(\begin{matrix} d & -b\\ -c & a \end{matrix}\right) \]

Matrix with no inverse

When a matrix does not have a full rank, it has no inverse, the reason being the determinant is zero and as a result: the calculation Equation 4 cannot be performed.

Elimination matrices

The \(E\) matrices always have inverses, thus:

\[ Ax=b \iff EAx=Eb \iff A'x=b' \]

are equivalent and as a result their truth sets are equal! Finding the truth set of the later is easier than the former, because the \(E\) eliminated most of the unknowns of the systems.

Properties of inverses

  • \((A^{-1})^{-1}=A\)

    Justification: since given an \(A\) the solution of \(AA^{-1}=I\) is the unique \(A^{-1}\), then given an \(A^{-1}\) the solution of \((A^{-1})^{-1}A^{-1}=I\) must be unique as well. Comparing both equation, it must be that \((A^{-1})^{-1}=A\).

  • \((AB)^{-1}=B^{-1}A^{-1}\)

    Justification: If first I put my socks on and then my shoes, the reverse of that is to first take my shoes and then my socks.

  • \((aA)^{-1}=\frac{1}{a}A^{-1}\)

    Justification: Just rewrite \(a\) as \(a I\) and then use the rule above.

  • \((A^{-1})^\intercal = (A^\intercal)^{-1}\)

    Justification: Start with what we know: \(A^{-1} A =I\), then take the transpose of both sides, \(A^\intercal (A^{-1})^\intercal=I\), but this is equivalent to \((A^{-1})^\intercal = (A^\intercal)^{-1}\).

  • \((A^{-1})^n=(A^n)^{-1}\)

    Justification: Start again with what we know \(A^{-1} A =I\), then modify it so that powers of \(n\) appear. Since what we know is equivalent to \((A^{-1} A )^n=I^n\), and this in turn is \((A^{-1})^n A^n =I\), rearranging we find our result.

  • \(|A^{-1}|=1/|A|\)

    Justification: Going back to Equation 3 we notice \(|A||C|=|A|\) which means \(|C|=1\), thus, from Equation 4 we conclude this result.