**This is an old revision of the document!** ----
====== Math 753/853 Norms, inner products, and orthogonality ====== Ok, this is a big set of topics, and nothing I've found covers the topic at the right level of detail or depth. So, here's a summary of two key points you should understand. **Matrix 2-norm:** The 2-norm of a matrix $A$ is defined as \begin{equation*} \|A\| = \sup_{x\neq0} \frac{\|Ax\|}{\|x\|} = \sup_{\|x\| = 1 } \|Ax\| \end{equation*} You can think of $\|A\|$ as the maximum amplification in length that can occur under the map $x \rightarrow Ax$. **Orthogonal matrix:** A matrix $Q$ is an orthogonal matrix if its inverse is its transpose: $Q^T Q = I$. The columns of an orthogonal matrix are a set of orthogonal vectors. https://en.wikipedia.org/wiki/Matrix_norm