§2 Matrices and Matrix Operations

1. The Big Idea

A matrix is a rectangular array of numbers.

That sounds simple, but matrices are powerful because they let us organize many numbers at once. In linear algebra, matrices can represent systems of equations, transformations, data tables, or rules for combining vectors.

Example:

$$A= \begin{bmatrix} 1&2&3\\ 4&5&6 \end{bmatrix}$$

This matrix has 2 rows and 3 columns, so its size is

$$2\times 3$$

Always read matrix size as

$$\text{rows}\times\text{columns}$$

Rows go across. Columns go down.

---

2. Matrix Anatomy

Before doing operations with matrices, it helps to know the vocabulary.

Entries

The entry in row $i$, column $j$ is written as

$$a_{ij}$$

For example, in

$$A= \begin{bmatrix} 1&2&3\\ 4&5&6 \end{bmatrix}$$

we have

$$a_{21}=4$$

because row 2, column 1 contains 4.

The order of the subscripts matters: $a_{21}$ means row 2, column 1, not row 1, column 2.

Square Matrices

A square matrix has the same number of rows and columns.

Example of a $2\times 2$ square matrix:

$$\begin{bmatrix} 1&2\\ 3&4 \end{bmatrix}$$

Example of a $3\times 3$ square matrix:

$$\begin{bmatrix} 1&2&3\\ 4&5&6\\ 7&8&9 \end{bmatrix}$$

Square matrices are special because ideas like determinants, inverses, and trace only make sense for square matrices.

Main Diagonal and Trace

In a square matrix, the main diagonal goes from top left to bottom right.

For

$$A= \begin{bmatrix} 1&2&3\\ 4&5&6\\ 7&8&9 \end{bmatrix}$$

the main diagonal entries are

$$1,\ 5,\ 9$$

The trace is the sum of those entries:

$$\operatorname{tr}(A)=1+5+9=15$$

For an $n\times n$ matrix,

$$\operatorname{tr}(A)=a_{11}+a_{22}+\cdots+a_{nn}$$

Trace is only defined for square matrices because non-square matrices do not have a complete main diagonal from corner to corner.

---

3. Entrywise Operations

Some matrix operations happen entry by entry. Nothing fancy happens with rows and columns yet: matching positions are combined with matching positions.

Addition and Subtraction

Matrices can be added or subtracted only if they have the same size.

If

$$A= \begin{bmatrix} 1&2\\ 3&4 \end{bmatrix}, \qquad B= \begin{bmatrix} 5&6\\ 7&8 \end{bmatrix}$$

then

$$A+B= \begin{bmatrix} 1+5&2+6\\ 3+7&4+8 \end{bmatrix} = \begin{bmatrix} 6&8\\ 10&12 \end{bmatrix}$$

You can subtract the same way:

$$A-B= \begin{bmatrix} 1-5&2-6\\ 3-7&4-8 \end{bmatrix} = \begin{bmatrix} -4&-4\\ -4&-4 \end{bmatrix}$$

If the sizes do not match, addition and subtraction are not defined.

Scalar Multiplication

A scalar is just a number.

To multiply a matrix by a scalar, multiply every entry by that scalar:

$$3 \begin{bmatrix} 1&2\\ 3&4 \end{bmatrix} = \begin{bmatrix} 3&6\\ 9&12 \end{bmatrix}$$

Scalar multiplication stretches every entry by the same factor.

---

4. Matrix Multiplication

Matrix multiplication is the first operation that is not entry-by-entry.

Instead, it combines rows of the first matrix with columns of the second matrix.

Size Rule

If $A$ is $m\times n$ and $B$ is $n\times p$, then $AB$ is defined and has size $m\times p$.

The inside dimensions must match:

$$(m\times n)(n\times p)$$

The outside dimensions give the size of the answer:

$$m\times p$$

So the quick rule is:

$$(m\times n)(n\times p)=m\times p$$

Example:

How to Multiply

To get entry $c_{ij}$ of $AB$:

1. Take row $i$ of $A$.
2. Take column $j$ of $B$.
3. Multiply matching entries.
4. Add the results.

This is a dot product between a row and a column.

Example:

$$A= \begin{bmatrix} 1&2\\ 3&4 \end{bmatrix}, \qquad B= \begin{bmatrix} 5&6\\ 7&8 \end{bmatrix}$$

Then

$$AB= \begin{bmatrix} 1(5)+2(7)&1(6)+2(8)\\ 3(5)+4(7)&3(6)+4(8) \end{bmatrix}$$

So

$$AB= \begin{bmatrix} 19&22\\ 43&50 \end{bmatrix}$$

Order Matters

Usually,

$$AB\neq BA$$

Matrix multiplication is not commutative.

This is not just a technical detail. Matrix multiplication often represents doing one action after another. If you change the order, you change the process.

For ordinary numbers, $2\cdot 3=3\cdot 2$. For matrices, that kind of swapping usually fails.

---

5. The Identity Matrix

The identity matrix is the matrix version of the number $1$.

For $2\times 2$:

$$I_2= \begin{bmatrix} 1&0\\ 0&1 \end{bmatrix}$$

For $3\times 3$:

$$I_3= \begin{bmatrix} 1&0&0\\ 0&1&0\\ 0&0&1 \end{bmatrix}$$

The identity matrix satisfies

$$AI=A$$

and

$$IA=A$$

when the multiplication is defined.

The phrase "when the multiplication is defined" matters. The identity matrix has to have the correct size. For example, if $A$ is $2\times 3$, then $AI_3=A$ and $I_2A=A$.

---

6. Transpose

The transpose of $A$, written $A^T$, is found by switching rows and columns.

If

$$A= \begin{bmatrix} 1&2&3\\ 4&5&6 \end{bmatrix}$$

then

$$A^T= \begin{bmatrix} 1&4\\ 2&5\\ 3&6 \end{bmatrix}$$

The first row of $A$ becomes the first column of $A^T$.

Transpose Rules

$$(A+B)^T=A^T+B^T$$ $$(A-B)^T=A^T-B^T$$ $$(A^T)^T=A$$ $$(cA)^T=cA^T$$

The important product rule is

$$(AB)^T=B^TA^T$$

Notice that the order reverses. Transpose flips how rows and columns interact, so a product has to be reversed when transposed.

---

7. Inverse Matrices

An inverse is something that undoes another thing.

For ordinary numbers, multiplying by $a^{-1}$ undoes multiplying by $a$. For matrices, multiplying by $A^{-1}$ undoes multiplying by $A$.

A square matrix $A$ is invertible if there exists a matrix $B$ such that

$$AB=BA=I$$

Then $B$ is called the inverse of $A$, written

$$A^{-1}$$

So an invertible matrix satisfies

$$AA^{-1}=A^{-1}A=I$$

Only square matrices can be invertible, but not every square matrix is invertible.

Why Inverses Matter

For ordinary equations, to solve

$$ax=b$$

we divide by $a$:

$$x=a^{-1}b$$

For matrices, division is replaced by multiplying by an inverse.

If

$$AX=B$$

then multiply on the left by $A^{-1}$:

$$X=A^{-1}B$$

But if

$$XA=B$$

then multiply on the right by $A^{-1}$:

$$X=BA^{-1}$$

The side matters because matrix multiplication is not commutative.

---

8. Inverse of a $2\times 2$ Matrix

For a $2\times 2$ matrix,

$$A= \begin{bmatrix} a&b\\ c&d \end{bmatrix}$$

the inverse is

$$A^{-1} = \frac{1}{ad-bc} \begin{bmatrix} d&-b\\ -c&a \end{bmatrix}$$

provided

$$ad-bc\neq 0$$

The number

$$ad-bc$$

is the determinant of $A$.

If

$$ad-bc=0$$

then $A$ is not invertible.

You will study determinants more deeply in the next section. For now, remember this practical test: for a $2\times 2$ matrix, the inverse formula only works if $ad-bc$ is not zero.

---

9. Algebraic Properties of Inverses

If $A$ and $B$ are invertible matrices of compatible sizes, then:

The product rule is the one to remember carefully:

$$(AB)^{-1}=B^{-1}A^{-1}$$

The order reverses for the same reason you undo steps backward. If you put on socks and then shoes, you remove shoes first and socks second.

---

10. Elementary Matrices

An elementary matrix is created by performing one elementary row operation on an identity matrix.

Start with

$$I_3= \begin{bmatrix} 1&0&0\\ 0&1&0\\ 0&0&1 \end{bmatrix}$$

If you swap two rows, multiply a row by a nonzero constant, or add a multiple of one row to another, the result is an elementary matrix.

Elementary matrices matter because row operations can be represented as matrix multiplication.

Row Operations as Multiplication

Suppose a row operation is represented by an elementary matrix $E$. Then applying that row operation to $A$ is the same as multiplying:

$$EA$$

So if a sequence of elementary matrices reduces $A$ to the identity matrix, we can write something like

$$E_k\cdots E_2E_1A=I$$

That means

$$A^{-1}=E_k\cdots E_2E_1$$

The original matrix $A$ is built by undoing those row operations in the opposite direction:

$$A=E_1^{-1}E_2^{-1}\cdots E_k^{-1}$$

For example, if three elementary matrices reduce $A$ to $I$,

$$E_3E_2E_1A=I$$

then

$$A^{-1}=E_3E_2E_1$$

and

$$A=E_1^{-1}E_2^{-1}E_3^{-1}$$

This is the theoretical reason row-reduction can find inverses.

Small Example

Let

$$A= \begin{bmatrix} 1&2\\ 3&7 \end{bmatrix}$$

Eliminate the $3$ below the first pivot:

$$R_2\to R_2-3R_1$$

This row operation is represented by

$$E_1= \begin{bmatrix} 1&0\\ -3&1 \end{bmatrix}$$

Then eliminate the $2$ above the second pivot:

$$R_1\to R_1-2R_2$$

This is represented by

$$E_2= \begin{bmatrix} 1&-2\\ 0&1 \end{bmatrix}$$

Since applying $E_1$ and then $E_2$ reduces $A$ to $I$, we have

$$E_2E_1A=I$$

Therefore

$$A^{-1}=E_2E_1$$

and the original matrix can be recovered by undoing those elementary matrices in reverse:

$$A=E_1^{-1}E_2^{-1}$$

The main takeaway is not that you should always write inverses this way. The takeaway is that row operations and matrix multiplication are deeply connected.

---

11. Row Equivalence

Two matrices $A$ and $B$ are row equivalent if one can be changed into the other using elementary row operations.

This means they have the same row-reduction structure.

For augmented matrices, row equivalence is especially important because row operations preserve the solution set of a system. So row-equivalent augmented matrices represent systems with the same solutions.

---

12. Finding an Inverse by Row Reduction

To find $A^{-1}$, place $A$ beside the identity matrix:

$$[A\mid I]$$

Then row-reduce until the left side becomes $I$.

If the row-reduction succeeds,

$$[A\mid I]\to[I\mid A^{-1}]$$

The right side is the inverse.

For a $2\times 2$ matrix, the setup looks like

$$\left[ \begin{array}{cc|cc} a&b&1&0\\ c&d&0&1 \end{array} \right]$$

Row-reduce the left side to the identity matrix. Whatever appears on the right side is $A^{-1}$.

When the Method Fails

If the left side cannot become $I$, then $A$ is not invertible.

So this method is both a way to find inverses and a way to test whether an inverse exists.

---

13. Equivalent Conditions for Invertibility

For a square matrix $A$, the following statements all mean the same thing:

$$A \text{ is invertible}$$ $$A \text{ row-reduces to } I$$ $$A\vec{x}=\vec{0} \text{ has only the trivial solution}$$ $$A \text{ can be written as a product of elementary matrices}$$ $$\det(A)\neq 0$$

These are different languages for the same idea: $A$ does not collapse information, so it can be undone.

---

14. Problem-Solving Routine

When working with matrices, check the structure before doing calculations.

1. Check sizes first. Many mistakes happen before any arithmetic begins.
2. For addition or subtraction, sizes must match. Add or subtract corresponding entries.
3. For scalar multiplication, multiply every entry. No size restriction beyond having a matrix.
4. For matrix multiplication, check inside dimensions. If they match, the outside dimensions give the answer size.
5. For inverses, check that the matrix is square. Non-square matrices do not have ordinary inverses.
6. When solving with inverses, multiply on the correct side. $AX=B$ and $XA=B$ are different situations.
7. To find an inverse, row-reduce $[A\mid I]$. If the left side becomes $I$, the right side is $A^{-1}$.

---

15. Key Takeaways

• Matrix size is always rows by columns.
• Addition, subtraction, and scalar multiplication happen entry-by-entry.
• Matrix multiplication uses row-dot-column products.
• For $AB$, the inside dimensions must match.
• Matrix multiplication usually depends on order: $AB\neq BA$.
• The identity matrix acts like $1$ for matrix multiplication.
• The inverse matrix undoes multiplication by a matrix.
• Inverse and transpose product rules reverse the order.
• Row-reduction can find inverses using $[A\mid I]\to[I\mid A^{-1}]$.

---

Mini-Self-Check

1. If $A$ is $2\times 3$ and $B$ is $3\times 4$, what size is $AB$?

2. Can a $2\times 3$ matrix have an ordinary inverse?

3. Why is $(AB)^{-1}=B^{-1}A^{-1}$ instead of $A^{-1}B^{-1}$?

4. What does $[A\mid I]\to[I\mid A^{-1}]$ mean?