Background

We started on this topic in the last post, so much this section is review - but critical for the upcoming discussion.

Vector Spaces

To have a vector space you need to be able to:

  1. Add any vectors and still be within the space
  2. Multiply by any number and stay within space
  3. Take any linear combination and stay within space

Examples

$R^2$ is a vector space. This is all real number vectors with 2 components. Here's a few examples.

$\begin{bmatrix}3\\2\end{bmatrix}$, $\begin{bmatrix}0\\0\end{bmatrix}$, $\begin{bmatrix}\pi\\e\end{bmatrix}$

$R^3$ is another vector space, but it contains all real 3D vectors. Here's a few examples.

$\begin{bmatrix}3\\2\\5\end{bmatrix}$, $\begin{bmatrix}3\\2\\0\end{bmatrix}$, $\begin{bmatrix}0\\0\\0\end{bmatrix}$, $\begin{bmatrix}\pi\\e\\5\end{bmatrix}$

$R^n$ is another with all column vectors with $n$ real components.

Subspaces

What if we just want a piece of $R^n$? We still need to be able to add vectors and multiply it together. So what are some subspaces?

  1. A line through $R^n$ that goes through the origin. Any multiple of a point on a line is still on the line. We need the origin because you can mutliply any vector by 0, which would give you the origin.
  2. Just the Origin is a subspace. You can add it to itelf, multiply it by anything, and take any combination and you will stil lhave the origin.
  3. In $R^3$ and above, a plane through the origin is also a subspace.

Combinations of Subspaces

If I have 2 subspaces, then all vectors in P or L or both does not make a subspace. I can't add all together. In $R^3$ I would have to make a cube which would be all of $R^3$.

However, if you all points that are in both subspaces does make a subspace it is a subspace. If you think about it, by definition 2 Vectors in the intersection are for the subspace 1 and they are both in subspace 2. We already know that if you add 2 vectors in subspace 1 you get another vector in subspace 1. We also know the same thing about subspace 2. So really, any linear combination has to be in both subspaces (probably a smaller subspace).

Subspace from a matrix

Column Space

$\begin{bmatrix}1&1&2\\2&1&3\\3&1&4\\4&1&5\end{bmatrix}$

Columns are in $R^4$ and all their linear combinations are a subspace. So we have 3 subspaces, 1 for each column. So the full subspace for this matrix must have a linear combination of all columns, or all 3 subspaces. With 3 columns do we get all of $R^4$? What exactly do we get? How big is this subspace? Does $Ax=b$ have a solution for every $b$? Which $b$ values are ok?

We know it doesn't, because we have 4 equations but only 3 unknowns. In the proper form we see.

$Ax=b$ $\begin{bmatrix}1&1&2\\2&1&3\\3&1&4\\4&1&5\end{bmatrix}$ $\begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix}$ $=$ $\begin{bmatrix}b_1\\b_2\\b_3\\b_4\end{bmatrix}$

Now, just because we can't solve for every b doesn't mean we can't solve it for any b. So which $b$'s allow this to be solved? Obviously we can solve for the 0 vector. Obviously we can solve for (1,2,3,4) or any other column as we can have $x_1 = 1$ with the other x's being 0. We could also solve for multiples of any column because $x_2$ could equal 2 or 3 with other uknowns being 0.

Note: To Summarize, Ax=b can be solved with $b$ is in $C(A)$, the column space of A.
Now column 3 and column 1 are redundant, so we can throw away column 3. Column 1 + column 2 = column 3. So really linear combinations of the first 2 columns gives the same subspace as linear combinations of all 3 columns. So in all practicality, we have 2 equations with 4 unknowns with 2 of the columns sitting on the same plane.

Nullspace

$\begin{bmatrix}1&1&2\\2&1&3\\3&1&4\\4&1&5\end{bmatrix}$

Nullspace of $A$ = all solutions of $x$ where $Ax=0$. Because there are 3 unknowns, it's a subspace in $R^3$. This is opposed to the column space that is in $R^4$. Naturally the 0 vector always satisfies this so is always in the nullspace. Here's a few others

$\begin{bmatrix}0\\0\\0\end{bmatrix}$, $\begin{bmatrix}1\\1\\-1\end{bmatrix}$, $\begin{bmatrix}2\\2\\-2\end{bmatrix}$, $\begin{bmatrix}-1\\-1\\1\end{bmatrix}$, $\begin{bmatrix}-2\\-2\\2\end{bmatrix}$

To summarize, we really have a line in $R^3$, which is a subspace: $c\begin{bmatrix}1\\1\\-1\end{bmatrix}$

Solution Spaces

Great! So we have defined the nullspace, which is just a subspace when $b$ is the 0 vector. Why not do the same thing with other $b$'s? For example:

$Ax=b$ $\begin{bmatrix}1&1&2\\2&1&3\\3&1&4\\4&1&5\end{bmatrix}$ $\begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix}$ $=$ $\begin{bmatrix}1\\2\\3\\4\end{bmatrix}$

Clearly it isn't a space because the zero vector is not a solution.

Possible Solutions = $\begin{bmatrix}1\\0\\0\end{bmatrix}$, $\begin{bmatrix}2\\1\\-1\end{bmatrix}$, $\begin{bmatrix}3\\2\\-2\end{bmatrix}$, $\begin{bmatrix}0\\-1\\1\end{bmatrix}$