Motivation

I am retaking this linear algebra course with 2 intentions.

First, I want to learn Dyalog APL - which is a super cool and powerful way to do linear algebra and numerical transforms. The second is to get a better understanding of Linear Algebra. I would love to achieve 'fluency' in linear algebra.

I will be doing everything in APL. So what is APL? I think of it as a better regex for linear algebra. Like regex, it is very condensed with different symbols representing different operations. Like regex, you can combine these symbols in many different ways to achieve almost anything (but in linear algebra instead of text). Unlike regex, APL is it's own full language in itself and not some string sitting inside of another language.

This blog post will be my notes about Lecture 1 from the following course:

Gilbert Strang. 18.06 Linear Algebra. Spring 2010. Massachusetts Institute of Technology: MIT OpenCourseWare, https://ocw.mit.edu. License: Creative Commons BY-NC-SA.

Matrix Multiplication

Imagine we want to multiply 2 matrices together, a1 and a2. How should we think about it?

+ a1 ← 2 2 ⍴ 2 5 1 3
+ a2 ← 2 1 ⍴ 1 2

The result is a linear combination of the columns of ar1 which we can of course write in APL like this

c1←a1[;1]×a2[1;] ⍝ Column 1 of a1 time row 1 of a2
c2←a1[;2]×a2[2;] ⍝ Column 2 of a1 time row 2 of a2
c1+c2 ⍝ Sum those up as a linear combination

A better way to do this in APL is to condense it down using an inner product.

a1+.×a2

So lets walk through the example of +.× and see why we get the correct answer and what it is doing.

What the . does is gives us a framework or structure for the answer. We take combinations of rows columns, which gives us the following equations to calculate our 2 numbers.

(2 ?1? 1) ?2? (5 ?1? 2)

(1 ?1? 1) ?2? (3 ?1? 2)

Then we need to define the 2 unknown operators (?1? and ?2?). These are defined by the operators on each side of the dot. Normal matrix multiplication means ?1? = * and ?2? = +, which is why +.x gives us matrix multiplication. But really we could put any operators in there. Addition, division, subtraction, exponents, min or max, or any other operator you can think of or create.

Another way to think about the same thing is to do this in three steps

Create our inner product matrices (rows * columns from original matrices):

$\begin{bmatrix}?&&1\\&?&2\\2&5&\end{bmatrix}$ $\begin{bmatrix}?&&1\\&?&2\\1&3&\end{bmatrix}$

Use the final operator (multiplication) to fill in the ?

$\begin{bmatrix}2&\\&10\end{bmatrix}$ $\begin{bmatrix}1&\\&6\end{bmatrix}$

Use the first operator to combine the ?

$10+2 = 12$

$1+6 = 7$

Now that we understand that, we can flip our operators and look at ×.+ instead of +.×. We can also do any number of other operators to do lots of different matrix operations.

⍝ using addition.multiplication (normal matrix multiplication) 
a1+.×a2 

⍝ using multiplication.addition
a1×.+a2 

⍝ using max.min
a1⌈.⌊a2

⍝ using addition.subtraction
a1-.+a2

⍝ using exponent.division
a1*.÷a2

⍝ using factorial.natural_log
a1!.⍟a2

Calculating Y values

Let's imaging we have a system of 2 equations and want to graph them. How would we solve for many values of x in APL? Let's just use the same example we had above

+ res ← a1 ÷ 2/a2 ⍝ Isolate function so right side (y) is 1 in all instanced

⍝ calculate y values for x is 1 through 10
xs ← ⍳10
(res[;1] ∘.× xs) + ((⍴ xs)/2 1 ⍴ res[;2])

And there you have it!

Elimination

+A ← 3 3 ⍴ 1 2 1 3 8 1 0 4 1
v ← 3 1 ⍴ 2 12 2
E21 ← 3 3 ⍴ 1 0 0 (A[2;1]×¯1÷A[1;1]) 1 0 0 0 1
E31 ← 3 3 ⍴ 1 0 0 0 1 0 ((E21+.×A)[3;1]×¯1÷(E21+.×A)[1;1]) 0 1
E32 ← 3 3 ⍴ 1 0 0 0 1 0 0 ((E31+.×E21+.×A)[3;2]×¯1÷(E31+.×E21+.×A)[2;2]) 1

E32+.×E31+.×E21+.×v