Linear Algebra with NumPy

What is Linear Algebra?

Linear algebra is the math of matrices (grids of numbers). It's fundamental to machine learning, graphics, and data science.

Why learn this:

Machine learning uses matrices everywhere
Image processing works with pixel matrices
Data transformations
Solving systems of equations

Matrix Basics

A matrix is a 2D array of numbers.

code.py

import numpy as np

matrix = np.array([[1, 2, 3], [4, 5, 6]])
print(matrix)
print("Shape:", matrix.shape)

Output:

[[1 2 3]
 [4 5 6]]
Shape: (2, 3)

What shape means: 2 rows, 3 columns (2×3 matrix).

Matrix Multiplication

Two types: element-wise and true matrix multiplication.

Element-wise (*)

code.py

import numpy as np

a = np.array([[1, 2], [3, 4]])
b = np.array([[5, 6], [7, 8]])

result = a * b
print(result)

Output:

[[5 12]
 [21 32]]

What happens: Each element multiplied individually.

True Matrix Multiplication (@)

code.py

import numpy as np

a = np.array([[1, 2], [3, 4]])
b = np.array([[5, 6], [7, 8]])

result = a @ b
print(result)

Output:

[[19 22]
 [43 50]]

Or use matmul:

code.py

result = np.matmul(a, b)

How it works: Row times column:

(1×5 + 2×7) = 19
(1×6 + 2×8) = 22
etc.

Dot Product

Multiply and sum elements.

1D Dot Product

code.py

import numpy as np

a = np.array([1, 2, 3])
b = np.array([4, 5, 6])

dot = np.dot(a, b)
print("Dot product:", dot)

Output: 32

Calculation: (1×4) + (2×5) + (3×6) = 4 + 10 + 18 = 32

Matrix Dot Product

code.py

import numpy as np

a = np.array([[1, 2], [3, 4]])
b = np.array([[5, 6], [7, 8]])

result = np.dot(a, b)
print(result)

Same as matrix multiplication for 2D arrays.

Transpose

Swap rows and columns.

code.py

import numpy as np

matrix = np.array([[1, 2, 3], [4, 5, 6]])
transposed = matrix.T
print("Original:")
print(matrix)
print("Transposed:")
print(transposed)

Output:

Original:
[[1 2 3]
 [4 5 6]]

Transposed:
[[1 4]
 [2 5]
 [3 6]]

Identity Matrix

Special matrix with 1s on diagonal, 0s elsewhere.

code.py

import numpy as np

identity = np.eye(3)
print(identity)

Output:

[[1. 0. 0.]
 [0. 1. 0.]
 [0. 0. 1.]]

Property: Any matrix × identity = same matrix.

Determinant

A number that tells you properties of a matrix.

code.py

import numpy as np

matrix = np.array([[1, 2], [3, 4]])
det = np.linalg.det(matrix)
print("Determinant:", det)

Output: -2.0

What determinant tells you:

0 = matrix is singular (can't be inverted)
Non-zero = matrix has inverse

Matrix Inverse

code.py

import numpy as np

matrix = np.array([[1, 2], [3, 4]])
inverse = np.linalg.inv(matrix)
print("Inverse:")
print(inverse)

verify = matrix @ inverse
print("Verify (should be identity):")
print(np.round(verify))

Output:

Inverse:
[[-2.   1. ]
 [ 1.5 -0.5]]

Verify:
[[1. 0.]
 [0. 1.]]

What inverse means: Matrix × Inverse = Identity

Solving Linear Equations

Solve systems like:

2x + 3y = 8
4x + 5y = 14

code.py

import numpy as np

A = np.array([[2, 3], [4, 5]])
b = np.array([8, 14])

solution = np.linalg.solve(A, b)
print("Solution:", solution)
print("x =", solution[0])
print("y =", solution[1])

verify = A @ solution
print("Verify:", verify)

Output:

Solution: [1. 2.]
x = 1.0
y = 2.0
Verify: [8. 14.]

Eigenvalues and Eigenvectors

Important in machine learning and physics.

code.py

import numpy as np

matrix = np.array([[4, 2], [1, 3]])
eigenvalues, eigenvectors = np.linalg.eig(matrix)

print("Eigenvalues:", eigenvalues)
print("Eigenvectors:")
print(eigenvectors)

Use cases:

Principal Component Analysis (PCA)
Stability analysis
Quantum mechanics

Norms

Measure size or length of vectors.

Vector Norm

code.py

import numpy as np

vector = np.array([3, 4])
norm = np.linalg.norm(vector)
print("Norm:", norm)

Output: 5.0

Calculation: √(3² + 4²) = √25 = 5

Matrix Norm

code.py

import numpy as np

matrix = np.array([[1, 2], [3, 4]])
norm = np.linalg.norm(matrix)
print("Matrix norm:", norm)

Trace

Sum of diagonal elements.

code.py

import numpy as np

matrix = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
trace = np.trace(matrix)
print("Trace:", trace)

Output: 15

Calculation: 1 + 5 + 9 = 15

Rank

Number of independent rows/columns.

code.py

import numpy as np

matrix = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
rank = np.linalg.matrix_rank(matrix)
print("Rank:", rank)

What rank tells you: How much independent information matrix has.

Practice Example

The scenario: Solve a real-world linear system.

Problem: A store sells apples and oranges.

Day 1: 3 apples + 2 oranges = 8 dollars
Day 2: 4 apples + 5 oranges = 17 dollars
Find: Price of each fruit

code.py

import numpy as np

print("Fruit Pricing Problem")
print("=" * 40)

A = np.array([[3, 2], [4, 5]])
b = np.array([8, 17])

print("System of equations:")
print("3x + 2y = 8")
print("4x + 5y = 17")
print()

prices = np.linalg.solve(A, b)
apple_price = prices[0]
orange_price = prices[1]

print("Solution:")
print("Apple price:", apple_price)
print("Orange price:", orange_price)
print()

verification1 = 3 * apple_price + 2 * orange_price
verification2 = 4 * apple_price + 5 * orange_price

print("Verification:")
print("Day 1:", verification1, "should be 8")
print("Day 2:", verification2, "should be 17")
print()

det = np.linalg.det(A)
print("Determinant:", det)
print("System is solvable:", det != 0)

What this solves: Uses linear algebra to find unknown prices.

Matrix Power

Multiply matrix by itself n times.

code.py

import numpy as np

matrix = np.array([[1, 2], [3, 4]])
squared = np.linalg.matrix_power(matrix, 2)
print("Matrix squared:")
print(squared)

Same as: matrix @ matrix

Outer Product

code.py

import numpy as np

a = np.array([1, 2, 3])
b = np.array([4, 5])

outer = np.outer(a, b)
print(outer)

Output:

[[4 5]
 [8 10]
 [12 15]]

Key Points to Remember

Use @ or np.matmul() for true matrix multiplication, * for element-wise.

np.linalg module has all linear algebra functions: det, inv, solve, eig.

Identity matrix (np.eye) is like number 1 for matrices. matrix @ inverse = identity.

np.linalg.solve() solves linear equations efficiently. Don't calculate inverse manually.

Eigenvalues and eigenvectors are used in PCA and machine learning.

Common Mistakes

Mistake 1: Wrong multiplication

code.py

a * b  # Element-wise
a @ b  # Matrix multiplication

Mistake 2: Incompatible shapes

code.py

a = np.array([[1, 2]])  # (1, 2)
b = np.array([[1], [2], [3]])  # (3, 1)
a @ b  # Error! 2 ≠ 3

Mistake 3: Inverting singular matrix

code.py

matrix = np.array([[1, 2], [2, 4]])
np.linalg.inv(matrix)  # Error! Determinant is 0

Check determinant first.

Mistake 4: Using inverse instead of solve

code.py

x = np.linalg.inv(A) @ b  # Slow and inaccurate
x = np.linalg.solve(A, b)  # Better!

What's Next?

Congratulations! You've completed Module 4: NumPy for Numerical Computing. You now know how to work with arrays, perform mathematical operations, do statistics, and apply linear algebra - essential skills for data science and machine learning.