AuthorJing Wu2018-06-11

Introduction

Many papers in statistics and quantitative finance make heavy use of linear algebra, so you need to have a working knowledge of it in order to read and apply them to your trading.

Vectors

A vector can be thought of as an arrow pointing from the origin to a specific point. Any vector or point can be represented by its coordinates i.e. an array of numbers, such as for a 2-dimensional vector, or for a 3-dimensional one. We usually write a vector as a column:

The scalar product of two vectors and in 2-dimensional space is defined as:

This definition can be easily generalized to n dimensional space. Clearly, we cannot take the scalar product of two vectors with different dimensions.

Matrices

If we have a few vectors with the same dimension, then we can put them side-by-side to form a matrix. For example, the vectors

can be combined to produce a matrix:

m is a 3 × 3 matrix. We typically describe the dimensions of a matrix as where m = number of rows and n = number of columns.

A square matrix is one with as many rows as columns.

Notation: refers to a specific value in row and column of a matrix . For example, is the number in the second row and third column of .

Python Implementation

In Python, the NumPy package deals with linear algebra. The array we learned in the NumPy chapter can be deemed as a vector:

import numpy as np
a = np.array([1,2,3])
b = np.array([2,2,2])
c = np.array([3,1,1])
matrix = np.column_stack((a,b,c))
print(matrix)
print(type(matrix))
[out]:
[[1 2 3]
 [2 2 1]
 [3 2 1]]
[out]: <class 'numpy.ndarray'>

It is worth noticing that we used column_stack() here to ensure that the vectors are vertical and placed side-by-side to form a matrix. Without the column_stack() function, the vectors will be made horizontal and stacked on top of one another:

matrix2 = np.array([a,b,c])
print(matrix2)
[out]:
[[1 2 3]
 [2 2 2]
 [3 1 1]]

Matrix Multiplication

How are two matrices multiplied? Suppose . Each entry of matrix is the scalar product of row from matrix with column from matrix . This is best illustrated with an example:

In NumPy, we can multiply matrices with the dot() function:

A = np.array([[2,3],[4,2],[2,2]])
B = np.array([[4,2],[4,6]])
x = np.dot(A,B)
print x
[out]:
[[20 22]
 [24 20]
 [16 16]]

Since matrix multiplication is defined in terms of scalar products, the matrix product exists only if has as many columns as has rows. It's useful to remember this shorthand: (m × n) × (n × p) = (m × p) which means that an (m × n) matrix multiplied by an (n × p) matrix yields an (m × p) matrix.

Reversing the order of multiplication results in an error since B does not have as many columns as A has rows:

x = np.dot(B,A)

A natrual consequence of this fact is that matrix multiplication is not commutative. In other words, in general.

Inverse

An identity matrix is a square matrix with ones on the main diagonal and zeros elsewhere. Here is an identity matrix:

Multiplying any matrix by an identity matrix (of the correct shape) is like multiplying a number by 1. Concretely, if is an matrix, then:

is the inverse matrix of a square matrix if:

Some caveats:

  • A rectangular matrix will not have an inverse, but it may have a pseudoinverse (not covered in this tutorial).
  • A square matrix may not have an inverse i.e. it may be "singular".
  • If a square matrix has an inverse, then its inverse is unique.

Inverse matrices are computed using the Gauss-Jordan method. In NumPy, we use the linalg.inv() function to do it:

print(matrix)
print('\n-------------------------\n')
print(np.linalg.inv(matrix))
[out]:
[[1 2 3]
 [2 2 1]
 [3 2 1]]

-------------------------

[[ 0.   -1.    1.  ]
 [-0.25  2.   -1.25]
 [ 0.5  -1.    0.5 ]]

Now let's check if the multiplication is :

inverse = np.linalg.inv(matrix)
print(np.dot(matrix, inverse))
print('\n-------------------------\n')
print(np.dot(inverse,matrix))
[out]:
[[  1.00000000e+00  -6.66133815e-16   6.66133815e-16]
 [  0.00000000e+00   1.00000000e+00   1.11022302e-16]
 [  0.00000000e+00  -2.22044605e-16   1.00000000e+00]]

-------------------------

[[  1.00000000e+00  -4.44089210e-16  -2.22044605e-16]
 [  6.66133815e-16   1.00000000e+00   0.00000000e+00]
 [  0.00000000e+00   0.00000000e+00   1.00000000e+00]]

Not surprisingly, we ended up with an identity matrix. We can form a non-invertible matrix by making one of its rows a multiple of another:

singular = np.array([[1,2,3],[1,2,3],[3,3,3]])
inv = np.linalg.inv(singular)   
[out]: numpy.linalg.linalg.LinAlgError: Singular matrix

Linear Equations

A common problem in linear algebra is solving linear equations. Consider the following linear equations:

If we let:

Then the linear equations above can be written as

If A is invertible, then we can multiply on both sides of the equation to obtain the solution:

As long as exists, we can compute it to solve the linear equations:

A = np.array([[2,1,-1],[-3,-1,2],[-2,1,2]])
b = np.array([[8],[-11],[-3]])
inv_A = np.linalg.inv(A)
print np.dot(inv_A, b)
[out]:
[[ 2.]
 [ 3.]
 [-1.]]

The solution is x = 2, y = 3, z = −1. However, computing the inverse matrix is not recommended, since it is numerically unstable i.e. small rounding errors can dramatically affect the result.

Instead, NumPy solves linear equations by LU decomposition:

print np.linalg.solve(A, b)
[out]:
[[ 2.]
 [ 3.]
 [-1.]]

Of course, we get the same solution. We can check the correctness of the solution by substituting x, y and z into the linear equations.

Summary

In this chapter we have introduced vectors, matrices, inverse matrices and linear equations. Some applications in finance include: finding arbitrage opportunities by solving linear equations, computing portfolio variance, etc. In the next chapter, we will introduce modern portfolio theory and CAPM.



QuantConnect Logo

Try the world leading quantitative analysis platform today

Sign Up

Previous: Multiple Linear Regression Next: Modern Portfolio Theory