Section 6.1

Orthogonality

>

>

Introduction

The notion of orthogonal projection arises in many applications. For example, in physics

when studying the motion of a moving particle along an inclined plane, we analyze the motion

by decomposing the force acting on the particle into a component along the direction, u, of the

moving particle and a component orthogonal to direction u. In the absence of a solution to a

system of linear equations Ax = b , one may be interested in constructing an approximate solution

to the system. The notion of orthogonal projection plays an important role in this construction.

Orthogonal Projection of b on span{a}

Let us begin with the following problem: Given two vectors a and b. How do we construct vector p

along a that is orthogonal to w = b-p.

> plottriangle3();

[Maple Plot]

>

Consider the two vectors

> a:=vector([a1,a2]); b:=vector([b1,b2]);

a := vector([a1, a2])

b := vector([b1, b2])

>

The vector, p , is the "orthogonal projection" of b upon a . Since vector p is along the

same line as vector a , it follows that p is a multiple of a ; that is, p = t*a where t is a scalar

to be determined subject to the condition that p is orthogonal to the vector b - p ; that is,

innerprod( p , b-p ) = innerprod( t*a , b-t*a ) = 0

This is equivalent to:

> t:=innerprod(a,b)/innerprod(a,a);

t := (a1*b1+a2*b2)/(a1^2+a2^2)

Vector p is then given by

> p:=evalm(t*a);

p := vector([(a1*b1+a2*b2)/(a1^2+a2^2)*a1, (a1*b1+a...

>

and vector w is

> w:=evalm(b-p);

w := vector([b1-(a1*b1+a2*b2)/(a1^2+a2^2)*a1, b2-(a...

Is p orthogonal to w? Compute the inner product of p and w

> innerprod(p,w);

0

>

Vector p is called the orthogonal projection of b upon a , t is the scalar projection

and w is called the orthogonal complement .

Example 1: Consider the vectors

> a:=vector([7,6]); b:=vector([4,2]);

a := vector([7, 6])

b := vector([4, 2])

Compute the orthogonal projection of b upon a .

Compute the scalar projection of u on v

> t:=innerprod(b,a)/innerprod(a,a);

t := 8/17

Therefore, the orthogonal projection p of b on a is

> p:=evalm(t*a);

p := vector([56/17, 48/17])

The component of u orthogonal to v is

> w:=evalm(b-p);

w := vector([12/17, -14/17])

>

Vector p is called the orthogonal projection of b upon a and t is the scalar projection.

>

> plottriangle4();

[Maple Plot]

>

Note: if a is in R^2 then the span{ a } is a line.

*******************************************************************

In general, if b and a are two vectors in R^n , then

t = innerprod(b,a)/innerprod(a,a) is called the scalar projection of b on span{ a }

p = t*a is the orthogonal projection of b upon span{ a }

Vector w = b - p can also be referred to as the orthogonal complement of the span{ a }.

This motivates us to consider the "orthogonal complement" of a set.

w = b - p is the component of b orthogonal to span{ a }

*****************************************************************

Orthogonal Complement of a Subspace

************************************************************************

Let W be a subspace of R^n . The set of all vectors in R^n that are orthogonal to every vector in

W is the orthogonal complement of W.

*************************************************************************

Example 2: Let us consider set W spanned by the vectors

> w1:=vector([1,0,0]); w2:=vector([0,1,0]);

w1 := vector([1, 0, 0])

w2 := vector([0, 1, 0])

Describe analytically the orthogonal complement of the set W. This is the set of all vectors

> v:=vector([x,y,z]);

v := vector([x, y, z])

>

orthogonal to w[1] and w[2] .

To determine this set, we compute

> innerprod(v,w1)=0; innerprod(v,w2)=0;

x = 0

y = 0

>

Therefore the orthogonal complement is W = {[0,0,z] | z in R}. Geometrically , this set is

the z-axis.

*********************************************************************

Finding the orthogonal complement of a subspace W of R^n

1.) Find a matrix A having row vectors a generating set for W.

2.) Find the nullspace of A, that is, the solution space to Ax=0. This

nullspace is the orthogonal complement of W.

**********************************************************************

Example 3: Compute the orthogonal complement of

> W =`span`([1,-1],[1,0]);

W = span([1, -1],[1, 0])

>

Create the matrix A.

> A:=matrix([[1,-1],[1,0]]);

A := matrix([[1, -1], [1, 0]])

>

Orthogonal complement of the row space:

> nullspace(A);

{}

Example 3: Compute the orthogonal complement of

> W =`span`(matrix(3,1,[1,-1,1]),matrix(3,1,[1,1,0]));

W = span(matrix([[1], [-1], [1]]),matrix([[1], [1],...

Create the matrix A.

> A:=matrix([[1,-1,1],[1,1,0]]);

A := matrix([[1, -1, 1], [1, 1, 0]])

>

Orthogonal complement of the row space :

> nullspace(A);

{vector([-1, 1, 2])}

>

*********************************************************************************

Theorem 6.1:

The orthogonal complement W^per of a subspace W of R^n has the following properties:

1.) W^per is a subspace of R^n

2.) dim( W^per ) = n -dim(W)

3.) (W^per)^per =W that is the orthogonal complement of W^per is W

4.) Each vector b in R^n can be expressed uniquely in the form b= b[W]+b[W^per] for

b[W] in W and b[W^per] in W^per

**********************************************************************************

***********************************************************************************

1.) b[W] is in the subspace W

2.) b[W^per] is orthogonal to every vector in W and is in the space W^per

3.) Let w be any vector in W. Then || b- w || >= || b - b[W] || that is the

vector b[W] is the closest vector in W to b.

Note: b[W] is the projection of b on W.

***********************************************************************************

*******************************************************************************

Steps to find the projection of b on W.

1.) Select a basis { v[1], v[2], v[3] , . . ., v[k] } for the subspace W. (Often this already given.)

2.) Find a basis { v[k+1], v[k+2] , . . ., v[n] } for W^per . Compute the nullspace of A which has as

rows the vectors v[1], v[2], v[3] , . . ., v[k] .

3.) Find the coordinate vector r = [ r[1], r[2] , . . . , r[n] ] of b relative to the basis ( v[1], v[2], v[3] , . . ., v[n] )

so that b = r[1]*v[1]+r[2]*v[2] + . . . + r[n]*v[n] .

4.) Then b[W] = r[1]*v[1]+r[2]*v[2] + . . . + r[k]*v[k]

*******************************************************************************

Example 4: Find the projection of

> b := vector([2,1,5]);

b := vector([2, 1, 5])

>

on the subspace

> W = `span`([1,2,1],[2,1,-1]);

W = span([1, 2, 1],[2, 1, -1])

>

Step1: Find a basis for W.

> A := augment([1,2,1],[2,1,-1]);

A := matrix([[1, 2], [2, 1], [1, -1]])

> rref(A);

matrix([[1, 0], [0, 1], [0, 0]])

>

Step 2: Find a basis for W^per

> transpose(A);

matrix([[1, 2, 1], [2, 1, -1]])

> nullspace(transpose(A));

{vector([1, -1, 1])}

>

Step3: Basis for R^3 is given as,

> v1 := vector([1,2,1]); v2 := vector([2,1,-1]); v3:=vector([1,-1,1]);

v1 := vector([1, 2, 1])

v2 := vector([2, 1, -1])

v3 := vector([1, -1, 1])

Set-up the linear system to find the scalars r[1], r[2], r[3] . Remember v[1], v[2] are

the basis for W and v[3] is the basis for W^per . Ordered basis!!!

> augment(v1,v2,v3)*matrix(3,1,[r[1],r[2],r[3]]) = matrix(3,1,[2,1,5]);

matrix([[1, 2, 1], [2, 1, -1], [1, -1, 1]])*matrix(...

> A := augment(v1,v2,v3,b);

A := matrix([[1, 2, 1, 2], [2, 1, -1, 1], [1, -1, 1...

> rref(A);

matrix([[1, 0, 0, 2], [0, 1, 0, -1], [0, 0, 1, 2]])...

Thus, we have

> r := vector([2,-1,2]);

r := vector([2, -1, 2])

>

and the projection of b on W is,

> b_W := evalm(2*v1 -1*v2);

b_W := vector([0, 3, 3])

>

and the projection of b on W^per is,

> 2*v3 = vector([2,-2,2]);

2*v3 = vector([2, -2, 2])

>

Notice that b[W]+b[W^per] = b = [2,1,5].

Exercise

1-21 (odd).