Section 6.1

Orthogonality

>

>

Introduction

The notion of orthogonal projection arises in many applications. For example, in physics

when studying the motion of a moving particle along an inclined plane, we analyze the motion

by decomposing the force acting on the particle into a component along the direction, u, of the

moving particle and a component orthogonal to direction u. In the absence of a solution to a

system of linear equations , one may be interested in constructing an approximate solution

to the system. The notion of orthogonal projection plays an important role in this construction.

Orthogonal Projection of b on span{a}

Let us begin with the following problem: Given two vectors a and b. How do we construct vector p

along a that is orthogonal to w = b-p.

> plottriangle3();

>

Consider the two vectors

> a:=vector([a1,a2]); b:=vector([b1,b2]);

>

The vector, p , is the "orthogonal projection" of b upon a . Since vector p is along the

same line as vector a , it follows that p is a multiple of a ; that is, where t is a scalar

to be determined subject to the condition that p is orthogonal to the vector b - p ; that is,

innerprod( , ) = innerprod( , ) = 0

This is equivalent to:

> t:=innerprod(a,b)/innerprod(a,a);

Vector p is then given by

> p:=evalm(t*a);

>

and vector w is

> w:=evalm(b-p);

Is p orthogonal to w? Compute the inner product of p and w

> innerprod(p,w);

>

Vector p is called the orthogonal projection of b upon a , t is the scalar projection

and w is called the orthogonal complement .

Example 1: Consider the vectors

> a:=vector([7,6]); b:=vector([4,2]);

Compute the orthogonal projection of b upon a .

Compute the scalar projection of u on v

> t:=innerprod(b,a)/innerprod(a,a);

Therefore, the orthogonal projection p of b on a is

> p:=evalm(t*a);

The component of u orthogonal to v is

> w:=evalm(b-p);

>

Vector p is called the orthogonal projection of b upon a and t is the scalar projection.

>

> plottriangle4();

>

Note: if a is in then the span{ a } is a line.

*******************************************************************

In general, if b and a are two vectors in , then

is called the scalar projection of b on span{ a }

is the orthogonal projection of b upon span{ a }

Vector w = b - p can also be referred to as the orthogonal complement of the span{ a }.

This motivates us to consider the "orthogonal complement" of a set.

w = b - p is the component of b orthogonal to span{ a }

*****************************************************************

Orthogonal Complement of a Subspace

************************************************************************

Let W be a subspace of . The set of all vectors in that are orthogonal to every vector in

W is the orthogonal complement of W.

*************************************************************************

Example 2: Let us consider set W spanned by the vectors

> w1:=vector([1,0,0]); w2:=vector([0,1,0]);

Describe analytically the orthogonal complement of the set W. This is the set of all vectors

> v:=vector([x,y,z]);

>

orthogonal to and .

To determine this set, we compute

> innerprod(v,w1)=0; innerprod(v,w2)=0;

>

Therefore the orthogonal complement is W = {[0,0,z] | z in R}. Geometrically , this set is

the z-axis.

*********************************************************************

Finding the orthogonal complement of a subspace W of

1.) Find a matrix A having row vectors a generating set for W.

2.) Find the nullspace of A, that is, the solution space to Ax=0. This

nullspace is the orthogonal complement of W.

**********************************************************************

Example 3: Compute the orthogonal complement of

> W =`span`([1,-1],[1,0]);

>

Create the matrix A.

> A:=matrix([[1,-1],[1,0]]);

>

Orthogonal complement of the row space:

> nullspace(A);

Example 3: Compute the orthogonal complement of

> W =`span`(matrix(3,1,[1,-1,1]),matrix(3,1,[1,1,0]));

Create the matrix A.

> A:=matrix([[1,-1,1],[1,1,0]]);

>

Orthogonal complement of the row space :

> nullspace(A);

>

*********************************************************************************

Theorem 6.1:

The orthogonal complement of a subspace W of has the following properties:

1.) is a subspace of

2.) dim( ) = n -dim(W)

3.) =W that is the orthogonal complement of is W

4.) Each vector b in can be expressed uniquely in the form b= for

in W and in

**********************************************************************************

***********************************************************************************

1.) is in the subspace W

2.) is orthogonal to every vector in W and is in the space

3.) Let w be any vector in W. Then || b- w || >= || b - || that is the

vector is the closest vector in W to b.

Note: is the projection of b on W.

***********************************************************************************

*******************************************************************************

Steps to find the projection of b on W.

1.) Select a basis { , . . ., } for the subspace W. (Often this already given.)

2.) Find a basis { , . . ., } for . Compute the nullspace of A which has as

rows the vectors , . . ., .

3.) Find the coordinate vector r = [ , . . . , ] of b relative to the basis ( , . . ., )

so that b = + . . . + .

4.) Then + . . . +

*******************************************************************************

Example 4: Find the projection of

> b := vector([2,1,5]);

>

on the subspace

> W = `span`([1,2,1],[2,1,-1]);

>

Step1: Find a basis for W.

> A := augment([1,2,1],[2,1,-1]);

> rref(A);

>

Step 2: Find a basis for

> transpose(A);

> nullspace(transpose(A));

>

Step3: Basis for is given as,

> v1 := vector([1,2,1]); v2 := vector([2,1,-1]); v3:=vector([1,-1,1]);

Set-up the linear system to find the scalars . Remember are

the basis for W and is the basis for . Ordered basis!!!

> augment(v1,v2,v3)*matrix(3,1,[r[1],r[2],r[3]]) = matrix(3,1,[2,1,5]);

> A := augment(v1,v2,v3,b);

> rref(A);

Thus, we have

> r := vector([2,-1,2]);

>

and the projection of b on W is,

> b_W := evalm(2*v1 -1*v2);

>

and the projection of b on is,

> 2*v3 = vector([2,-2,2]);

>

Notice that = b = [2,1,5].

Exercise

1-21 (odd).