**Section 6.1**

**Orthogonality**

`> `

`> `

**Introduction**

The notion of orthogonal projection arises in many applications. For example, in physics

when studying the motion of a moving particle along an inclined plane, we analyze the motion

by decomposing the force acting on the particle into a component along the direction, u, of the

moving particle and a component orthogonal to direction u. In the absence of a solution to a

system of linear equations , one may be interested in constructing an approximate solution

to the system. The notion of orthogonal projection plays an important role in this construction.

**Orthogonal Projection of b on span{a}**

Let us begin with the following problem: Given two vectors
**a**
and
**b. **
How do we construct vector
**p**

along
**a**
that is orthogonal to
**w = b-p.**

`> `
**plottriangle3();**

`> `

Consider the two vectors

`> `
**a:=vector([a1,a2]); b:=vector([b1,b2]); **

`> `

The vector,
**p**
, is the "orthogonal projection" of
**b**
upon
**a**
. Since vector
**p**
is along the

same line as vector
** a**
, it follows that
**p**
is a multiple of
**a **
; that is,
where
**t**
is a scalar

to be determined subject to the condition that
**p**
is orthogonal to the vector
**b - p**
; that is,

innerprod( , ) = innerprod( , ) = 0

This is equivalent to:

`> `
**t:=innerprod(a,b)/innerprod(a,a);**

Vector p is then given by

`> `
**p:=evalm(t*a);**

`> `

and vector w is

`> `
**w:=evalm(b-p);**

Is p orthogonal to w? Compute the inner product of p and w

`> `
**innerprod(p,w);**

`> `

Vector
**p**
is called the
orthogonal projection
of
**b**
upon
** a**
,
**t**
is the
scalar projection
** **

and
**w**
is called the
orthogonal complement
.

**Example 1:**
Consider the vectors

`> `
**a:=vector([7,6]); b:=vector([4,2]);**

Compute the orthogonal projection of
**b**
upon
**a**
.

Compute the scalar projection of u on v

`> `
**t:=innerprod(b,a)/innerprod(a,a); **

Therefore, the orthogonal projection p of
**b**
on
**a**
is

`> `
**p:=evalm(t*a);**

The component of u orthogonal to v is

`> `
**w:=evalm(b-p);**

`> `

Vector
**p**
is called the orthogonal projection of
**b**
upon
**a**
and
**t**
is the scalar projection.

`> `

`> `
**plottriangle4();**

`> `

Note: if
**a**
is in
then the span{
**a**
} is a line.

*******************************************************************

In general, if
**b**
and
**a**
are two vectors in
, then

is called the
** scalar projection**
of

is the
** orthogonal projection**
of

Vector
**w = b - p**
can also be referred to as the
** orthogonal complement**
of the span{

This motivates us to consider the "orthogonal complement" of a set.

**w**
=
**b - p**
is the component of
**b **
orthogonal to span{
**a**
}

*****************************************************************

**Orthogonal Complement of a Subspace**

************************************************************************

Let W be a subspace of . The set of all vectors in that are orthogonal to every vector in

W is the orthogonal complement of W.

*************************************************************************

**Example 2:**
Let us consider set W spanned by the vectors

`> `
**w1:=vector([1,0,0]); w2:=vector([0,1,0]);**

Describe
**analytically**
the orthogonal complement of the set W. This is the set of all vectors

`> `
**v:=vector([x,y,z]); **

`> `

orthogonal to and .

To determine this set, we compute

`> `
**innerprod(v,w1)=0; innerprod(v,w2)=0;**

`> `

Therefore the orthogonal complement is W = {[0,0,z] | z in R}.
** Geometrically**
, this set is

the z-axis.

*********************************************************************

Finding the orthogonal complement of a subspace W of

1.) Find a matrix A having row vectors a generating set for W.

2.) Find the nullspace of A, that is, the solution space to Ax=0. This

nullspace is the orthogonal complement of W.

**********************************************************************

**Example 3:**
Compute the orthogonal complement of

`> `
**W =`span`([1,-1],[1,0]);**

`> `

Create the matrix A.

`> `
**A:=matrix([[1,-1],[1,0]]);**

`> `

Orthogonal complement of the row space:

`> `
**nullspace(A);**

**Example 3:**
Compute the orthogonal complement of

`> `
**W =`span`(matrix(3,1,[1,-1,1]),matrix(3,1,[1,1,0]));**

Create the matrix A.

`> `
**A:=matrix([[1,-1,1],[1,1,0]]);**

`> `

Orthogonal complement of the row space :

`> `
**nullspace(A);**

`> `

*********************************************************************************

__Theorem 6.1:__

The orthogonal complement of a subspace W of has the following properties:

1.) is a subspace of

2.) dim( ) = n -dim(W)

3.) =W that is the orthogonal complement of is W

4.) Each vector b in can be expressed uniquely in the form b= for

in W and in

**********************************************************************************

***********************************************************************************

1.) is in the subspace W

2.) is orthogonal to every vector in W and is in the space

3.) Let w be any vector in W. Then || b- w || >= || b - || that is the

vector is the closest vector in W to b.

Note: is the projection of b on W.

***********************************************************************************

*******************************************************************************

Steps to find the projection of b on W.

1.) Select a basis { , . . ., } for the subspace W. (Often this already given.)

2.) Find a basis { , . . ., } for . Compute the nullspace of A which has as

rows the vectors , . . ., .

3.) Find the coordinate vector r = [ , . . . , ] of b relative to the basis ( , . . ., )

so that b = + . . . + .

4.) Then + . . . +

*******************************************************************************

**Example 4:**
Find the projection of

`> `
**b := vector([2,1,5]);**

`> `

on the subspace

`> `
**W = `span`([1,2,1],[2,1,-1]);**

`> `

Step1: Find a basis for W.

`> `
**A := augment([1,2,1],[2,1,-1]);**

`> `
**rref(A);**

`> `

Step 2: Find a basis for

`> `
**transpose(A);**

`> `
**nullspace(transpose(A));**

`> `

Step3: Basis for is given as,

`> `
**v1 := vector([1,2,1]); v2 := vector([2,1,-1]); v3:=vector([1,-1,1]);**

Set-up the linear system to find the scalars . Remember are

the basis for W and is the basis for . Ordered basis!!!

`> `
**augment(v1,v2,v3)*matrix(3,1,[r[1],r[2],r[3]]) = matrix(3,1,[2,1,5]);**

`> `
**A := augment(v1,v2,v3,b);**

`> `
**rref(A);**

Thus, we have

`> `
**r := vector([2,-1,2]); **

`> `

and the projection of b on W is,

`> `
**b_W := evalm(2*v1 -1*v2);**

`> `

and the projection of b on is,

`> `
**2*v3 = vector([2,-2,2]);**

`> `

Notice that = b = [2,1,5].

**Exercise**

1-21 (odd).