Section 6.2

Gram-Schmidt Process

>

> with(linalg):

Warning, the protected names norm and trace have been redefined and unprotected

>

Introduction

A basis of a linear space is its building block. Every element of the linear space can

be uniquely expressed in terms of the basis. In the Euclidean space R^n , it is easy to express

a vector in terms of the standard basis. However, the process can be very tedious if the

basis is not the standard one. Another special and useful basis is the "orthonormal basis".

Every vector can be easily expressed in terms of this orthonormal basis.

Orthogonal and Orthonormal Bases

******************************************************************

Let { v[1], v[2] ,..., v[n] } in R^n be an orthogonal set of non-zero vectors, i.e

innerprod( v[i], v[j] ) = 0 for all i, j ( i not equal to j) . Then

this set is independent and consequently is a basis for the subspace

span{ v[1], v[2] ,..., v[n] }.

*******************************************************************

*******************************************************************

Theorem 6.3

Let { v[1], v[2] ,..., v[k] } be an orthogonal basis for a subspace W of R^n , and let

b be any vector in R^n . The projection of b on W is

b[W] = b*v[1]/(v[1]*v[1]) v[1] + b*v[2]/(v[2]*v[2]) v[2] + . . . + b*v[k]/(v[k]*v[k]) v[k]

********************************************************************

Example 1: Let B = {u,v} be an orthogonal basis for R^2

>

> u:=vector([-2,3]); v:=vector([3,2]);

u := vector([-2, 3])

v := vector([3, 2])

> innerprod(u,v);

0

>

The two vectors u and v are orthogonal.

Let w be a vector in R^2

> w:=vector([-1,1]);

w := vector([-1, 1])

>

If w = c[1]*u+c[2]*v , then the inner product of w with u and with v respectively yields

>

> c1:=innerprod(w,u)/innerprod(u,u);

c1 := 5/13

> c2:=innerprod(w,v)/innerprod(v,v);

c2 := -1/13

> w := c1*evalm(u) + c2*evalm(v);

w := 5/13*vector([-2, 3])-1/13*vector([3, 2])

>

Example 2: Consider a subspace W of R^3 consisting of the vectors

>

> u1:=vector([-1,1,1]); u2:=vector([2,1,1]);

u1 := vector([-1, 1, 1])

u2 := vector([2, 1, 1])

>

> W = `span`(evalm(u1),evalm(u2));

W = span(vector([-1, 1, 1]),vector([2, 1, 1]))

>

Do the vectors u[1] and u[2] form an orthogonal basis for W? Form the inner product:

>

> innerprod(u1,u2);

0

>

Since the set is orthogonal we can quickly compute b[W] .

For example if b is the vector

>

> b :=vector([-1,2,3]);

b := vector([-1, 2, 3])

>

then we can find the following c[1] and c[2]

>

> c1:=innerprod(b,u1)/innerprod(u1,u1);

c1 := 2

> c2:= innerprod(b,u2)/innerprod(u2,u2);

c2 := 1/2

>

Thus the projection vector b[W] is equal to

> c1*evalm(u1)+ c2*evalm(u2);

2*vector([-1, 1, 1])+1/2*vector([2, 1, 1])

>

It is sometimes desirable to normalize the vectors in an orthogonal basis. That is change the

length of each vector to be 1. Then all vectors in the orthogonal basis are unit vectors.

******************************************************************

An orthonormal basis { v[1], v[2] ,...., v[k] } of a subspace W of R^n is a basis in which the

elements are mutually orthogonal unit vectors. If { v[1], v[2] ,..., v[k] } is an orthonormal basis

for W and if b is any vector in W, then

b[W] = c[1]*v[1]+c[2]*v[2]+c[3]*v[3] +....+ c[k]*v[k]

where c[i] = innerprod(b,v[i]) .

******************************************************************

Example 3 Consider the orthogonal set of vectors

>

> u1:=vector([1,1,-4]); u2:=vector([2,2,1]);

u1 := vector([1, 1, -4])

u2 := vector([2, 2, 1])

> W = `span`(evalm(u1),evalm(u2));

W = span(vector([1, 1, -4]),vector([2, 2, 1]))

>

The length of each vector is

> Lu1:=sqrt(innerprod(u1,u1));

Lu1 := 3*sqrt(2)

> Lu2:=sqrt(innerprod(u2,u2));

Lu2 := 3

>

Dividing each vector by its length yields three mutually orthogonal unit vectors:

>

> w1:=evalm(u1/Lu1);

w1 := vector([1/6*sqrt(2), 1/6*sqrt(2), -2/3*sqrt(2...

> w2:=evalm(u2/Lu2);

w2 := vector([2/3, 2/3, 1/3])

>

Does this set form a basis for W ? This set is linearly independent.Hence, it is a basis for W .

Recall, a basis consisting of mutually orthogonal unit vectors is called an orthonormal basis .

Consider the vector

>

> b :=vector([-5,56,72]);

b := vector([-5, 56, 72])

>

Compute b[W] = c[1]*w[1]+c[2]*w[2] .

>

> c1:=innerprod(b,w1);

c1 := -79/2*sqrt(2)

> c2:=innerprod(b,w2);

c2 := 58

>

Therefore b[W] is equal to

>

> c1*evalm(w1)+ c2*evalm(w2);

-79/2*sqrt(2)*vector([1/6*sqrt(2), 1/6*sqrt(2), -2/...

>

As we see having an orthonormal basis makes calculations a lot easier. We now

describe how to change a linearly independent set of vectors into an orthonormal

set of vectors.

Gram-Schmidt Process

Step 0 . Check whether the given set is linearly independent. If it is independent ,

proceed to construct an orthonormal basis; otherwise, extract an independent set and

then proceed. Now assume the set W = { a[1], a[2] ,...., a[k] }is independent;

Step 1 . Construct the first vector

v[1] = a[1] / || a[1] ||

Step 2 . Find the orthogonal projection of a[2] upon v[1]

p[1] = i nnerprod(a[2],v[1])*v[1]

and construct the vector

w[2] = a[2]-p[1]

Then the second member of the orthonormal basis is

v[2] = w[2] / || w[2] ||

Step 3 . Find the projection of a[3] onto the span( v[1], v[2] )

p[2] = innerprod(a[3],v[2])*v[2] + innerprod(a[3],v[1])*v[1]

and construct the vector

w[3] = a[3]-p[2]

The third element in the orthonormal basis is

v[3] = w[3] / || w[3] ||

Step 4 Repeat the process until all the v's of the original set have been exhausted.

The set { v[1], v[2], v[3] ,.... } is an orthonormal basis.

Learning the Process

Here is an example

>

> a1:=vector([1,0,1]);

a1 := vector([1, 0, 1])

> a2:=vector([0,1,2]);

a2 := vector([0, 1, 2])

> a3:=vector([2,1,0]);

a3 := vector([2, 1, 0])

>

Step 0: Are the vectors linearly independent?

>

> A := augment(a1,a2,a3);

A := matrix([[1, 0, 2], [0, 1, 1], [1, 2, 0]])

>

> rref(A);

matrix([[1, 0, 0], [0, 1, 0], [0, 0, 1]])

>

>

Step 1: Construct the first vector.

>

> v1 := evalm(a1/sqrt(innerprod(a1,a1)));

v1 := vector([1/2*sqrt(2), 0, 1/2*sqrt(2)])

>

Step 2: Find the orthogonal projection of a[2] upon v[1]

>

> p1 := evalm(innerprod(a2,v1)*v1);

p1 := vector([1, 0, 1])

>

and construct the vector (Note: this vector is orthogonal to v[1] .)

>

> w2 := evalm(a2-p1);

w2 := vector([-1, 1, 1])

>

Then the second member of the orthonormal basis is

>

> v2 := evalm(w2/sqrt(innerprod(w2,w2)));

v2 := vector([-1/3*sqrt(3), 1/3*sqrt(3), 1/3*sqrt(3...

>

Step 3 . Find now the projection of a[3] onto the span( v[1], v[2] )

>

> p2 := evalm(innerprod(a3,v2)*v2 + innerprod(a3,v1)*v1);

p2 := vector([4/3, -1/3, 2/3])

>

and construct the vector (Note: this vector is orthogonal to v[1] and v[2] .)

>

> w3 := evalm(a3-p2);

w3 := vector([2/3, 4/3, -2/3])

>

The third element in the orthonormal basis is

>

> v3 := evalm(w3/sqrt(innerprod(w3,w3)));

v3 := vector([1/6*sqrt(6), 1/3*sqrt(6), -1/6*sqrt(6...

>

> W = `span`(evalm(a1),evalm(a2),evalm(a3));

W = span(vector([1, 0, 1]),vector([0, 1, 2]),vector...

>

Orthonormal basis,

> `span`(evalm(v1),evalm(v2),evalm(v3));

span(vector([1/2*sqrt(2), 0, 1/2*sqrt(2)]),vector([...

>

Exercises

1-13(odd), 17, 19.