## Vladimir Dobrushkin

http://math.uri.edu/~dobrush/

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the appendix entitled GNU Free Documentation License.

# Singular Value Decomposition

A **basis** β for a vector space *V* is a linearly independent subset of *V*
that generates

or span *V*. If β is a basis for *V*, we also say that elements of β form a basis for
*V*.

This means that every vector from *V* is a finite linear combination of elements from the basis.

Recall that a set of vectors β is said to generate or span a vector space *V* if every element from
*V* can be represented as a linear combination of vectors from β.

*n*-dimensional real space, and it is called the

**standard basis**. Its dimension is

*n*.

*i*-th row and

*j*-th column. Then the set \( {\bf M}_{i,j} \ : \ 1 \le i \le m , \ 1 \le j \le n \) is a basis for the set of all such real matrices. Its dimension is

*mn*.

*n*. It has dimension

*n*+1. ■

Theorem: Let *V* be a vector space and
\( \beta = \left\{ {\bf u}_1 , {\bf u}_2 , \ldots , {\bf u}_n \right\} \) be a subset of
*V*. Then β is a basis for *V* if and only if each vector *v* in *V* can be uniquely
decomposed into a linear combination of vectors in β, that is, can be uniquely expressed in the form

If the vectors \( \left\{ {\bf u}_1 , {\bf u}_2 , \ldots , {\bf u}_n \right\} \)
form a basis for a vector space *V*, then every vector in *V* can be uniquely expressed in the form

**v**determines a unqiue

*n*-tuple of scalars \( \left[ \alpha_1 , \alpha_2 , \ldots , \alpha_n \right] \) and, conversely, each

*n*-tuple of scalars determines a unique vector \( {\bf v} \in V \) by using the entries of the

*n*-tuple as the coefficients of a linear combination of \( {\bf u}_1 , {\bf u}_2 , \ldots , {\bf u}_n . \) This fact suggests that

*V*is like the

*n*-dimensional vector space \( \mathbb{R}^n , \) where

*n*is the number of vectors in the basis for

*V*.

A vector space is called **finite-dimensional** if it has a basis consisting of a finite

number of elements. The unique number of elements in each basis for *V* is called

the **dimension** of *V* and is denoted by dim(*V*). A vector space that is not finite-

dimensional is called
**infinite-dimensional**.

The next example demonstrates how *Mathematica* can determine the basis or set of linearly independent
vectors from the given set. Note that basis is not unique and even changing the order of vectors, a software can provide
you another set of linearly independent vectors.

```
MatrixRank[m =
{{1, 2, 0, -3, 1, 0},
{1, 2, 2, -3, 1, 2},
{1, 2, 1, -3, 1, 1},
{3, 6, 1, -9, 4, 3}}]
```

Then each of the following scripts determine a subset of linearly independent vectors:

```
m[[ Flatten[ Position[#, Except[0, _?NumericQ], 1, 1]& /@
```

Last @ QRDecomposition @ Transpose @ m ] ]]

or, using subroutine

```
MinimalSublist[x_List] :=
```

Module[{tm, ntm, ytm, mm = x}, {tm = RowReduce[mm] // Transpose,

ntm = MapIndexed[{#1, #2, Total[#1]} &, tm, {1}],

ytm = Cases[ntm, {___, ___, d_ /; d == 1}]};

Cases[ytm, {b_, {a_}, c_} :> mm[[All, a]]] // Transpose]

we apply it to our set of vectors.
```
m1 = {{1, 2, 0, -3, 1, 0}, {1, 2, 1, -3, 1, 2}, {1, 2, 0, -3, 2,
1}, {3, 6, 1, -9, 4, 3}};
```

MinimalSublist[m1]

`{{1, 1, 1, 3}, {0, 1, 0, 1}, {1, 1, 2, 4}}`

One can use also the standard *Mathematica*command:

**IndependenceTest**. ■