An option ineigshowdemonstrates SVD, the matrix singular value decomposition. The emphasis is on orthogonality.

Contents

The default matrix

This is the third posting about one of my favorite MATLAB programs,eigshow. The first two postings were about the'eig'option for eigenvalues. This posting is about the'svd'option for the singular value decomposition.

The default matrix, shown in the figure window title, is

formatrat, formatcompactA = [1 3; 4 2]/4

A = 1/4 3/4 1 1/2

With the'svd'option, you have two green vectors,xandy, that are perpendicular to each other and that have unit length. They remain perpendicular as you movexaround the unit circle with the mouse, so they always form an orthonormal basis for two-dimensional space.

The two blue vectors,AxandAy, are images ofxandyunder the mapping induced by the matrixA. The angle between them changes and their lengths change as you move around the space. The instructions at the bottom of the frame say

Make A*x perpendicular to A*y

The SVD

When you reach the configuration where not onlyxis perpendicular toybut alsoAxis perpendicular toAy, you have found the singular value decomposition. The SVD is usually described as the matrix factorization

$$ A = U \Sigma V^T $$

with orthogonal $U$ and $V$ and diagonal $\Sigma$. This can be written

$$ A V = U \Sigma $$

For each column of $V$ and the corresponding column of $U$,

$$ A v_j = \sigma_j u_j $$

We can see that whenAxis perpendicular toAy, the vectorsxandyare the columns of $V$ and the vectorsAxandAyare the columns of $U \Sigma$. The blue curve, which is the image of the unit circle under the transformation induced by $A$, is an ellipse. The vectorsAxandAyare its major and minor axes and the half lengths of these axes are the singular values of the matrix.

For this first matrix it looks like the length ofAxmight be about 1.25 and the length ofAymight be about 0.50. We can check this by actually computing the singular values.

formatshortsigma = svd(A)

sigma = 1.2792 0.4886

Existence

In our previous posts about eigenvalues, we found that real eigenvectors don't always exist. If the eigenvectors are complex,eigshowcan never find them. And if the matrix is defective,eigshow发现啊nly one eigenvector, not the usual two. The situation is quite different for SVD. The real singular value decomposition always exists. It is always possible to rotatexandyto a position that makesAxorthogonal toAy.

In the language of abstract linear algebra, we say that the green vectors are an orthonormal basis for thedomainof the transformation and, if the blue vectors were scaled to have unit length, they would be an orthonormal basis for therangeof the transformation.

A defective matrix

Our final matrix in the eigenvalues post was defective -- it had only one eigenvector. But its SVD is obtained with no difficulty.

formatratA = [6 4; -1 2]/4

A = 3/2 1 -1/4 1/2

At first it looks like the final blue vectors happen to be parallel to the display axes. This would say that the matrix $U$ in the SVD is the identity matrix. But a careful look at the final result shows slight stair-stepping inAxandAy. This is confirmed by actually computing the full SVD and observing that $U$ is not quite the identity.

formatshort[U,S,V] = svd(A)

U = -0.9991 -0.0424 -0.0424 0.9991 S = 1.8042 0 0 0.5542 V = -0.8247 -0.5655 -0.5655 0.8247

A singular matrix

Our singular example turns out to be a little tricky because the blue ellipse collapses to a one-dimensional line segment. It is hard to tell whenAxis perpendicular toAy. It turns out to be when one of them is the zero vector; in other words, whenxoryis the null vector ofA. When one ofAxorAyreaches zero, the other reaches its maximum length, the half-length of the line segment. This length is the nonzero singular value.

formatratA = [2 4; 2 4]/4

A = 1/2 1 1/2 1

Get the MATLAB code

Published with MATLAB® R2013b

Published with MATLAB® R2013b