PyTorch – How to compute Singular Value Decomposition (SVD) of a matrix?

torch.linalg.svd() computes the singular value decomposition (SVD) of a matrix or a batch of matrices. Singular value decomposition is represented as a named tuple (U, S, Vh).

• U and Vh are orthogonal for real matrix and unitary for input complex matrix.

• Vh is transpose of V when V is a real value and conjugate transpose when V is complex.

• S is always real valued even when the input is complex.

Syntax

U, S, Vh = torch.linalg.svd(A, full_matrices=True)

Parameters

• A – PyTorch tensor (matrix or batch of matrices).

• full_matrices – If True, the output is a full SVD, else a reduced SVD. Default is True.

Output

It returns a named tuple (U, S, Vh).

Steps

• Import the required library.

import torch

• Create a matrix or batch of matrices.

A = torch.randn(3,4)

• Compute the SVD of above-created matrix or batch of matrices.

U, S, Vh = torch.linalg.svd(A)

• Display U, S and Vh.

print("U:
",U)
print("S:
",S)
print("Vh:
",Vh)

Example 1

The following Python program shows how to compute the SVD of a matrix.

# import necessary library
import torch

# create a matrix
A = torch.randn(3,4)
print("Matrix:
", A)

# compute SVD
U, S, Vh = torch.linalg.svd(A)

# print U, S, and Vh
print("U:
",U)
print("S:
",S)
print("Vh:
",Vh)

Output

Matrix:
 tensor([[-1.5122, -0.4714, -0.1173, -0.3914],
    [ 0.4288, -1.9329, 0.9171, -1.0288],
    [ 0.1143, 0.1989, 0.3290, 0.3031]])
U:
 tensor([[ 0.1769, 0.9716, 0.1569],
    [ 0.9815, -0.1860, 0.0448],
    [-0.0728, -0.1460, 0.9866]])
S:
 tensor([2.4383, 1.6226, 0.4119])
Vh:
 tensor([[ 0.0595, -0.8182, 0.3508, -0.4516],
    [-0.9649, -0.0787, -0.2050, -0.1438],
    [-0.2554, 0.0864, 0.8433, 0.4650],
    [ 0.0092, -0.5629, -0.3519, 0.7478]])

Example 2

The following Python program shows how to compute the SVD of a complex matrix.

# import necessary library
import torch

# create a matrix of complex random number
A = torch.randn(2,2, dtype = torch.cfloat)
print("Complex Matrix:
", A)

# compute SVD
U, S, Vh = torch.linalg.svd(A)

# print U, S, and Vh
print("U:
",U)
print("S:
",S)
print("Vh:
",Vh)

Output

Complex Matrix:
 tensor([[-0.2761-0.6619j, -1.4248-0.3026j],
    [-0.2797+0.2036j, 0.2143+1.3459j]])
U:
 tensor([[-0.2670-0.7083j, 0.3372+0.5597j],
    [-0.4943+0.4273j, -0.4737+0.5905j]])
S:
 tensor([2.1358, 0.2259])
Vh:
tensor([[ 0.3595+0.0000j, 0.4981-0.7891j],
    [-0.9332+0.0000j, 0.1919-0.3040j]])

Example 3

The following Python program shows how to compute the SVD of a batch of three matrices.

# import necessary library
import torch

# create a batch of three 2x3 matrices
A = torch.randn(3,2,3)
print("Matrices:
", A)

# compute SVD
U, S, Vh = torch.linalg.svd(A)

#print U, S, and Vh
print("U:
",U)
print("S:
",S)
print("Vh:
",Vh)

Output

Matrices:
 tensor([[[ 0.2195, -1.3015, -1.0770],
    [-0.5884, -0.8269, 0.0135]],
    [[ 1.0753, -1.7080, -0.3692],
    [-1.3024, 0.2581, -1.2018]],
    [[-0.3576, -1.0531, -0.6192],
    [ 0.8453, 0.4187, -0.1622]]])
U:
 tensor([[[ 0.9242, -0.3818],
    [ 0.3818, 0.9242]],
    [[ 0.8178, 0.5755],
    [-0.5755, 0.8178]],
    [[ 0.8604, 0.5097],
    [-0.5097, 0.8604]]])
S:
 tensor([[1.8131, 0.8030],
    [2.2789, 1.4912],
    [1.4146, 0.7317]])
Vh:
 tensor([[[-0.0120, -0.8376, -0.5462],
    [-0.7815, -0.3329, 0.5276],
    [ 0.6237, -0.4332, 0.6506]],
    [[ 0.7148, -0.6781, 0.1710],
    [-0.2993, -0.5176, -0.8015],
    [-0.6321, -0.5217, 0.5730]],
    [[-0.5221, -0.7913, -0.3182],
    [ 0.7448, -0.2413, -0.6221],
    [ 0.4155, -0.5617, 0.7154]]])