Universitext Editors (North America): S. Axler, F.W. Gehring, and K.A. Ribet Aksoy/Khamsl: Nonstandard Methods in Fixed Point Theory Andersson: Topics in Complex Analysis Aupetit: A Primer on Spectral Theory Berberian: Fundamentals of Real Analysis Booss/Bleecker: Topology and Analysis Borkar: Probability Theory: An Advanced Course Béttcher/Silhermann: Introduction to Large Truncated Toeplitz Matrices Carleson/Gamelin: Complex Dynamics Ceell: Lie Sphere Geometry: With Applications to Submanifolds Chae: Lebesgue Integration (2nd ed.) Charlap: Bieberbach Groups and Flat Manifolds Chern: Complex Manifolds Without Potential Theory Cohn: A Classical Invitation to Algebraic Numbers and Class Fields Curtis: Abstract Linear Algebra ‘Curtis: Matrix Groups DiBenedetto: Degenerate Parabolic Equations Dimca: Singularities and Topology of Hypersurfaces Edwards: A Formal Background to Mathematics | a/b Edwards: A Formal Background to Mathematics LI a/b Foulds: Graph Theory Applications Friedman: Algebraic Surfaces and Holomorphic Vector Bundles Fubrmann: A Polynomial Approach to Linear Algebra Gardiner: A First Course in Group Theory Garding/Tamhour: Algebra for Computer Science Goldhlatt: Orthogonality and Spacetime Geometry Gustafson/Rao: Numerical Range: The Field of Values of Linear Operators and Matrices Hahn: Quadratic Algebras, Clifford Algebras, and Arithmetic Witt Groups Holmgren: A First Course in Discrete Dynamical Systems Howe/Tan: Non-Abelian Harmonic Analysis: Applications of SI(2, R) Howes: Modem Analysis and Topology Hsleh/Sthuya: Basic Theory of Ordinary Differential Equations Humi/Miller: Second Course in Ordinary Differential Equations Hurwitz/Kritikos: Lectures on Number Theory Jennings: Modem Geometry with Applications Jones/Morris/Pearson: Abstract Algebra and Famous Impossibilities Kannan/Krueger: Advanced Analysis Kelly/Matthews: The Non-Euclidean Hyperbolic Plane Kostrikin: Introduction to Algebra Luecking/Rubel: Complex Analysis: A Functional Analysis Approach MacLane/Moerdijk: Sheaves in Geometry and Logic Marcus: Number Fields McCarthy: Introduction to Arithmetical Functions Meyer: Essential Mathematics for Applied Fields Mines/Richman/Ruitenhurg: A Course in Constructive Algebra Molse: Introductory Problems Course in Analysis and Topology Morris: Introduction to Game Theory Poister: A Geometrical Picture Book Porter/Woods: Extensions and Absolutes of Hausdorff SpacesFuzhen Zhang Matrix Theory Basic Results and Techniques LS SpringerFuzhen Zhang Department of Math, Science, and Technology Nova Southeastern University Fort Lauderdale, FL 33314 USA Editorial Board (North America) S. Axler F.W. Gehring Mathematics Department Mathematics Department San Francisco State University East Hall San Francisco, CA 94132 University of Michigan USA Ann Arbor, MI 48109 USA K.A. Ribet Department of Mathematics University of California at Berkeley Berkeley, CA 94720-3840 USA Mathematics Subject Classification (1991): 15-01 Library of Congress Cataloging-in-Publication Daa Zhang. Fuzhen, 1961— Matrix theory : basic resulis and techniques / Fuzhen Zhang. em. — (Universitex1) Includes bibliographical references and index. ISBN 0-387-98696-0 (he ; alk. paper) 1. Matrices. 1. Title QA188.247 1999 512.9°434—dc21 98-51754 Primed on acid-free paper. © 1999 Springer-Verlag New York Inc. All rights reserved. This work may not be iranslated or copied in whole or in part withour ihe written permission of the publisher (Springer-Verlag New York, Inc., 175 Fifth Avenue, New York, NY 10010, USA), except for brief excerpts in connection wiih reviews or scholarly analysis. Use in connection with any form of information storage and resrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is forbidden. ‘The use of general descriptive names, rade names, trademarks, etc., in this publication, even if the former are not especially identified, is noi to be 1aken as a sign thar such names, as undersiood by the Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone. Production managed by Terry Kornak; manufaciuring supervised by Jeffrey Taub. Camera-ready copy prepared by the author. Printed and bound by R. R. Donnelley and Sons, Harrisonburg, VA. Primed in the United Siaies of America. 987654321 ISBN 0-387-98696-0 Springer-Verlag New York Berlin Heidelberg SPIN 10707272To my wife Cheng, daughter Sunny, and son Andrewviii PREFACE Professor T.-G. Lei (National Natural Science Foundation of China), Professor J.-S. Li (University of Science and Technology of China), Professor R.-C. Li (University of Kentucky), Professor Z.-S. Li (Georgia State University), Professor D. Simon (Nova Southeastern University), Professor G. P. H. Styan (McGill University), Professor B.-Y. Wang (Beijing Normal University), and Professor X.-P. Zhang (Beijing Normal University). F.-Z. Zhang Ft. Lauderdale March 5, 1999 zhang@polaris.nova.edu http://www.polaris.nova.edu/~zhangContents Preface Frequently Used Notation and Terminology xi Frequently Used Theorems .. ++ xii 1 Elementary Linear Algebra Review 1 1.1 Vector Spaces ... al 1.2 Matrices .. 1.3 Linear Transformations and Eigenvalues 1,4 Inner Product Spaces 2 Partitioned Matrices 2.1 Elementary Operations of Partitioned Matrices 2.2 The Determinant and Inverse of Partitioned Matrices . 2.3 The Inverse of a Sum 2.4 The Rank of Product and Sum . 2.5 Eigenvalues of AB and BA 2.6 The Continuity Argument .. 3 Matrix Polynomials and Canonical Forms 59 3.1 Commuting Matrices .. 3.2 Matrix Decompositions 3.3. Annihilating Polynomials of Matrices ... 3.4 Jordan Canonical Forms ............. 3.5 The Matrices AT, A, A*, ATA, A*A, and AA . : 3.6 Numerical Range ........ 2.22.0. ce esses eee e cece ener eet eeeee 4 Special Types of Matrices 93 4.1 Idempotence, Nilpotence, Involution, and Projection .. 4.2 Tyidiagonal Matrices . 4.3 Circulant Matrices 4.4 Vandermonde Matrices 4.5 Hadamard Matrices . sees 4.6 Permutation and Doubly Stochastic Matrices ...x ConTENTS 5 Unitary Matrices and Contractions 131 5.1 Properties of Unitary Matrices ... 131 5.2 Real Orthogonal Matrices . .137 5.3. Metric Space and Contractions .142 5.4 Contractions and Unitary Matrices . - 148 5.5 The Unitary Similarity of Real Matrices 5.6 A Trace Inequality of Unitary Matrices . 6 Positive Semidefinite Matrices 159 6.1 Positive Semidefinite Matrices 159 6.2 A Pair of Positive Semidefinite Matrices -166 6.3 Partitioned Positive Semidefinite Matrices .. +175 6.4 Schur Complements and Determinantal Inequalities . 184 6.5 The Kronecker Product and Hadamard Product .. -190 6.6 Schur Complements and Hadamard Products ... +198 6.7 The Cauchy-Schwarz and Kantorovich Inequalities . 203 7 Hermitian Matrices 208 7.1 Hermitian Matrices + 208 7.2 The Product of Hermitian Matrices 7.3. The Min-Max Theorem and Interlacing Theorem . 7.4 Eigenvalue and Singular Value Inequalities ... 7.5 A Triangle Inequality for the Matrix (A*A)? aes 8 Normal Matrices 240 8.1 Equivalent Conditions --240 8.2 Normal Matrices with Zero and One Entries .. see 8.3 A Cauchy-Schwarz Type Inequality for Matrix (A*A)* ....255 8.4 Majorization and Matrix Normality .........-...+0eeseeeee 260 References ... 265 Notation IndexFrequently Used Notation and Terminology Mn c dim V (u,v) lel I A= (ais) rank (A) trA det A At AT A At JA [A] Ker(A) Im(A) Amax(A) Omax(A) A>0 A>B diag(A1, A2,---,An) AoB A@B Ann Xx n matrix A is upper-triangular diagonalizable similar to B unitarily similar to B unitary positive semidefinite Hermitian normal nxn matrices with complex number entries column vectors of n complex components dimension of vector space V inner product of vectors u and v norm or length of vector 2, ie., [|x| = (D; |as(?)? identity matrix matrix A with entries a:3 rank of matrix A trace of matrix A determinant of matrix A inverse of matrix A transpose of matrix A conjugate of matrix A conjugate transpose of matrix A, i.e., A* = a" determinant for a block matrix A or matrix (A*A)* principal submatrix of matrix A kernel or null of A, ie, Ker(A) = {x : Ax = 0} image space of A, i.e., Im(A) = {Az} largest eigenvalue of matrix A largest singular value of matrix A A is positive semidefinite A-— B is positive semidefinite diagonal matrix with 1, A2,-..,An on the diagonal Hadamard product of matrices A and B Kronecker product of matrices A and B said to be if all entries below the diagonal are zero if P-! AP is diagonal for some invertible matrix P if P-1 AP = B for some invertible matrix P if U"AU = B for some unitary matrix U if AA* = ATA=I if z* Az > 0 for all vectors z € C” if A= A* if A*A = AA* € C is an eigenvalue of A € M, if Az = Az for some nonzero x € C”. KNIHOVNA MAT.FYZ rarulT.Frequently Used Theorems « Cauchy-Schwarz inequality: Let V be an inner product space over a number field (R or C), Then for all 2 and y inV lew)? < @2)(y9). Equality holds if and only if z and y are linearly dependent. e Theorem on the eigenvalues of AB and BA: Let A and B be m Xn and n xm complex matrices, respectively. Then AB and BA have the same nonzero eigenvalues, counting multiplicity. Thus tr(AB) = tr(BA). e Schur triangularization theorem: For any square matrix A, there exists a unitary matrix U such that U*AU is upper-triangular. e Jordan decomposition theorem: Let A be an n-square complex matrix. Then there exists an n x 7 invertible matrix P such that A=P(N@L®---Jk)P, where each J;, i =1, 2,..., k, is a Jordan block. Spectral decomposition theorem: Let A be an n x n normal matrix with eigenvalues 1, 2, .-., An. Then there exists ann x n unitary matrix U such that A =U" diag(i 210+) An)U. . In particular, if A is positive semidefinite, then A; > 0; if A is Her- mitian, then the ; are real; and if A is unitary, then |A;| = 1. e Singular value decomposition theorem: Let A be an mx n complex matrix with rank r. Then there exist an m x m unitary matrix U and ann x n unitary matrix V such that A=UDV, where D is the m x n matrix with (i, ¢)-entries the singular values of A,i=1,2,...,7, and other entries 0. If m =n, then D is diagonal.CHAPTER 1 Elementary Linear Algebra Review InrRODUCTION We briefly review, mostly without proof, the basic concepts and results taught in an elementary linear algebra course. The subjects are vector spaces, basis and dimension, linear transfor- mations, the similarity of matrices, and inner product spaces. 1.1 Vector Spaces Let V be a set of objects and let F be a field, mostly the real number field R or the complex number field C throughout this book. The set V is called a vector space over F if the operations addition uty, uve, and scalar multiplication cv, cEF, vev, are defined so that the addition is associative, is commutative, has an additive identity 0 and additive inverse —v in V for each v EV, and so that the scalar multiplication is distributive, is associative, and has an identity 1 for which lv = v for every v € V. The elements of a vector space are called vectors. 12 ELEMENTARY LINEAR ALGEBRA REVIEW Cuap. 1 For example, R”, the set of all real column vectors Ty x2 » Written as (21,22,...,2n)", Ln is a vector space over R with respect to the addition (21, 2, +665 2n)7 + (Yr, Y29-6- Yn)” = (a1 + y1, 22 + y2,- ++) 2n+Yn)” and the scalar multiplication ¢ (£1, £2,...,2n)” = (x1, CL2,...,C2n)". Note that the real row vectors also form a vector space over R. For convenience, we reserve F” for the column vectors in future dis- cussions involving the matrix and vector product Az. Let S$ = {v1,v2,..., U4} be a subset of a vector space V over a field F. Denote by Span S$ the collection of all linear combinations of the vectors in S, that is, Span S = {cyv) + cove +--+ + cyv_ : each c; € F and v; € S}. The set Span $ is also a vector space over F. If Span$ = V, then every vector in V can be expressed as a linear combination of vectors in S. In such cases we say that the set S spans the vector space V. A set S = {v1,02,..., v4} is said to be linearly independent if the vector equation C1U1 + Cgvo +++ + env, = 0 has only the trivial solution cy = co = --- = cy = 0. If there are also nontrivial solutions, then S is said to be linearly dependent. For instance, both {(1,0)7, (0,1)7, (1,1)7} and {(1,0)7, (0,1)7} span R?, The first set is linearly dependent and the second set is linearly independent. A basis of a vector space V is a linearly independent set that spans V. If V possesses a basis of an n-vector set S = {v1,v2,...,Un}, we say that V is of dimension n, written as dim V = n. Conventionally,Sec. 1.1 VECTOR SPACES 3 the dimension of a zero vector space is 0. If any finite set cannot span V, then V is infinite-dimensional. For instance, C is a vector space of dimension 2 over R with basis {1,2}, where i = /—1, and of dimension 1 over C with basis {1}. C", the set of column vectors of n complex components, is a vector space over C having standard basis e1 = (1,0,...,0,0)", €2 = (0,1,...,0,0)", en = (0,0,...,0,1)". If {u1, u2,.-., Un} is a basis for a vector space V of dimension n, then every z in V can be uniquely expressed as a linear combination of the basis vectors: © = TU + 2Quat++++LpUn, where the 2; are scalars. The n-tuple (21,22,...,2n) is called the coordinate of vector x with respect to the basis. Let V bea vector space of dimension n, and let {v1, v2,..-, vz} be a linearly independent subset of V. Then k < n, and it is not difficult to see that if k < n, then there exists a vector vz+1 € V such that the set {v1, v2,..., Ug, Ue41} is linearly independent (Problem 13). It follows that the set {v1,v2,..., vg} can be extended to a basis of V. Let W be a subset of a vector space V. If W is also a vector space under the addition and the scalar multiplication for V, then W is called a subspace of V. For subspaces Vi and V2, the sum of Vi and V2 is defined to be Vi + Vo = {ur tv: 11 € Vi, v2 € Vo}. It follows that the sum Vj + V2 is also a subspace. In addition, the intersection V; V2 is a subspace, and VU CViCU+Ve, 4=1,2.4 ELEMENTARY LINEAR ALGEBRA REVIEW Cuap. 1 Theorem 1.1 (Dimension Identity) Let V be a finite-dimensiona vector space, and let V; and V2 be subspaces of V. Then dim V, + dim V2 = dim(V, + Va) + dim(V, N V2). The proof is done by first choosing a basis {uj,..., ug} for ViNVa, extending it to a basis {ui,..., uk, Uk+1,--+»Us} for Vi and a basis {u1,..., Uk; We41,+--, we} for Vo, and then showing that {t,o 6+ Wher Uebty +69 Vsy Witty «+s We} is a basis for Vi + Vo. It follows that subspaces Vi and V2 contain nonzero comimon vectors if the sum of their dimensions exceeds dim V. Problems 1. Show explicitly that R? is a vector space over R but not over C. 2. Can a vector space have two different additive identities? Why? 3. Show that F,,[z], the collection of polynomials over a field F with degree at most n, is a vector space over F with respect to the ordinary addition and scalar multiplication of polynomials. Is F[z], the set of polynomials with any degree, a vector space over F? What is the dimension of F,, [2] or F[2]? 4, Determine whether or not the vectors vy) = 1 +2 — 227, uw = 2+ 5a — 2, and v3 = x + 2” in Fs[z] are linearly independent. 5. Show that {(1,4)7, (,-1)7} is a linearly independent subset of C? over the real R but not over the complex C. 6. Determine whether or not R?, with the operations (21, 91)" + (22, Yo)" = (trea, yiye)™ and eer, 71)" = (en, cy)", is a vector space over R. 7. Let V be the set of all real numbers in the form a+bv2+ev5, where a, b, and c are rational numbers. Show that V is a vector space over the rational number field Q. Find dim V and a basis of V.Sec. 1.1 VECTOR SPACES 5 8. 10. 11. 12, 13. 14. 15. 16. 17. Let V be a vector space. If u,v, w € V are such that au-+bu-+cw =0 for some scalars a,b,c, ac # 0, show that Span{u, v} = Span{v, w}. . Let V be a vector space over F and let W be a subset of V. Show that W is a subspace of V if and only if for all u,v € W and ce F ut+veW and cueW. Is the set {(x, y)” € R? : 22—3y = 0} a subspace of R?? How about {(z, y)” € R? : 22 — 3y = 1}? Give a geometric explanation. Show that the set {(z, y—2, y)7: 2, y € R} is a subspace of R?. Find the dimension and a basis of the subspace. Show that if W is a subspace of vector space V of dimension n, then dim W < n. Is it possible that dim W = n for a proper subspace W? Let {uj,...,us} and {v,,..., v2} be two sets of vectors. If s > ¢ and each u; can be expressed as a linear combination of v4,...,v,, show that u,...,us are linearly dependent. Let V be a vector space over a field F. Show that ifu eV andceF such that cv = 0, then c= 0 or v= 0. The sum V, + V2 of two subspaces Vi and V2 of a finite-dimensional vector space V is called a direct sum, symbolized by Vi @ Va, if uty.=0, nEV,wEVe > vy =w2=0. Show that Vi + V2 is a direct sum if and only if dim(V, + V2) = dim V, + dim V2. Conclude that if {u1,...,us} is a basis for Vj and {v1,...,u:} is a basis for V2, then {w),..., Us, V1,--.,U} is a basis for V; @ V2. Let Vi and V2 be subspaces of a vector space of finite dimension such that dim(Vj + V2) = dim(V; NV2) +1. Show that V, C V2 or Va CV. Let 51,52, and S3 be subspaces of a vector space of dimension n. Show that dim(S1N S29 S3) > dim S, + dim S2 + dim $3 — 2n.6 ELEMENTARY LINEAR ALGEBRA REVIEW Cuap. 1 1.2 Matrices An m x n matrix A over a field F (C or R) is a rectangular array of m-rows and n-columns of entries in F: a, 412 Qin a2 O92... Gan A= . m1 m2 ++. Omn Such a matrix, written as A = (aj;), is said to be of size m x n. The set of all m x n matrices over a field F is a vector space with respect to matrix addition by adding corresponding entries and to scalar multiplication by multiplying each entry of the matrix by the scalar. The dimension is mn, and the matrices with one entry equal to 1 and 0 entries elsewhere form a basis. In case of square matrices, that is, m =n, the dimension is n?. We denote by Mn the set of all complex n-square matrices throughout the book. The product AB of two matrices A = (aj;) and B = (bij) is defined to be the matrix whose (3, j)-entry is given by aiib1; + ainba; + +++ + ainbn;. Thus, in order that AB make sense, the number of columns of A must equal the number of rows of B. Take, for example, 1 -l 3.4 5 4=(9 7): 8=(6 08): _(-3 4 -3 AB= ( 122 0 16 ). Note that BA is undefined. Sometimes it is useful to write the matrix product AB, with B = (bi, bo,...,bn), where the b; are the column vectors of B, as Then AB = (Abj, Abp,..., Abn).SEc. 1.2 Matrices 7 The transpose of an m x n matrix A = (aj;) is an n x m matrix, denoted by A’, whose (i,j)-entry is a; and the conjugate of A is a matrix of the same size as A, symbolized d by A, whose (i, j)-entry is aj. We denote the conjugate transpose X of A by A*. The n x n identity matrix I, or simply I, is the n-square matrix with all diagonal entries 1 and off-diagonal entries 0. A scalar matrix is a multiple of J, and a zero matrix 0 is a matrix with all entries 0. A square complex matrix A = (a;;) is said to be diagonal if ay =0, iF j, upper-triangular if aj = 0, i > j, symmetric if AT = A, Hermitian if A* = A, normal if A*A = AA*, unitary if A*A = AA* =I, and orthogonal if A"A = AAT =I. A submatriz of a given matrix is an array lying in specified subsets of the rows and columns of the given matrix. For example, 1 2 o=(3 f) o 1 2 A=[i 3 } n V3 -1 lying in rows one and two and columns two and three. If we write B = (0,4)", D = (m), and E = (V3, —1), then BC A=(5 §): The right-hand side is called a partitioned or block form of A. The manipulation of partitioned matrices is a basic technique in matrix theory. One can perform addition and multiplication of (appropriately) partitioned matrices as with ordinary matrices. For instance, if A, B, C, X, Y, U, V are n-square matrices, then AB X Y\_f AX+BU AY+BV 0c uv) cU cv . is a submatrix of8 ELEMENTARY LINEAR ALGEBRA REVIEW Cuap. 1 2 x 2 block matrices have appeared to be the most useful parti- tioned matrices. We shall emphasize the techniques for block matri- ces of this kind most of the time in this book. Elementary row operations for matrices are those that i. interchange two rows, ii. multiply a row by a nonzero constant, or iii. add a multiple of a row to another row. Elementary column operations are similarly defined, and similar operations on partitioned matrices will be discussed in Section 2.1. An n-square matrix is called an elementary matriz if it can be obtained from J, by a single elementary row operation. Elementary operations can be represented by elementary matrices. Let EF be the elementary matrix by performing an elementary row (or column) operation on Im (or J, for column). If the same elementary row (or column) operation is performed on an m x n matrix A, then the resulting matrix from A via the elementary row (or column) operation is given by the product A (or AE, respectively). For instance, A = ( 123 ) is brought by elementary row 45 6 wo 100 and column operations into ( 010 123 100 Rokk ( j 5 8) Oe= (4 1 0): ) . Write in equations where and This generalizes as follows.Sgc. 1.2 MATRICES 9 Theorem 1.2 Let A be an mx n complet matrix. Then there exist PEM, and Q € Mp, products of elementary matrices, such that pag (4 0): (1.1) The partitioned matrix in (1.1), written as I, @ 0 and called a direct sum of I, and 0, is uniquely determined by A. The size r of the identity I, is the rank of A, denoted by rank (A). If A = 0, then rank (A) = 0. Clearly rank (A”) = rank (A) = rank (A*) = rank (A). An application of this theorem shows that the dimension of the solution space to the linear equation system Ax = 0, where A has n columns, is n —rank (A) (Problem 14). A notable fact about a linear equation system is that Az =0 ifand only if (A*A)z=0. The determinant of a square matrix A, denoted by det A, or |A| if A is in a partitioned form, is a number associated to A. For a(S) the determinant is defined to be a b det A = d | =ad-—be. In general, the determinant of A € Mn may be defined inductively, a det A = S\(-1)'ay; det A(1[), j=l where A(1|j) is a submatrix of A obtained by deleting row 1 and column j of A. This formula is referred to as the Laplace expansion along row 1. One can also expand a determinant along other rows or columns to get the same result.10 ELEMENTARY LINEAR ALGEBRA REVIEW Cuap. 1 The determinant of a matrix has the following properties: a. the determinant changes sign if two rows are interchanged; b. the determinant is unchanged if a constant multiple of one row is added to another row; c. the determinant is a linear function of any row when all the other rows are held fixed. Similar properties are true for columns. Two often-used facts are det(AB) = det A det B, A, BE Mh, and AB o¢ A square matrix A is said to be invertible or nonsingular if there exists a matrix B of the same size such that AB = BA=I. | = aet4 detC, AEMn, CEMp. Such a B, which can be proven to be unique, is called the inverse of A and denoted by A~!. The inverse of A, when it exists, can be obtained from the adjoint of A, written as adj(A), whose (i, j)-entry is the cofactor of a;;, that is, (—1)4+ det A(j|é), where A(j|é) stands for the submatrix of A obtained by deleting row j and column i. To be exact, in symbols, le A” = Taa 4 adj(A). (1.2) An effective way to find the inverse of a matrix A is to apply elementary row operations to the matrix (A,J) to get a matrix in the form (J, B). Then B = Aq! (Problem 17). If A is a square matrix, then AB = J if and only if BA=I. It is easy to see that rank(A) = rank(PAQ) for invertible ma- trices P and Q of appropriate sizes (meaning that the involved op- erations for matrices can be performed), It can also be shown that the rank of A is the largest number of linearly independent columns (rows) of A. In addition, the rank of A is r if and only if there ex- ists an r-square submatrix of A with nonzero determinant, but all (r+1)-square submatrices of A have determinant zero (Problem 18).Theorem 1.3 The following statements are equivalent for A € Mn 1. A is invertible; 2 AB=I or BA=I for some B € Mn; 3. A is of rank n; 4. A is a product of elementary matrices; 5. Az =0 has only the trivial solution x = 0; 6. Az =b has a unique solution for each b € C"; 7. det A #0; 8. the column vectors of A are linearly independent; 9. the row vectors of A are linearly independent. Problems 123 1. Find the rank of ( ; ; . } by performing elementary operations 2. Evaluate the determinants 2-3 10 l+z 1 1 1 2 -2], 1 ol+y 1 0 1 -3 1 1 ltz 3. Show the 3 x 3 Vandermonde determinant identity 11. @, @2 a3 at a a3 = (az — a1)(a3 — a1)(a3 — a2) and evaluate the determinant la @-be 1 6 B-ca 1c c-ab KNIHOVNA MATFYZ. FAKULTY12 ELEMENTARY LINEAR ALGEBRA REVIEW Cuap. 1 4. Does there exist a real matrix A of odd order such that A? =—J? 5. Let A € Mn. Show that A* +A is Hermitian and A* — A is normal. 6. Let A and B be complex matrices of appropriate sizes. Show that, (a) AB=AB, (b) (AB)" = BTA’, (c) (AB)* = BtAt, (d) (AB)~! = B~'A~ if A and B are invertible. 7. Ifa,,c, and d are complex numbers such that ad—bce # 0, show that, ab\"* 1 d -b cd ~ ad—be\ -c a )* 8. Find the inverse of each of the following matrices: lao 110 110 0104), oll), 111 4f. 061 ool o1i1 9. Compute for every positive integer k 0 1)\* 1 1\* 11\* 10)? lo)’ o1}° 10. Show that for any square matrices A and B of the same size ATA-B'B = 5U(A +BY'(A—B) +(A—B)*(A+B)). 11. Show that any two of the following three properties imply the third: (a) A= A’; (b) A* = A7}; (c) A =I. 12, Let A be a square complex matrix. Show that I-A™! = (1- A)(I+ A+ A? 4---+A™). 13. Let A, B, C, and D be n-square complex matrices. Compute (A 4) = (6 3)(% 2).Sec. 1.2 MATRICES 13 14. 15. 16. 17. 18. 19. 20. 21. Show that the solution set to the linear system Az = 0 is a vector space of dimension n — rank (A) for any m x n matrix A over R or C. Let A, B € My. If AB =0, show that rank (A) + rank (B) rank (A) +rank (B). ®14 ELEMENTARY LINEAR ALGEBRA REVIEW Cuap. 1 1.3 Linear Transformations and Eigenvalues Let V and W be vector spaces over a field F. A map A: V +> W is called a linear transformation from V to W if for all uy EV, ce F A(u+v) = A(u) + A(v) and A(ev) = cA(v). It is easy to check that A: R? +4 R?, defined by A(zi, 22)" = (21 + 22,21 — 22)", is a linear transformation and that the differential operator D, from C’a, b], the set (space) of functions with continuous derivatives on the interval [a,b], to C{a, b], the set of continuous functions on [a,}], defined by af we Dz(f) = » £€Cla,d), is a linear transformation. Let A be a linear transformation from V to W. The subset in W Im(A) = {A(v) : ve V} is a subspace of W, called the image of A, and the subset in V Ker(A) = {u EV: A(v) =0€ W} is a subspace of V, called the kernel or null space of A. Theorem 1.4 Let A be a linear transformation from a vector space V of dimension n to a vector space W. Then dim Im(A) + dim Ker(A) = n.Sec. 1.3 LINEAR TRANSFORMATIONS AND EIGENVALUES 15 This is seen by taking a basis {u,...,us} for Ker(A) and ex- tending it to a basis {u1,...,Us,v1,---,v} for V, where s+t =n. It is easy to show that {A(v1),...,A(v1)} is a basis of Im(A). Given an m xn matrix A with entries in F, one can always define a linear transformation A from F" to F” by A(z) = Az, xe F. (1.3) Conversely, linear transformations can be represented by matri- ces. Consider, for example, A: R? + R® defined by A(x1, 22)" = (321, 241 + 22,21 — 229)". Then A is a linear transformation. We may write in the form A(z) = Az, 3.20 2=(z1,22)7, A= 2 li. -l -2 Let A bea linear transformation from V to W. Once the bases for V and W have been chosen, A has a unique matrix representation A as in (1.3) determined by the images of the basis vectors of V under A, and there is a one-to-one correspondence between the linear transformations and their matrices. A linear transformation may have different matrices under different bases. In what follows we show that these matrices are similar when V = W. Two square matrices A and B of the same size are said to be similar if P-!AP = B for some invertible matrix P. Let A be a linear transformation on a vector space V with a basis {ui,...,Un}. Since each A(u;) is a vector in V, we may write where A(ui) = SY ajuy, a=1,...,n, (1.4) j=l and call A = (a,j) the matric of A under the basis {u1,...,Un}- Write (1.4) conventionally as Alun, .-.5 tn) = (Alta), .+-»A(tn)) = (uty stn) A.16 ELEMENTARY LingaR ALGEBRA REVIEW Cuap. 1 Let v EV. If vy =2yu) +-++ + 2pUq, then Av) = Sa: A(uz) = (Alta), ---sAltn)) 2 = (try; tn)Aay, i=] where z is the column vector (21,...,2n)7. In case of R® or C” with the standard basis uy = €1,..-;Un = en, we have A(v) = Az. Let {v1,...,0n} also be a basis of V. Expressing each uj as a linear combination of v1,...,vn gives an n X n matrix B such that (uy, .-+5 Un) = (vis --+s On) B. It can be shown (Problem 9) that B is invertible since {u1,..., un} is a linearly independent set. It follows by using (1.4) that A(u1,..-;%n) A((u1;---;tn)BO?) (u1,..-)%n)ABM! (v1,-+.,n)(BAB™). This says the matrices of a linear transformation under different bases {ui,...,Un} and {v1,..., un} are similar. Given a linear transformation on a vector space, it is a central theme of linear algebra to find a basis of the vector space so that the matrix of a linear transformation is as simple as possible, in the sense that the matrix contains more zeros or has a particular structure. In the words of matrices, the given matrix is reduced to a canonical form via similarity. This will be discussed in Chapter 3. Let A be a linear transformation on a vector space V over C. A nonzero vector v € V is called an eigenvector of A belonging to an eigenvalue d € C if A(v) = dv, v #0. If, for example, A is defined on R? by A(z, 9)? = (us2)", then A has two eigenvalues, 1 and —1. What are the eigenvectors?Sec. 1.3 LINEAR TRANSFORMATIONS AND EIGENVALUES 17 If \1 and Ag are different eigenvalues of A with respective eigen- vectors x; and x2, then z; and 2 are linearly independent, for if yx) + loa =0 (1.5) for some scalars ; and lz, then applying A to both sides yields L121 + lergre = 0. (1.6) Multiplying both sides of (1.5) by A1, we have A121 + larAyz2 = 0. (1.7) Subtracting (1.6) from (1.7) results in lo(Ar — A2)e2 = 0. It follows that 12 =0, and thus /; = 0 from (1.5). This can be generalized by induction to the following statement: If a;, are linearly independent eigenvectors corresponding to an eigenvalue ,;, then the set of all eigenvectors Qi; for these eigenvalues A; together is linearly independent. Simply put: Theorem 1.5 The eigenvectors belonging to different eigenvalues are linearly independent. If A happens to have n linearly independent eigenvectors belong- ing to (not necessarily distinct) eigenvalues A1,A2,---,An, then A, under the basis formed by the corresponding eigenvectors, has a di- agonal matrix representation AL 0 A2 0 An To find eigenvalues and eigenvectors, one needs to convert A(v) = dv18 ELEMENTARY LINEAR ALGEBRA REVIEW Cuap. 1 under a basis into a linear equation system Az = dz. Therefore, the eigenvalues of A are those € F such that det(AI — A) = 0, and the eigenvectors of A are the vectors whose coordinates under the basis are the solutions to the equation system Az = Ax. Suppose A is an n x n complex matrix. The polynomial in det(AIn — A) (1.8) is called the characteristic polynomial of A, and the zeros of the polynomial are called the eigenvalues of A. It follows that every n-square matrix has n eigenvalues over C (including repeated ones). The trace of an n-square matrix A, denoted by tr A, is defined to be the sum of the eigenvalues \1,..., An of A, that is, trA=dAi+---+An- It is easy to see from (1.8) by expanding the determinant that trA=ante + enn and n det A = [] A. i=1 Problems 1. Show that the map A from R° to itself defined by Alay, 2)? = (e+ y,2—y,2)" is a linear transformation. Find its matrix under the standard basis. 2. Find the dimensions of Im(A) and Ker(.A), and find their bases for the linear transformation A on R? defined by A(z,y,2)" = (@ — 2z,y + 2,0)".Sec. 1.3 LINEAR TRANSFORMATIONS AND EIGENVALUES 19 3. Define a linear transformation A: R? ++ R? by A(z,y) = (y,0)". (a) Find Im(A) and Ker(A). (b) Find a matrix representation of A. (c) Verify that dim R? = dim Im(A) + dim Ker(A). (d) Is Im(A) + Ker(A) a direct sum? (e) Does R? = Im(A) + Ker(.A)? 4, Find the eigenvalues and eigenvectors of the differential operator Dz. 5. Find the eigenvalues and corresponding eigenvectors of the matrix _f{14 4=(3 5): 6. Let A be an eigenvalue of A on a vector space V, and let. Vv ={vEeV: A(v) = rr}, called the eigenspace of X. Show that Vy is an invariant subspace of V under A, that is, it is a subspace and A(v) € Vy for every v € Vj. 7. Define linear transformations A and B on R? by Ale)” =(2+4, 9)", Bley)? =(2+y, 2-y)’. Find all eigenvalues of A and B and their eigenspaces. 8. Let p(x) = det(zI — A) be the characteristic polynomial of matrix A Mh. If J is an eigenvalue of A such that p(x) = (2 —A)*q(z) for some polynomial q(x) with (A) # 0, show that k>dimVy. 9. Let {u1,-..,Un} and {t1,...;Un} be two bases of a vector space V. Show that there exists an invertible matrix B such that (u1,.- 65 Un) = (Vi,--.) Un) B. 10. Let {u1,..-,Un} be a basis for a vector space V and let {v1,..., 0x} be a set of vectors in V. If u, = je aijuj, i=1,...,k, show that dim Span{vj,...,v%} = rank (A).20 1. 12. 13. 14. 15. 16. 17. 18. 19. ELEMENTARY LINEAR ALGEBRA REVIEW Cuap. 1 Show that similar matrices have the same trace and determinant. Let 1 and v2 be eigenvectors of matrix A belonging to different eigenvalues A; and 2, respectively. Show that v, + ve is not an eigenvector of A. How about av, + bv2, a, be R? Let A € M, and let S € M, be nonsingular. If the first column of S71AS is (4,0,...,0)7, show that A is an eigenvalue of A and that the first column of S is an eigenvector of A belonging to A. Let « € C”. Find the eigenvalues and eigenvectors of the matrices A; =22" and a= (8 =): If each row sum (i.e., the sum of entries in a row) of matrix A is 1, show that 1 is an eigenvalue of A. If A is an eigenvalue of A € M,, show that \? is an eigenvalue of A? and that if A is invertible, then A~! is an eigenvalue of A7}. A minor of a matrix A € Mi, is the determinant of a square submatrix of A, Show that det(AZ — A) = A" — 6: N"7} +. 6 A™-? — --- + (-1) "det A, where 6; is the sum of all principal minors of order 7, i = 1,2,...,n—1. (A principal minor is the determinant of a submatrix indexed by the same rows and columns, called a principal submatriz.) A linear transformation A on a vector space V is said to be invertible if there exists a linear transformation B such that AB = BA = T, the identity. If dim V < 00, show that the following are equivalent: (a) A is invertible; (b) if A(z) =0, then z = 0, that is, Ker(A) = {0}; (c) if {u1,..., Un} is a basis, then so is {Aui,..., Aun}; (d) A is one-to-one; (e) A is onto, that is, Im(.A) = V; (£) A has a nonsingular matrix representation under some basis. Let A be a linear transformation on a vector space of dimension n with matrix representation A. Show that dimIm(A) = rank(A) and dim Ker(A) = n—rank (A).SEc. 1.3 LINEAR TRANSFORMATIONS AND EIGENVALUES 21 20. 21. 22, 23. 24, Let A and B be linear transformations on a finite-dimensional vector space V having the same image, that is, Im(.A) = Im(B). If V =Im(A) © Ker(A) = Im(8) @ Ker(B), does it follow that Ker(A) = Ker(B)? Consider the vector space F{z] of all polynomials over F(= R or Q). For f(£) = Gat” + Qn—12""! +--+ +012 +40 € Fiz], define an On-1 a Ss - grt a2 (F(@)) = Sah + a bt oa? + aor and T(f(x)) = nanz™! + (n— Van-12"-? +++ tar. Compute ST and TS. Does ST = TS? Define P : C? ++ C™ by P(z) = (0,0,23,---,2,)". Show that P isa linear transformation and P? = P. What is Ker(P)? Let A be a linear transformation on a finite-dimensional vector space V, and let W be a subspace of V. Denote A(W) = {A(w): we WH. Show that A(W) is a subspace of V. Furthermore, show that dim(A(W)) + dim(Ker(A) NW) = dim W. Let V be a vector space of dimension n over C and let {ui,..-,Un} be a basis of V. For = zu, +--+ + Zptn € V, define T(x) = (1,--.,2n)” €C*, Show that T is an isomorphism, or T is one-to-one, onto, and satisfies T(ax + by) =aT(z)+6T(y), 2, yEV, a, bEC. . Let V be the vector space of all sequences (c1,¢2,-..), GEC, #=1,2,.... Define a linear transformation on V by S(C1,€25°++) = (0, c15 €2)°**)- Show that S has no eigenvalues. Moreover, if we define S*(c1,€2,€3,°++) = (c2,€3,°**), then S*S is the identity, but SS* is not. °22 ELEMENTARY LINEAR ALGEBRA REVIEW Cuap. 1 1.4 Inner Product Spaces A vector space V over field C or R is called an inner product space if it is equipped with an inner product (, ) satisfying for all u,v,w EV and scalar ¢ 1. (u,u) > 0, and (u, u) = 0 if and only if u=0, 2. (ut v,w) = (u,w) + (0,0), 3. (cu, v) = c(u,v), and 4. (u,v) = (v,u). C” is an inner product space over C with the inner product (t,y) = yr =Gi ti +--+ Tn Tn. The Cauchy-Schwarz inequality for an inner product space is one of the most useful inequalities in mathematics. Theorem 1.6 (Cauchy-Schwarz Inequality) Let V be an inner product space. Then for all x and y inV, ley)? < @2)(y)- Equality holds if and only if x and y are linearly dependent. The proof of this can be done in a number of different ways. The most common proof is to consider the quadratic function in ¢ (x + ty, 2 + ty) and to derive the inequality from the nonnegative discriminant. One may also obtain the inequality from (z,z) > 0 by setting _ 2) (x,2) and showing that (z,z) = 0 and then (z, z) = (z,y) > 0. z=y z, «#0,Sec, 1.4 INNER PRobucT SPACES 23 A matrix proof is left as an exercise for the reader (Problem 12). For any vector x in an inner product space, the positive square root of (z,x) is called the length or norm of the vector xz and is denoted by ||x||, that is, llaell = VG, a) Thus, the Cauchy-Schwarz inequality is rewritten as Ie,y)] < Ill My Theorem 1.7 For all vectors x and y in an inner product space, i |lel] 20; ii, llex|] =lelflel,c€ C; iti. |x + yl] < [lel] + lly. The last inequality is referred to as the triangle inequality. A unit vector is a vector whose length is 1. For any nonzero vector u, = u is a unit vector. Two vectors z and y are said to be orthogonal if (x,y) = 0. An orthogonal set is a set in which any two of the vectors are orthogonal. Such a set is further said to be orthonormal if every vector in the set is of length 1. For example, {v;, v2} is an orthonormal set in R?, where n= (J 4) = (4) 1=\ yp ya) Vi va) The column (row) vectors of a unitary matrix are orthonormal. Let S be a subset of an inner product space V. Denote by S+ the collection of the vectors in V that are orthogonal to all vectors in S, that is, += {vEV: (v,s)=0 for all s € S}. It is easy to see that S“ is a subspace of V. If S$ contains only one element, say z, we simply use z+ for $+. For two subsets $, and So, if (x,y) = 0 for all x € S| and y € So, we write 5,159. As we saw in the first section, a set of linearly independent vectors of a vector space of finite dimension can be extended to a basis for the vector space. Likewise a set of orthogonal vectors of an inner24 ELEMENTARY LINEAR ALGEBRA REVIEW Cuap. 1 product space can be extended to an orthogonal basis of the space. The same is true for a set of orthonormal vectors. Consider C”, for instance. Let uw; be a unit vector in C”. Pick a unit vector ue in up if n > 2. Then wu and uz are orthonormal. Now if n > 3, let ug be a unit vector in (Span{ui,u2})* (equivalently, (u1,us) = 0 and (ug,u3) = 0). Then 11, ue, ug are orthonormal. Continuing this way, one obtains an orthonormal basis for the inner product space. We summarize this as a theorem for C”, which will be freely and frequently used in the future. Theorem 1.8 Jf uj,..., ug are k linearly independent vectors in C”, 1 QizTix;. i, j=l Upon computation, we have (Az, y) = (2, A*y) and, with ImA = {Ax: x €C"} and KerA={xeEC": Az =0}, Ker A* = (ImA)+, ImA* = (Ker A)?. (1.11) My, is an inner product space with the inner product (A, B)y = tr(B*A), A, BE Mh. It is immediate by the Cauchy-Schwarz inequality that | tr(AB)|? < tr(A*A) tr(B*B) and that tr(A*A) =0 ifandonlyif A=0. We end this section by presenting an inequality of the angles between vectors. This inequality is intuitive and obvious in R? and R3. The good part of this theorem is the idea in its proof of reducing the problem to R? or R°. Let V be an inner product space over R. For any nonzero vectors z and y, define the angle between z and y by Le y= cos7) (x,y) . me lelillyll Theorem 1.9 For any nonzero vectors x, y, and z in an inner prod- uct space V over R, Le, S Lay + Ly, 2 Equality occurs if and only if y = ax + bz, a,b >0.26 ELEMENTARY LINEAR ALGEBRA Review Cuap. 1 Proor. Since the inequality involves only the vectors z, y, and z, we may focus on the subspace Span{z, y,z}, which has dimension at most 3. We can further choose an orthonormal basis (a unit vector in case of dimension one) for this subspace. Let x, y, and z have coordinates a, 8, and y under the basis, respectively. Then the inequality holds if and only if it holds for real vectors a, 8, and ¥, due to (1.9) and (1.10). Thus, the problem is reduced to R, R?, or R? depending on whether the dimension of Span{z, y, z} is 1, 2, or 3, respectively. For R, the assertion is trivial. For R? or R°, a simple graph will do the job. @ Problems 1. If V is an inner product space over C, show that for z, ye V, ce C (z,cy) =@z,y) and (2,y)(y,2) =|(e,y)?. 2. Find all vectors in R? (with the usual inner product) that are orthog- onal to (1,1)". Is (1,1)? a unit vector? 3. Show that in an inner product space over R or C (zy) =0 = |e +yll? = [2ll? + lly? and that the converse is true over R but not over C. 4. Show that for any two vectors x and y in an inner product space lle — ll? + [lz + yl? = 2¢lleI?? + llyll?). 5. Is [, ] defined as [x,y] = 2141 +-+++2n¥n an inner product for C"? G6. For what diagonal D € Mp is [z, y] = y* Dz an inner product for C”? 7. Let {ui,...,Un} be an orthonormal basis of an inner product space V. Show that for s €V (uj,z)=0, 4=1,....n, & 2=0, and that for z, ye V (ui,2) = (vay), T=1,....7, saySec. 1.4 INNER PRODUCT SPACES 27 10. 11. 12. 13. 14. 15. 16. 17, . Let {v1,.--,;%} be a set of vectors in an inner product space V. Denote the matrix with entries (v;,v;) by G. Show that det G = 0 if and only if v1,...,Un are linearly dependent. . Let A be an n-square complex inatrix. Show that, tr(AX) =0 forevery XEM, @& A=0O. Let A € M,,. Show that for any unit vector z € C”, |c*Ag|? < 2*A* Az. Let A = (aij) be a complex matrix with rows a;. Show that tr(A*A) = tr(AA*) = > lais[?. ag Use the fact that tr(A"A) > 0 with equality if and only if A = 0 to show the Cauchy-Schwarz inequality for C” by taking A = ry*—yz*. Let A and B be complex matrices of the same size. Show that |tr(AB)] < (tr(A*A)tr(B*B))? < 5(tr(4*A) +tr(B*B)). Let A and B be n-square complex matrices. For every x € C", if (Az, x) = (Bz, 2), does it follow that A = B? What if c € R®"? Show that for any n-square complex matrix A C* =ImA@ (Im A)* = ImA4 © Ker A” and that C” = Im A @ Ker A if and only if rank (A?) = rank (A). Let 6; and 4; be positive numbers and J>j_, 6: = 1. Show that n n 1< (daa) (do7"). #1 i=1 Let {ui,..., Un} be an orthonormal basis of an inner product space V. Show that 11, ..., 2% in V are pairwise orthogonal if and only if n See ws) mm) =0, 43. kel28 19. ELEMENTARY LINEAR ALGEBRA REVIEW Cuap. 1 . If {u1,..., ug} is an orthonormal set in an inner product space V of dimension n, show that k Sole.) i=l Let {u,..-,un} and {v1,...,0n} be two orthonormal bases of an inner product space. Show that there exists a unitary U such that (uy, --+y Un) = (t1,---) on UL 20, Prove or disprove for unit vectors u,v,w in an inner product space 21. [(4, wl S [u,v + 1, ¥)L- Let V; and V2 be subsets of an inner product space V. Show that Uch = Vvicyv. 22. Let V, and Vp be subspaces of an inner product space V. Show that (Vi + Va)* = Vib Vt and (Vi V2)* = Vy + V5h 23. Let Vi and V2 be subspaces of an inner product space V of dimension 24, n. If dim Vi > dim V2, show that there exists a subspace V3 such that V3cVi, ValVe, dimVs > dimVY, —dimV2. Give a geometric explanation in R°. Let A and B be m x n complex matrices. Show that ImAlImB «= A‘B=0. . Let A be an n-square complex matrix. Show that for any x, y € C* 4(Az,y) = (As, s) — (At, t) + i(Au, u) — (Av, v), where s=a2+y,t=2—-y, u=at+iy, v=2—iy. 26. Show that for any nonzero z and y in C” 1 1 1 le — yll > =(llell + —2—- — |. lle — all > S(llell + yl a’ ©CHAPTER 2 Partitioned Matrices IntRopucTION This chapter is devoted to the techniques of parti- tioned (block) matrices. Topics include elementary operations, de- terminants, and inverses of partitioned matrices. We begin with the elementary operations of block matrices, followed by discussions of the inverse and rank of the sum and product of matrices. We then present four different proofs of the theorem that the products AB and BA of matrices A and B of sizes m x n and n x m, respectively, have the same nonzero eigenvalues. At the end of this chapter we discuss the often-used matrix technique of continuity argument. 2.1 Elementary Operations of Partitioned Matrices The manipulation of partitioned matrices is a basic tool in matrix theory. The techniques for manipulating partitioned matrices resem- ble those for ordinary numerical matrices. We begin by considering a2x 2 matrix (2 a): a, bo de. An application of an elementary row operation, say, adding the sec- ond row multiplied by —3 to the first row, can be represented by the30 PARTITIONED MATRICES Cuap. 2 matrix multiplication 1 -3 a b\_ fa-—3e b—3d ool cd} c d . Note that the determinant of the matrix does not change. Elementary row or column operations for matrices play an im- portant role in elementary linear algebra. These operations (page 8) can be generalized to partitioned matrices as follows: I. interchange two (block) rows (columns), II. multiply a (block) row (column) from the left (right) by a non- singular matrix of appropriate size, and III. multiply a (block) row (column) by a matrix from the left (right), then add it to another row (column). Write in matrices, say, for type III elementary row operations, AB 5 A B Cc D C+XA D+XB }’ where A € Mm, D € Mn, and X isn x m. Note that A is multiplied by X from the left (when row operations are performed). Generalized elementary matrices are those obtained by applying a single elementary operation to the identity matrix. For instance, 0 In In 0 (1.0) = (2 i) are generalized elementary matrices of type I and type III. Theorem 2.1 Let G be the generalized elementary matrix obtained by performing an elementary row (column) operation on I. If that same elementary row (column) operation is performed on a block matrix A, then the resulting matriz is given by the product GA (AG). Proor. We show the case of 2 x 2 partitioned matrices since we shall deal with this type of partitioned matrix most of the time. An argument for the general case is similar.Sec, 2.1 ELEMENTARY OPERATIONS OF PARTITIONED MATRICES 31 Let A, B, C, and D be matrices, where A and D are m- and n-square, respectively. Suppose we apply a type III operation, say, adding the first row times an n X m matrix E from the left to the second row, to the matrix AB (¢ 5). Then we have, by writing in equation, A B _ {Im 0 AB\y C+EA D+EB} \ E In cD)’ As an application, if A is invertible, we can make the lower-left and upper-right submatrices 0 by subtracting the first row multi- plied by CA7! from the the second row, and by subtracting the first column multiplied by A~!B from the second column. In symbols, A BY\_W(A B 5 A 0 cD 0 D-CA"'B 0 D-CA"'B }° and in equation, Im 0 AB In -A 1B -CA? I, cD 0 In A 0 = ( 0 D-CA"B ). (2.1) Note that by taking determinants, A Bl _ _oart | cD |= aer4 det(D — CA~'B). The method of manipulating block matrices by elementary oper- ations and the corresponding generalized elementary matrices as in (2.1) will be used repeatedly in this book. For practice, we now consider expressing the block matrix (¢ 2) (2.2)32 PARTITIONED MATRICES Cuap. 2 as a product of block matrices of the forms (07): (1) In other words, we want to get a matrix in the above form by per- forming type III operations on the block matrix in (2.2). Add the first row of (2.2) times A~! to the second row to get A B IA V+A'B)° Add the second row multiplied by I — A to the first row to get I A‘+A1B-I I A‘+4A 'B . Subtract the first row from the second row to get I A ‘+A1B-I 0 I , which is in the desired form. Putting these steps in identity, we have (2) T)C4 1) Co ) =(4 ant ANB ") Can) G7) (45) Gee) (1 1) (0 “7") (IGG) is a product of type III generalized elementary matrices. Therefore, (¢ &)Sec. 2.1 ELEMENTARY OPERATIONS OF PARTITIONED MATRICES 33 Problems 1. Let E = Efi(c) — §] denote the elementary matrix obtained from In, by adding row i times c to row j. (a) Show that E* = E[j(@) — 7). (b) Show that E~! = Efi(—c) — jj. (c) How is E obtained via an elementary column operation? 2. Let X be any n x m complex matrix. Show that Im 0\7'_( Im 0 Xm) ~\-x mh)’ 3. Show that for any n-square complex matrix X X mh\'_(0 In In 0 “\ In -X J Does it follow that 0 Im \*_( 0 Im \o In 0 “\In 0 ]° 4, Show that every 2x2 matrix of determinant 1 is the product of some matrices of the following types, with y # 0: 10 le 01 y 0 10 a lj’ 01)? 10)’ o1/' Oy) 5. Let X and Y be matrices with the same number of rows. Multiply xX Y xX* 0 and xX* 0 x Y 0.60 Yy* 0 Yy* 0 0 0 /}* 6. Let X and Y be complex matrices of the same size. Verify that I+XX* X+Y _ Ix IY X*+Y" I+Y*Y a y* I xe I xX I x* T I Y* IY ( * Jone ( Pa Jax.34 7. 10. 11. 12. PARTITIONED MATRICES Cuap. 2 Let @1,@2,..., @n be complex numbers, a = (—a2,...,—@n), and a-( 0 Ina ). a) a Find det A. Show that A is invertible if a, 4 0 and that ao -i -1_ a a A =( 7 * ): Let @1,G2,..., @ be nonzero complex numbers. Find 0am 0... 0 \ 0 O ag 0 0 0 0 1. ant a 0 0 ... 0 . Show that the following two block matrices commute: In A In B om) Lon): Show that a generalized elementary matrix * can be written as the product of the same type of elementary matrices with only one nonzero off-diagonal entry. (Hint: Consider how to get 2;; in the matrix by a type iii elementary operation on page 8.) -1 Let A and B be nonsingular matrices. Find ( 4 $ ) ‘ Let A and B be m- and n-square matrices, respectively. Show that A+\*_ (A + o B) ~\o By where the * are some matrices, and that if A and B are invertible, (5) -(4) &)-Sec. 2.1 ELEMENTARY OPERATIONS OF PARTITIONED MATRICES 35 13. 14. 15. 16. 17. 18. Let A and B be n x n complex matrices. Show that oA Bo and that if A and B are invertible, then (34y"-(4 8) Let A and B be n x n matrices. Apply elementary operations to ( Ao ) to get ( ABO ). Deduce det(AB) = det Adet B. | = (-1)" det Adet B IB Bod Let A and B be n x n matrices. Apply elementary operations to IA I-AB 0 I 0 (3 i Joe 0 T and (j 1 ’pa )+ Conclude that J — AB and J — BA have the same rank for A, B € My. Let A and B be n x n matrices. Apply elementary operations to ( 4 , ) to get ( A 4? 2 ) and derive the rank inequality rank (A + B) < rank (A) + rank (B). Let A be a square complex matrix partitioned as _{ 4u Ar a=(40 Ay)? Ai € Mm, Aoz € Mn. Show that for any B € Mm BAy BA An Az and for any n x m matrix C Au Aiz An +CAy Ao2+CArz = det B detA = det A. Let A be a square complex matrix. Show that IA « * av 7 |= 17> MIM + 30 MEM — 50 MgMa+-, where each Mx, is a minor of order k = 1,2,.... (Hint: Reduce the left-hand side to det(J — A*A) and use Problem 17 of Section 1.3.) °36 PARTITIONED MATRICES Cuap. 2 2.2 The Determinant and Inverse of Partitioned Matrices Let M be a square complex matrix partitioned as AB m=(6 5): where A and D are m- and n-square matrices, respectively. We dis- cuss the determinants and inverses of matrices in this form. The re- sults are fundamental and used almost everywhere in matrix theory, such as matrix computation and matrix inequalities. The methods of continuity and finding inverses deserve special attention. Theorem 2.2 Let M be a square matriz partitioned as above. Then det M = det A det(D—CA™'B), if A is invertible, and det M = det(AD-—CB), if AC=CA. Proor. When A~? exists, it is easy to verify (see also (2.1)) that In O0\(AB\_(A B -cA? i )\C D)>\0 D-ca"B ): By taking determinants for both sides, we have A B 0 D-CA™B det A det(D ~ CA~'B). det M For the second part, if A and C’ commute, then A, B, C, and D are of the same size. We show the identity by the so-called continuity argument method. First consider the case where A is invertible. Following the above argument and using the fact that det(XY) = det X det YSec. 2.2 THe DETERMINANT AND INVERSE OF PARTITIONED MATRICES 37 for any two square matrices X and Y of the same size, we have det M I det A det(D — CA~1B) det(AD - ACA-'B) det(AD — CAA~1B) det(AD — CB). Now assume that A is singular. Since det(A + I) as a polynomial in ¢€ has a finite number of zeros, we may choose 6 > 0 such that det(A +I) #0 whenever 0 : In 0 At BC40 PARTITIONED MATRICES Cuap. 2 4. Let A, B, and C be n-square complex matrices. Show that In A Bc = det(C — BA). 5. Let A and B be m x nand n x m matrices, respectively. Show that In B|_|Im A A Im|~| B In and conclude that det (Im — AB) = det(I, - BA). Is it true that rank (J, — AB) = rank (J, — BA)? 6. Can any two of the following expressions be identical for general complex square matrices A, B, C, D? det(AD-—CB), det(AD— BC), det(DA-—CB), det(DA- BC), AB c D\| 7. If A is an invertible matrix, show that rank ( as ) = rank (A) + rank (D — CA71B). In particular, rank (4 fe ) =n rank (x -¥). 8. Does it follow from the identity (2.3) that any principal submatrix (A) of a singular matrix (M) is singular? 9. Find the determinant and the inverse of the 2m x 2m block matrix _ (alm blm _ a-(or ar) ad— be #0. 10. If U is a unitary matrix partitioned as U = ( ’ U, ) , where ueéC, show that |u| =| det U,|. What if U is real orthogonal?