Stabilizing the Generalized Schur Algorithm

This paper provides a detailed analysis that shows how to stabilize the generalized Schur algorithm, which is a fast procedure for the Cholesky factorization of positive-definite structured matrices R that satisfy displacement equations of the form $R - FRF^T = GJG^T$, where J is a $2 \times 2$ signature matrix, F is a stable lower-triangular matrix, and G is a generator matrix. In particular, two new schemes for carrying out the required hyperbolic rotations are introduced and special care is taken to ensure that the entries of a Blaschke matrix are computed to high relative accuracy. Also, a condition on the smallest eigenvalue of the matrix, along with several computational enhancements, is introduced in order to avoid possible breakdowns of the algorithm by assuring the positive-definiteness of the successive Schur complements. We use a perturbation analysis to indicate the best accuracy that can be expected from any finite-precision algorithm that uses the generator matrix as the input data. We then show that the modified Schur algorithm proposed in this work essentially achieves this bound when coupled with a scheme to control the generator growth. The analysis further clarifies when pivoting strategies may be helpful and includes illustrative numerical examples. For all practical purposes, the major conclusion of the analysis is that the modified Schur algorithm is backward stable for a large class of structured matrices.