BIB-VERSION:: CS-TR-v2.0
          ID:: STAN//CS-TR-72-261
       ENTRY:: October 16, 1995
ORGANIZATION:: Stanford University, Department of Computer Science
       TITLE:: The differentiation of pseudoinverses and nonlinear least
               squares problems whose variables separate.
        TYPE:: Technical Report
      AUTHOR:: Golub, Gene H.
      AUTHOR:: Pereyra, Victor
        DATE:: February 1972
       PAGES:: 52
    ABSTRACT:: For given data ($t_i\ , y_i), i=1, \ldots ,m$ , we consider
               the least squares fit of nonlinear models of the form

               F($\underset ~\to a\ , \underset ~\to \alpha\ ; t) =
               \sum_{j=1}^{n}\ g_j (\underset ~\to a ) \varphi_j (\underset
               ~\to \alpha\ ; t) , \underset ~\to a\ \epsilon R^s\ ,
               \underset ~\to \alpha\ \epsilon R^k\ $.

               For this purpose we study the minimization of the nonlinear
               functional

               r($\underset ~\to a\ , \underset ~\to \alpha ) =
               \sum_{i=1}^{m} {(y_i - F(\underset ~\to a , \underset ~\to
               \alpha , t_i))}^2$.

               It is shown that by defining the matrix ${ \{\Phi (\underset
               ~\to \alpha\} }_{i,j} = \varphi_j (\underset ~\to \alpha ;
               t_i)$ , and the modified functional $r_2(\underset ~\to
               \alpha ) = \l\ \underset ~\to y\ - \Phi (\underset ~\to
               \alpha )\Phi^+(\underset ~\to \alpha ) \underset ~\to y
               \l_2^2$, it is possible to optimize first with respect to the
               parameters $\underset ~\to \alpha$ , and then to obtain, a
               posteriori, the optimal parameters $\overset ^\to {\underset
               ~\to a}$. The matrix $\Phi^+(\underset ~\to \alpha$) is the
               Moore-Penrose generalized inverse of $\Phi (\underset ~\to
               \alpha$), and we develop formulas for its Frechet derivative
               under the hypothesis that $\Phi (\underset ~\to \alpha$) is
               of constant (though not necessarily full) rank. From these
               formulas we readily obtain the derivatives of the orthogonal
               projectors associated with $\Phi (\underset ~\to \alpha$),
               and also that of the functional $r_2(\underset ~\to \alpha$).
               Detailed algorithms are presented which make extensive use of
               well-known reliable linear least squares techniques, and
               numerical results and comparisons are given. These results
               are generalizations of those of H. D. Scolnik [1971].
       NOTES:: [Adminitrivia V1/Prg/19951016]
         END:: STAN//CS-TR-72-261