Re: Octave-maintainers Digest, Vol 23, Issue 47

[Jonathan Stickel]

Date: Fri, 15 Feb 2008 10:28:14 +0100
From: "Jaroslav Hajek" <address@hidden>
Subject: Re: minpack code for least squares?
To: address@hidden

Jaroslav Hajek wrote:
> On Fri, Feb 15, 2008 at 7:35 AM, Olaf Till <address@hidden>
> wrote:
> > > > Further, MINPACK's LMDER and LMDIF are likely the most widely 
> > > > used
> > > nonlinear least-squares codes in history, so they're sort of 
> > > "proven quality".
> >
> > 'leasqr' (m-code, in 'optim') also provides an l/m-algorithm, 
> > optionally with user-supplied jacobian.

> After quickscan it seems that leasqr relies on SVD factorization of 
> the jacobian, while MINPACK uses pivoted QR (faster). Also, MINPACK 
> features a trust-region subproblem to select the actual step, whereas
>  leasqr seems to use some heuristics to select the l/m parameter in 
> successive steps. Trust-region techniques have the reputation to 
> improve global convergence [see e.g. Nocedal]

> > Seems to work well, we had problems in which it did converge, 
> > though Matlabs 'lsqcurvefit' l/m-method only pretended to and 
> > lingered at the starting values. It should be tested carefully if 
> > 'lmder' and 'lmdif' are indeed better. Even in this case I would 
> > vote to have the 'leasqr' code kept in place.

> no question; even if MINPACK gave better fit in all cases (which is 
> rarely seen), there are other useful options and output statistics in
>  leasqr that are not found in MINPACK.

In general it is a good idea to mimic the Matlab user interface when 
possible.  Therefore, when writing a new non-linear least squares 
function, I suggest that you name it "lsqnonlin" and have it take inputs 
and give outputs similar to what can be found on the Matlab help pages:

http://www.mathworks.com/access/helpdesk/help/toolbox/optim/

The existing leasqr function can remain as another option for octave users.

Regards,
Jonathan