plans for some cleanup in optim package

[Olaf Till]

Hi all,

already a while there are new frontends in optim,
nonlin_residmin/nonlin_curvefit/residmin_stat/curvefit_stat for
residual based optimization and statistics, and nonlin_min for
minimization of scalar valued functions. Some backends for them are
also already there.

An elder frontend, minimize, is intentionally similar to nonlin_min,
but unmaintained, conceptionally different, and causing the dependency
of optim on package miscellaneous. For simplification of maintainance,
I would suggest to drop it. Since this is only possible after a period
of deprecation, I'd suggest to copy its dependency function in
miscellaneous (read_options.m) under a different name to
optim/inst/private for now, to get rid of the dependency at once. It
can be deleted again once minimize should be deleted.

minimize.m can call 3 backends: a gradient-less method
("nelder_mead_min") and two gradient methods ("d2_min",
"bfgsmin"). All 3 can also be called directly. (But d2_min had a
usage-preventing typo-bug since 4 years up to 2014-06-11. And nobody
complained.) Some properties of the 2 gradient methods and gradient
optimizers not called by minimize:

optimizer     constraints       supplied
              possible?         hessian function?
-------------------------------------------------
minimize/     no                required
d2_min

minimize/     no                not used
bfgsmin

nonlin_min/   yes, throughout   optional
lm_feasible   optimization

sqp (in       yes, in           optional (but currently
core Octave)  end result        problems if supplied)


And some tests with them, using unconstrained problems in
optim_problems.m, showing the number of needed iterations:


optimizer     Schittkowski-281   Rosenbrock   Rosenbrock, initial
                                              values nearer to minimum
----------------------------------------------------------------------
minimize/     4049               1145         12
d2_min
(hessian
 supplied)

minimize/     171                75           20
bfgsmin
(no hessian
 supplied)

nonlin_min/   136                84           15
lm_feasible
(hessian
 supplied)

nonlin_min/   158                failure      22
lm_feasible
(no hessian
 supplied)

sqp (in       89                 73           23
core Octave)
(no hessian
 supplied)


Note that in d2_min the ratio of user-function-calls to iterations was
a bit higher than in the other optimizers (have no exact values for
all others).

Both bfgsmin and d2_min could have advantages for certain problems. It
could be worth to include them as backends into the new frontend, but
this would need rewriting. I'd guess d2_min would become too "out of
sight" to be much used if not included. bfgsmin will probably be used
anyway, and maybe it should remain there in original form even if a
rewritten copy is included as backend.

My plan is to ask on the help mailing list if someone uses the
frontend minimize at all, and if not, to go on with deprecation as
outlined above. Afterwards, I'd take a look at the "orphaned"
gradient-based backends, possibly adapting them (or a copy of them)
for the new frontend nonlin_min. I don't think I'll currently consider
the gradient-less backend (nelder_mead_min).

But please speak up if necessary.

Olaf

-- 
public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net

signature.asc
Description: Digital signature