octave-maintainers
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: bfgsmin and fminunc


From: Michael Creel
Subject: Re: bfgsmin and fminunc
Date: Mon, 18 May 2009 10:29:49 +0200

On Mon, May 18, 2009 at 10:08 AM, Jaroslav Hajek <address@hidden> wrote:
> On Fri, May 15, 2009 at 9:44 AM, Michael Creel <address@hidden> wrote:
>>
>> I have had a look at fminunc.m, and it looks nice in general. I would suggest
>> using central diff gradient. In my experience, the increased time per
>> iteration is usually offset by fewer iterations needed for convergence, due
>> to better accuracy. I see a "guarded_eval" - what is that? I would also
>> suggest setting max iters to infinite, or a lot higher than 400. Some
>> problems are just hard to solve.
>>
>> I may try to take advantage of the fast  cholupdate code in bfgsmin,
>> probably this summer when I have time.
>>
>> Maybe it would be nice to have a set of test functions for optimization
>> algorithms, to be able compare performance and verify the capability of
>> finding correct solutions. I used a set when I first wrote bfgsmin, but I
>> haven't used it in a while. I'll see if I can resurrect it and give it a
>> friendly interface.
>>
>> M.
>>
>
> Michael,
>
> thanks for your comments. I have reworked the FD part of fsolve and
> fminunc so that they now use central diffs by default, controllable
> using the FinDiffType option.
> While the superlinear/subquadratic termination of the BFGS seems to
> work fine in fminunc (which is the primary aim), the global
> convergence props for strongly nonquadratic functions (such as
> Rosenbrock's) is something that depends, according to my observations,
> crucially on the strategy of updating the trust region.
> So far the strategy was very simple, based on the original MINPACK
> code. I replaced it with something slightly more elaborate now, but I
> think there's still a lot to improve. It seems most researchers
> consider the trust region strategy either equally or more powerful
> compared to line searches, but OTOH the line search strategies seem
> far better investigated.
> In the present state fminunc differs pretty much from bfgsmin - trust
> region/line search, factorized full BFGS/direct inverse full BFGS or
> LBFGS. I wanted fminunc to stay close to fsolve and exploit the
> "duality" between solving nonlinear systems and minimizing
> functionals, and to be suitable for small or moderate size problems,
> just like fsolve is. I think that a basic, Matlab-compatible local
> minimization tool should be part of Octave core, but an user seriously
> concerned with optimization should undoubtedly get the optimization
> toolbox.
> The optim toolbox needs some clean-up and also should eventually use
> the optimget/optimset mechanism now provided by core Octave, that
> allows case-insensitive option setting and warns when options are
> misspelled.
> Any more testing/suggestions/code contributions etc. are surely welcome.
>
> regards
>
> --
> RNDr. Jaroslav Hajek
> computing expert & GNU Octave developer
> Aeronautical Research and Test Institute (VZLU)
> Prague, Czech Republic
> url: www.highegg.matfyz.cz
>

Hi Jaroslav,
Thanks for the explanation, and I'm glad to see optimization getting
some attention. I'm not too well informed, but what I have observed
suggests that the trust region approach is used more widely in more
recent work.  I don't know too much about it, though, and I can't
comment on the relative merits compared to line search. I would say
that there's no reason not to combine the two if it leads to a
flexible, robust, and reliable algorithm.  I agree that the optim
toolbox could use some work. My contributions there are bfgsmin and
samin. I would like to modify them to have an interface more in line
with the "industry standard". Right now, these algorithms allow for
multiple arguments to the function to be minimized, passed as a cell,
and the one to minimize with respect to can be specified. That works
well for me, but it is not standard. I need to study the optimset
mechanism, I still don't know how it works. I will try to spend some
time this summer on this. If you could have a look at __bfgsmin.cc for
improvements, I would appreciate it. With an eye to improvements, I
believe that there is probably some low hanging fruit in that code.
Cheers, Michael



reply via email to

[Prev in Thread] Current Thread [Next in Thread]