octave-maintainers
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Contribution to the optimization toolbox


From: Jaroslav Hajek
Subject: Re: Contribution to the optimization toolbox
Date: Tue, 8 Sep 2009 16:20:36 +0200

On Tue, Sep 8, 2009 at 1:11 PM, Michael Creel<address@hidden> wrote:
> Thanks for the new version. Note that when dim is increased, bfgsmin
> need to have max iters increased, otherwise it chokes before
> converging. Setting control={10000,0} with dim=100, I get the results
>
> octave:4> compare
> 100/100
> Results using analytic gradient
> ans =
>
>   3.1157e-01   4.6628e-08   1.2482e-15   4.3526e-01   6.0199e-02   1.6613e-03
>
> ans =
>
>   2.9021e-01   2.9620e-08   3.5626e-16   4.3474e-01   4.5174e-02   5.0746e-04
>
> so bfgsmin is finding the correct answer.
>
>
> When dim is high, it makes sense to try lbfgs. Setting
> control={10000,0,1,1,5} to use a memory of 5 iterations, the results
> for dim=100 are
>
> octave:6> compare
> 100/100
> Results using analytic gradient
> ans =
>
>   9.0097e-02   1.1931e-06   1.6012e-12   4.3407e-01   5.7046e-02   1.5292e-03
>
> ans =
>
>   8.7479e-02   3.4221e-07   4.5890e-14   4.3370e-01   3.7205e-02   3.5980e-04
>
> so lbfgs is considerably faster than bfgs.
>
> For setting up a bank of test problems, do you think that this test
> script captures the important metrics: speed and accuracy? Anything
> else to add? I may take the project on,
>
> Cheers, M.
>

Looking at the problem from a slightly different perspective, it now
seems to me that taking the dogleg TR step selection method from
fsolve may not have been that good idea as I thought initially. For
fsolve, it shines because fsolve updates the model after each trial
step, even an unsuccessful one. Uhm. Maybe a line search technique
would be better. At least that would allow simply employing all three
updating techniques in fminunc: BFGS, factored dual BFGS, and LBFGS.

Btw., when I noticed that the rosenbrock function is actually a sum of
squares, just for fun I tried rewriting the problem as a nonlinear
least squares and feed it to fsolve. Doing thath, fsolve literally
crushed the problem converging in 7 iterations (8 function and one
jacobian evaluation). But of course that is a different story; it
exploits the structure of the problem.

Do you have any references for the line search used in bfgsmin?

-- 
RNDr. Jaroslav Hajek
computing expert & GNU Octave developer
Aeronautical Research and Test Institute (VZLU)
Prague, Czech Republic
url: www.highegg.matfyz.cz



reply via email to

[Prev in Thread] Current Thread [Next in Thread]