[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Contribution to the optimization toolbox
From: |
Jaroslav Hajek |
Subject: |
Re: Contribution to the optimization toolbox |
Date: |
Tue, 8 Sep 2009 16:20:36 +0200 |
On Tue, Sep 8, 2009 at 1:11 PM, Michael Creel<address@hidden> wrote:
> Thanks for the new version. Note that when dim is increased, bfgsmin
> need to have max iters increased, otherwise it chokes before
> converging. Setting control={10000,0} with dim=100, I get the results
>
> octave:4> compare
> 100/100
> Results using analytic gradient
> ans =
>
> 3.1157e-01 4.6628e-08 1.2482e-15 4.3526e-01 6.0199e-02 1.6613e-03
>
> ans =
>
> 2.9021e-01 2.9620e-08 3.5626e-16 4.3474e-01 4.5174e-02 5.0746e-04
>
> so bfgsmin is finding the correct answer.
>
>
> When dim is high, it makes sense to try lbfgs. Setting
> control={10000,0,1,1,5} to use a memory of 5 iterations, the results
> for dim=100 are
>
> octave:6> compare
> 100/100
> Results using analytic gradient
> ans =
>
> 9.0097e-02 1.1931e-06 1.6012e-12 4.3407e-01 5.7046e-02 1.5292e-03
>
> ans =
>
> 8.7479e-02 3.4221e-07 4.5890e-14 4.3370e-01 3.7205e-02 3.5980e-04
>
> so lbfgs is considerably faster than bfgs.
>
> For setting up a bank of test problems, do you think that this test
> script captures the important metrics: speed and accuracy? Anything
> else to add? I may take the project on,
>
> Cheers, M.
>
Looking at the problem from a slightly different perspective, it now
seems to me that taking the dogleg TR step selection method from
fsolve may not have been that good idea as I thought initially. For
fsolve, it shines because fsolve updates the model after each trial
step, even an unsuccessful one. Uhm. Maybe a line search technique
would be better. At least that would allow simply employing all three
updating techniques in fminunc: BFGS, factored dual BFGS, and LBFGS.
Btw., when I noticed that the rosenbrock function is actually a sum of
squares, just for fun I tried rewriting the problem as a nonlinear
least squares and feed it to fsolve. Doing thath, fsolve literally
crushed the problem converging in 7 iterations (8 function and one
jacobian evaluation). But of course that is a different story; it
exploits the structure of the problem.
Do you have any references for the line search used in bfgsmin?
--
RNDr. Jaroslav Hajek
computing expert & GNU Octave developer
Aeronautical Research and Test Institute (VZLU)
Prague, Czech Republic
url: www.highegg.matfyz.cz
- Re: Contribution to the optimization toolbox, (continued)
- Re: Contribution to the optimization toolbox, Michael Creel, 2009/09/04
- Re: Contribution to the optimization toolbox, Jaroslav Hajek, 2009/09/06
- Re: Contribution to the optimization toolbox, Michael Creel, 2009/09/07
- Re: Contribution to the optimization toolbox, John W. Eaton, 2009/09/07
- Re: Contribution to the optimization toolbox, Michael Creel, 2009/09/07
- Re: Contribution to the optimization toolbox, Jaroslav Hajek, 2009/09/07
- Re: Contribution to the optimization toolbox, Michael Creel, 2009/09/08
- Re: Contribution to the optimization toolbox, Jaroslav Hajek, 2009/09/08
- Re: Contribution to the optimization toolbox, Michael Creel, 2009/09/08
- Re: Contribution to the optimization toolbox, Jaroslav Hajek, 2009/09/08
- Re: Contribution to the optimization toolbox,
Jaroslav Hajek <=
- Re: Contribution to the optimization toolbox, Jaroslav Hajek, 2009/09/08
- Re: Contribution to the optimization toolbox, Michael Creel, 2009/09/08