Re: bfgsmin and fminunc

[Michael Creel]

On Mon, May 18, 2009 at 10:47 AM, Jaroslav Hajek <address@hidden> wrote:
> On Mon, May 18, 2009 at 10:29 AM, Michael Creel <address@hidden> wrote:
>>
>> Hi Jaroslav,
>> Thanks for the explanation, and I'm glad to see optimization getting
>> some attention. I'm not too well informed, but what I have observed
>> suggests that the trust region approach is used more widely in more
>> recent work.  I don't know too much about it, though, and I can't
>> comment on the relative merits compared to line search. I would say
>> that there's no reason not to combine the two if it leads to a
>> flexible, robust, and reliable algorithm.  I agree that the optim
>> toolbox could use some work. My contributions there are bfgsmin and
>> samin. I would like to modify them to have an interface more in line
>> with the "industry standard". Right now, these algorithms allow for
>> multiple arguments to the function to be minimized, passed as a cell,
>> and the one to minimize with respect to can be specified. That works
>> well for me, but it is not standard.
>
> Yes, in some cases that approach is convenient, and in the past it was
> even necessary to avoid global variables, but I now consider it
> superseded by anonymous function handles, which provide a more general
> and flexible convenient mechanism to accomplish the same:
>
> fminunc (@(x) my_func(x, y, z), x0,....)
>
> the *big* advantage here is that the argument selection and binding
> part is kept out of the function; fminunc just deals with a single
> function of a single argument and stays simple. Thus, this mechanism
> works exactly the same for all optimization, integration and any other
> functions. The disadvantage, at least for now, is somewhat lower
> performance, but I have plans for improvements here.
>
>
>> I need to study the optimset
>> mechanism, I still don't know how it works. I will try to spend some
>> time this summer on this.
>
> Basically, Octave's optimset/optimget now maintain internally a list
> of "registered options" (see __all_opts__).
> When you set an option via optimset, it looks up the list and possibly
> adds proper casing, otherwise it warns you if an unregistered option
> is being set, such as when you type "GradientObj" instead of "GradObj"
> or so. Options are registered at Octave's startup using a "default"
> query for each registered function. In a package, newly registered
> function are added using PKG_ADD. See Octave's fsolve, fminunc, fzero
> and lsqnonneg. Other functions (sqp, qp, glpk) either do not yet
> conform to this interface, or do not need it (qp).
>
>> If you could have a look at __bfgsmin.cc for
>> improvements, I would appreciate it. With an eye to improvements, I
>> believe that there is probably some low hanging fruit in that code.
>> Cheers, Michael
>>
>
> I plan to do some work here, but probably not until Octave 3.2 gets out.
>
> cheers
>
>
> --
> RNDr. Jaroslav Hajek
> computing expert & GNU Octave developer
> Aeronautical Research and Test Institute (VZLU)
> Prague, Czech Republic
> url: www.highegg.matfyz.cz
>

Another suggestion would be to add an example script that shows how to
use anonymous functions, set options, etc. I find this stuff a little
confusing, though I haven't yet made much effort to understand it. For
me, a working example always helps a lot.
M.