Re: fmincon wrapping nonlin_min

[Olaf Till]

On Thu, Jul 02, 2015 at 03:43:32AM +0100, Asma Afzal wrote:
> >> - Output Gradient and Hessian at the solution as in Matlab.
> >
> > What I currently think is we should introduce the options
> > 'ret_objf_grad' and 'ret_hessian' to nonlin_min, and compute a
> > gradient and Hessian for the final parameters if set. They could be
> > returned as .objf_grad and .hessian fields of the 'outp' structure
> > (fourth output argument of nonlin_min). And I'd say both or nothing
> > should be returned, regardless whether ret_hessian or ret_objf_grad or
> > both had been set. As long as I've not implemented this, you could
> > just set these options and check if the output fields are present.
> >
> In my current implementation, I am removing fields "objf_grad",
> "hessian" and "lambda" from outp
> 
>   if (out_args >= 4)
>     outp = min_out{4};
>     outp = rmfield (outp, {"lambda", "objf_grad", "hessian"});
>     varargout{4} = outp;
>   endif
> 
>   if (out_args >= 5)
>      varargout{5} = min_out{4}.lambda;
>   endif
> 
>   if (out_args >= 6)
>      varargout{6} = min_out{4}.objf_grad;
>   endif
> 
>   if (out_args >= 7)
>      varargout{7} = min_out{4}.hessian;
>   endif
> 
> This will only work when the lm_feasible algorithm is used. Should I
> just set lm_feasible inside fmincon? or have checks on each of these
> output arguments whether a field lambda/hessian etc exists in the outp
> structure.

Sorry for my late reply ...

I'd say always check for presence of these fields, and also if lambda
is a structure (since current sqp returns a vector). The more because
these features are not yet implemented in any backend of nonlin_min.

And for now it's probably also necessary to check before calling
nonlin_min:

any (strcmp (fieldnames (nonlin_min ("defaults")), "new_option_name"))

to avoid warnings due to unrecognized options.

As for the choice of the backend, it could depend on whether fmincon
(or its configured algorithm) performes feasible-path-optimization or
not. If the the parameters are supposed to respect all constraints
(including equality constraints) throughout optimization (but for
non-linear constraints, this is generally only possible at the end of
each iteration), 'lm_feasible' is suitable. If constraints need only
be respected for the final result, 'octave_sqp' may perform better (in
particular if non-linear constraints are given, I'd think).

You can probably test this in the original fmincon by printing the
parameter argument and the returned values in the function for
non-linear equalitiy constraints (and maybe you can even set an option
to indicate the starts of itereations?). If the returned values for
non-linear equality constraints assume the value zero several times
before the final function calls, this indicates it's a feasible path
algorithm. If they get zero only in the last function calls, it's
probably different.

But if nothing in the documentation of original fmincon mentions this
aspect, you are probably free to choose the algorithm which seems best
to you. An interesting choice, which seems quite suitable to me, would
be to use lm_feasible if no non-linear constraints are present and
octave_sqp in the other cases (non-linear constraints can't be
respected for each call of the user function anyway, even in a
feasible path algorithm).

> > But it would be good if you would provide some examples
> > for easy testing of your code.
> 
> Do you mean examples as tests for the function or just a collection? I
> am testing my functions with various examples in runlsqnonlin.m,
> runfmincon.m etc. in the repository but the files are cluttered. I can
> refine them if you say.

These example scripts are ok, sorry for the noise.

> - I added an additional feature in fmincon [1] to cater for non linear
> constraint function set using deal();
> Consider the example:
> 
> c = @(x) [x(1) ^ 2 / 9 + x(2) ^ 2 / 4 - 1;
>         x(1) ^ 2 - x(2) - 1];
> ceq = @(x) tanh (x(1)) - x(2);
> nonlinfcn = @(x) deal (c(x), ceq(x));
> obj = @(x) cosh (x(1)) + sinh (x(2));
> z = fmincon (obj, [0;0], [], [], [], [], [], [], nonlinfcn)
> 
> z =
>    -0.6530
>    -0.5737

With your current code, I get:

z = fmincon (obj, [0;0], [], [], [], [], [], [], nonlinfcn)
error: could not find stepwidth to satisfy inactive and non-binding general 
inequality constraints
error: called from:
error:   /usr/nfs/share/octave/packages/optim-1.4.1/private/__lm_feasible__.m 
at line 341, column 13
...

It seems that a feasible path algorithm like 'lm_feasible' has
difficulties with this problem.

Regards,

Olaf

-- 
public key id EAFE0591, e.g. on x-hkp://pool.sks-keyservers.net

signature.asc
Description: Digital signature