kavka

I am trying to implement a robust estimation algorithm instead of nonlinear
least squares minimization:

My nonlinear least-squares cost if of the form:

J(e) = \sum_{i=1}^{N} [y_i - f(u_i, x)]^2, (Eq. 1)

where f= f(u,x) is the model output for an input "u" and x is the vector (in
\R^p) of model parameters.
The variable y_i is the measured output for an input u_i at time i.

I have an algorithm to solve the above LS problem given a nonlinear function
f(u,x) and the set { (u_i,y_i), i = 1,...,N }.


I want to use the same algorithm with minimal modifications to solve the
following robust minimization problem (which I've seen in a book):

\sum_{i=1}^{N} \psi(y_i - f(u_i,x)) grad_{x_k}(f) = 0, (similar to normal
equations) (Eq. 2)

k = 1, ..., p,

where \psi(x) is the Huber function:

\psi(x) = max(-k, min(x,k)),

and grad_{x_k)(f) is the gradient vector of partial derivaties: df/dx(k), k
= 1,...,p.


My problem is that I don't know how to pose the robust problem, Eq. 2,
(solution of the above "normal equations") in a similar form to the least
squares minimizations problem, Eq. 1.

I suspect it is NOT simply


J_r(e) = \sum_{i=1}^{N} \psi(y_i - f(u_i, x))^2.

arnold neumaier

It is
J_r(e) = \sum_{i=1}^{N} \phi(y_i - f(u_i, x)),
where phi is the integral of psi. Setting the gradient to zero
gives the above conditions.

Arnold Neumaier

alienseducer

Hi
I would like to know what Algorithm you use to solve this problem by
nonlinear least-squares.
Algorithm, you only provide (u_i,y_i) and the nonlinear function.

Thanks.
Alien+

paul victor birke

of some interest is the smoothed huber function found in

http://isi-eh.usc.es/trabajos/80_30_fullpaper.pdf

kavka

I use "lsqnonlin" in Matlab's Optimization Toolbox. I have an m-file, say
fun.m, in which I implement my nonlinear function "f(u;x)" given the set of
model parameters,x, and the set of inputs, {u_i}. The output of the model
is \hat{y}_i, which is compared with experimental data, y_i. The
differences are the residuals, which I try to minimize using the cost
function J(e) given below.