[FIXED] Non-linear Least Squares Fitting (2-dimensional) in Python

Issue

I was wondering what the correct approach to fitting datapoints to a non-linear function should be in python.

I am trying to fit a series of data-points

t = [0., 0.5, 1., 1.5, ...., 4.]
y = [6.3, 4.5,.................]

using the following model function

f(t, x) = x1*e^(x2*t)

I was mainly wondering which library routine is appropriate for this problem and how it should be setup. I tried using the following with unsuccessful results:

t_data = np.array([0.5, 1.0, 1.5, 2.0,........])
y_data = np.array([6.8, 3., 1.5, 0.75........])

def func_nl_lsq(x, t, y):
    return [x[0]*np.exp(x[1]*t)] -  y

popt, pcov = scipy.optimize.curve_fit(func_nl_lsq, t_data, y_data)

I know it's unsuccessful because I am able to solve the "equivalent" linear least squares problem (simply obtained by taking the log of the model function) and its answer doesn't even come close to the one I am getting by doing the above.

Thank you

Solution

First, you are using the wrong function. Your function func_nl_lsq calculates the residual, it is not the model function. To use scipy.otimize.curve_fit, you have to define model function, as answers by @DerWeh and @saullo_castro suggest. You still can use custom residual function as you like with scipy.optimize.least_squares instead of scipy.optimize.curve_fit.

t_data = np.array([0.5, 1.0, 1.5, 2.0])
y_data = np.array([6.8, 3., 1.5, 0.75])

def func_nl_lsq(x, t=t_data, y=y_data):
    return x[0]*np.exp(x[1]*t) -  y
    # removed one level of []'s

scipy.optimize.least_squares(func_nl_lsq, [0, 0])

Also, please note, that the remark by @MadPhysicist is correct: the two problems you are considering (the initial problem and the problem where model function is under logarithm) are not equivalent to each other. Note that if you apply logarithm to your model function, you apply it also to the residuals, and residual sum of squares now means something different. This lead to different optimization problem and different results.

Answered By - Ilya V. Schurov