Solved – method for predicting a curve

constrained regressioncurve fittingexponential-smoothing

I have data on several curves. the data is of the form:
curve_id x y

and there are many x/y pairs for each curve and x is limited to some known range. overall, the curves look quite similar in shape, although, there is of course some noise in the data, and the location of the curves change from one another. some are a bit stretched upward or leftward and so on.

I want to train a model to predict the curve as a whole. at prediction time, I will have very few x/y pairs for the new curve (their number is fixed and known in advance), and I would like to come up with the formula for the curve based on the few "anchor" points.

so I thought about fitting a polynomial or other combinations exponents of x to the data. However, this is not a standard curve fitting in the sense that every curve should have a different equation and the anchor points should help predict the equation.

Also, since there is a limited number of points (let's say K), I can only fit K different exponents, and I think that the main challenge is to find out, based on the data which exponents to use. then, once the new data arrives I can quickly solve for the coefficients.

So, how do I find which K exponents gives the best fit over all curves I have in the training set?

Note: when I played with the data I found out the adding x and x^(-0.5) got me real close. closer than any standard polynomial I tried, which means I am searching K-tuples of exponents that are not necessary positive integers. they can be negative, fractions, etc.

Is there literature on how to search this "exponent space"?

Thanks.

Best Answer

It is not clear from your question whether this is time series data or not. If it is then you would use a time series model. A plot of the data would be nice. Otherwise for non-time series data, playing around with polynomials (or exponentials) is a bad idea. A much more better approach would be to use (natural) splines. This should be available in most software and is much more flexible. With a spline you would define the number of "knots" (or anchor points if you will).

Related Solutions

Solved – Theil-Sen estimator for polynomial

James Phillips' suggestion on how to expand the Theil-Sen algorithm to a second degree polynomial worked surprisingly well. There were 762 (x,y)-points in the dataset I tested. Selecting three different points from the 762 can be made in 73 million ways, so instead I put the points into groups of 11 and calculated the median x och y values of each group. $\binom{762/11}{3} = \binom{69}{3} \approx 52000$ which is a more reasonable number of combinations to use.

For each combination of three points, I calculated the $a$ coefficient for $y = ax^2 + bx + c$ using:

$a = \frac{x_3(y_2 - y_1) + x_2(y_1 - y_3) + x_1(y_3 - y_2)}{(x_1 - x_2)(x_1 - x_3)(x_2 - x_3)}$

Using the median of all $a$'s, for each combination of two points I calculated the slope of the line $y - ax^2 = bx + c$ using:

$b = \frac{(y_2 - ax_2^2) - (y_1 - ax_1^2)}{x_2 - x_1}$

Finally, using the the medians of all $a$'s and $b$'s, for each point I calculated the intercept of $y - ax^2 - bx = c$ using:

$c = y_n - ax_n^2 - bx_n$

I used the Remedian median value approximation and the algorithm implemented in C# took ~15 ms to run for 69 points. If the initial median filter is reduced to 5 points, the execution time increases to ~110 ms which still is ok. Running the algorithm on all 762 points results in an execution time of 13 seconds.

Even with the fast 11 point filter, the results looks very good:

Curve Fitting – Fitting the Second Integral of a Normal Distribution

Let $\Psi(z) = \int_{- \infty}^z \Phi(t) \> dt$. Note $\Phi$ is approximately constant as $\vert z \vert \to \infty$. This means that that $\Psi$ is going to be linear in these same regions.

Because of this behaviour, I think it is reasonable to use a natural spline to approximate $\Psi$. A Natural spline is linear in the tails of the data (just like our target function). The question becomes: where the knots should be placed? I'm sure the knots could be placed intelligently leveraging properties of $\Phi$ and the Gaussian density, but I'll just let the rms library pick them for me today. If you were able to intelligently select knot locations, then you could pass them to rcs using the parms argument.

Using R...

library(pracma)
library(rms)

# Generate data
x = seq(-8, 8, 0.25)
f = pnorm(x)
Psi = cumtrapz(x, f)

train_ix = abs(x)<=5
test_ix = abs(x)>5

xtrain = x[train_ix]
xtest = x[test_ix]

ytrain = Psi[train_ix]
ytest = Psi[test_ix]


model = ols(ytrain ~ rcs(xtrain))

plot(x, predict(model, newdata=list(xtrain=x)), col='red', type='l')
points(xtrain, ytrain)
points(xtest, ytest, col='blue', pch=2)

Which produces the following fit

The triangles here are data which were not used to fit the model. Because we have used a natural spline (which is linear in its tails) we do fairly well. Shown below is the absolute error between model and integral (pay attention to those $x$ which are larger than 5 in absolute value, those are the test points)

Because $\Phi$ is not actually linear in the tails (although nearly) our model suffers slightly as evidenced by the growth in absolute error. The relative error is quite good in the right tail, but appears to explode in the left tail since $\Psi$ is quite small in those regions.

As for a model formula, here it is

Apologies for the awful formatting, I the latex function from rms doesn't seem to play nice with typesetting here.

Best Answer

Related Solutions

Solved – Theil-Sen estimator for polynomial

Curve Fitting – Fitting the Second Integral of a Normal Distribution

Related Question