MATLAB: How to fit a line through this data “smartly”

curve fittingMATLAB

I have the x-y data shown in the figure below (for the curious, it is the logarithm of the amplitude of a Hilbert transform).
I'm trying to write an algorithm to automatically get the slope of the initial portion of the plot, i.e. before the noise overcomes the signal.
What's causing the task to be even more challenging, are the small saw-tooth jumps that you can see in the zoomed-in portion on the right.
I tried several approaches, but could not find a reliable way to automatically fit a line only of the left-hand side linear portion.
Do you have any suggestion?
PS: I know it would help, but I don't (and cannot have) the curve fitting toolbox.

Best Answer

One approach (using the Signal Processing Toolbox medfilt1 function):
D = load('data.mat');
X = D.X;
Y = D.Y;
Ts = mean(diff(X));
YF = medfilt1(Y, 2500);
dYdX = gradient(YF,Ts);
gt0 = find(dYdX > -0.001, 1, 'first');
b = polyfit(X(1:gt0),Y(1:gt0),1);
Yf = polyval(b, X(1:gt0));
figure
plot(X, Y)
hold on
plot(X(1:gt0), Yf,, ':r', 'LineWidth',2)
hold off
text(10, -5, sprintf('y = %6.3f x %6.3f', b))
This code does a median filter (medfilt1) on ‘Y’, then uses the gradient function to calculate the derivative. It then uses a relatively unsophisticated threshold (the find call) to determine the end of the relatively linear section of the data, does a linear regression on that section, and plots the result.
I cannot determine how well it will work with any other data you have, so you will likely need to experiment with it.
EDIT — (13 Jan 2019 at 18:05)
Added plot image.
EDIT #2 — (13 jan 2019 at 18:50)
Another option, using the Signal Processing Toolbox findchangepts function:
D = load('data.mat');
X = D.X;
Y = D.Y;
ipt = findchangepts(X, 'Statistic','std');
b = polyfit(X(1:ipt), Y(1:ipt), 1);
Yf = polyval(b, X(1:ipt));
figure
plot(X, Y)
hold on
plot(X(1:ipt), Yf, ':r', 'LineWidth',2)
hold off
text(10, -5, sprintf('y = %6.3f x %6.3f', b))
This may be more robust. It produces similar statistics and plot.