The jaggedness is called aliasing and it happens when in image, shape or function is downsampled, for instance when an exact function like sin(x) is shown at 100x100 pixel resolution.
If you want to save figures without aliasing you could use export_fig as darova suggested but I find it overrated. You would probably be able to solve the problem by just saving the figure with a higher resolution, like this:
x = 0:0.01:2.*pi;
y = sin(x);
figure
plot(x,y)
print('lots_of_aliasing.png','-dpng','-r80')
print('less_aliasing.png','-dpng','-r200')
However, in your question you show a plot inside an open Matlab figure window rather than a saved file. If you want to reduce aliasing in matlab figure windows I don't know if that's really possible. It would depend on your graphics hardware+software and probably the resolution of the monitor you view it on, you could check out these threads and maybe there is some gold to be had there:
Hope this helps,
M.
Best Answer