I would probably go with your original model with your full dataset. I generally think of these things as facilitating sensitivity analyses. That is, they point you towards what to check to ensure that you don't have a given result only because of something stupid. In your case, you have some potentially influential points, but if you rerun the model without them, you get substantively the same answer (at least with respect to the aspects that you presumably care about). In other words, use whichever threshold you like—you are only refitting the model as a check, not as the 'true' version. If you think that other people will be sufficiently concerned about the potential outliers, you could report both model fits. What you would say is along the lines of,
Here are my results. One might be concerned that this picture only emerges due to a couple unusual, but highly influential, observations. These are the results of the same model, but without those observations. There are no substantive differences.
It is also possible to remove them and use the second model as your primary result. After all, staying with the original dataset amounts to an assumption about which data belong in the model just as much as going with the subset. But people are likely to be very skeptical of your reported results because psychologically it is too easy for someone to convince themselves, without any actual corrupt intent, to go with the set of post-hoc tweaks (such as dropping some observations) that gives them the result they most expected to see. By always going with the full dataset, you preempt that possibility and assure people (say, reviewers) that that isn't what's going on in your project.
Another issue here is that people end up 'chasing the bubble'. When you drop some potential outliers, and rerun your model, you end up with results that show new, different observations as potential outliers. How many iterations are you supposed to go through? The standard response to this is that you should stay with your original, full dataset and run a robust regression instead. This again, can be understood as a sensitivity analysis.
Best Answer
1. A data point can still be considered influential if it has a large Cook's Distance, even if it has a low standardized residual.
The following image (taken from p214 of Andy Field’s Discovering Statistics Using IBM SPSS 3e) may help to clarify the difference between these two concepts.
The red line depicts the regression model, while the dotted blue line represents the regression model if data point 8 is removed. Note that data point 8 has a very small residual statistic, as it’s very close to the red line. However, it would have a massive influence statistic (according to the textbook, it has a Cook’s distance of 227.14!) since the model changes radically when it is omitted.
2. I don't think it's right to say "Sometimes when we delete an influential point, the regression line does't change much".
If the regression model didn't change much with the omission of the data point, then I don't think it's fair to say that the data point was influential.
Here's Wikipedia's definition of an influential data point: