There are multiple Issues at hand, and I think we should handle them one by one.
I feel that you are trying to ask
How to georefrence a map so as to have the least RMS error?
If this is so, I would suggest that you edit your question, and change the title accordingly.
To understand how to reduce the RMS error, you need to understand what RMS Error means.
Suppose there are n points; For each point, you have the coordinates that you have entered, and you have the coordinates that are calculated. The difference between these is calculated using simple euclidean geometry, and this is called the error.
To get the overall error, we add up these errors. we don't take a simple arithmetic mean, but use an RMS of these errors. There are many scientific reasons for this, but my statistical knowledge is far too weak to explain it to you.
So basically you calculate the RMS error using the Following Formula:
RMS error=Square Root(Σ(e^2)/n)
Now coming to the question that you are really asking. How can we reduce this RMS Error? To do that you need to pay attention to how the calculated coordinates are actually calculated. There are two main points to tackle here:
Firstly you need to select the proper transformation for georeferencing. There are multiple transformations (affine/Spline, 1st order, 2nd order and so on). I can best quote whuber, who in this excellent answer says:
Use a method that can represent the distortions that might have occurred. With paper map scanning, the distortions can be local and irregular, so consider splines. With changes of projection (including those that occur in most aerial and satellite image processing) the proper transformation to use is a projective one. Projective transformations are neither polynomials (in general) nor splines.
Secondly, you need to take care while selecting the control points for your georeferencing. Whuber in his answer linked above makes multiple pointers in this direction.
You need to select points that would be present in both the images. Things like monuments, road crossings, permanent structures etc are usually used. Try to use objects at, or closer to ground level. Do not use high building, church spires, or towers, like you have mentioned in the question.
The reason is simple. Most rasters are takes from an angle, and provide an oblique view. Hence tall objects will appear to lean in an direction pointing outwards from the Focal Axis of the sensor. For example look at the following Google Maps Image of the Eiffel Tower. The Red point is approximately where the center should be, but you see the top of the tower at the Cyan Point. (This is just illustrative. Google Map's Satellite view is processed to removed these kinds of artifacts, but many still remain)
The coordinates in your gdal_translate command should be all positive.
The GCP list is, so I think this could be a bug of the GDAL script creator.
EDIT
I could replicate your error, and think that your use case is not the intended use of GCP.
In "normal" georeferencing, the GCP points map pixel coordinates from an unreferenced picture to projected or geographical coordinates. The pixel coordinates are relative to the upper left corner.
In your use case, you take projected coordinates from a georeferenced tif, and compare them to "correct" coordinates. The coordinates on the left side are not relative to the upper left corner, but to the origin of the projection. If you start the reprojection process, this information will be distroyed and replaced by the GCP information. This does not seem to work.
In the GDAL batch file, you can add -of VRT to get vrt instead of tif files. You can open those vrt in a text editor and see that the matching goes terribly wrong.
Best Answer
This happens when you do the update georeferencing for the second time in another word the raster image already has a world file associated with.
Workaround: either using "Rectify" rather than update georeferencing or delete all associated files