In short, the problem is that when Geoereferencer is reopened with a previously edited raster, all the previously saved control points have moved off the image to a location thousands of kilometres away. (QGIS 2.6, Windows 64 bit).
In Georeferencer, I create a number of GCPs (i.e. control points) by referencing to objects in the map canvas. After adding points, I start the georeferencing calculation. Operation is successful, the raster appears on the map canvas, along with the control points. The control points are also visible in the Georeferencer window, still overlying the points chosen for georeferencing. I save the control points and close the Georeferencer. All good so far.
Now for the problem. Having closed the Georeferencer, I reopen it, and open the new, georeferenced, raster image. But now, the control points in the Georeferencer window no longer overlay the raster in the Georeferencer window. However, they do appear correctly on the map canvas (the main QGIS screen). In the Georeferencer window, they have in fact displayed, but half a world away to the south. That is, if the raster covers a rectangle with coordinates 300,000, 5834300 ; 362100, 5782700, the control points occupy an area bounded by coordinates 0,0 ; 6800, -5900. These small coordinates seem to correspond to the number of pixels in the raster image. So, it is no longer practically possible to edit the GCPs.
How do I solve the problem?
See screenshots below.
Image 1: Georeferencer window, and map canvas. After reopening Georeferencer, GCPs do not show in window, but show on canvas (the small red dots).
Image 2: Zooming out in Georeferencer window shows the GCPs are there but far away.
Best Answer
The GCP points do the matching between the source coordinates of the image and the destination coordinates of the CRS. In the first run, your image has only pixel coordinates, and the table is correct.
Once you have georeferenced the image, the georeferencer takes the georeferenced coordinates as source CRS, that's why the points do not match anymore.
Another irritating thing about the georeferencer is that it takes the srcY coordinates all as negative values, while GDAL (which works in the background) needs them inverted to positive values. These inverted values are added to your image, and will not match to the ones you see on the screen. You could try removing the minus from srcY to see if that makes it better.
You don't mention whether you saved the georeferenced image to another filename. That would keep the original file clean. Otherwise you would have to make a copy of the original before georeferencing.