I just downloaded one of the files in the repository that you linked to (brazil_yield.asc) and tried to import the file into another GIS program. I also ran into an import error. I finally tracked the error down to an attempt to convert a blank cell to a numerical type. In other words, there are blanks in the file because they have been improperly formatted. Rewriting the import tool to ignore blank cells resulted in a successful import. There are several places in the file that have two spaces in a row. See the figure below.
For a proper space delimited file, there can only be one space between each number.
This is probably easiest to do using gdal_translate. I'm sure it can also be done within QGIS itself, I'm just more used to using the command line :)
Let's break it down to converting one file.
I downloaded the first map from here for an example, a georeferenced antiquarian map of Edinburgh in georeferenced JPEG format.
It doesn't appear to have a CRS attached, but no problem... we can set this with QGIS. I also had to add a NODATA value of 0 in the transparency tab to get rid of the black border...
The following is a suggestion - check out the manual page as there may be other settings you'll need in your case.
It's possible to apply JPEG compression to a TIFF to keep the file size down. This only makes sense for scanned maps and aerial imagery, though.
In my case the file is 74400010.jpg, so I gave the same name to the tiff
gdal_translate -of Gtiff -co COMPRESS=JPEG -co JPEG_QUALITY=90 -co PHOTOMETRIC=YCBCR 74400010.jpg 74400010.tiff
You should now have a geoTIFF. This stores the metadata (equivalent to the jpw world file) in the tiff file itself.
Looking at the two versions side-by-side, I see no quality degradation:-
In my exampe the JPEG was 7.1Mb in size, the TIFF slightly bigger (11 Mb). Without applying JPEG compression the tiff is 133Mb, more than 10 times bigger.
I used a fairly conservative quality setting, you can reduce that value or leave it out (default is 75). Only your eyes will tell you where the quality/file size trade-off is right. With my example map, I could get a TIFF that was smaller than the original JPEG, but still acceptable quality.
If you're planning to serve these over a web service like WMS/WMTS, you should look into using the various tiling options, and consider using gdaladdo to add overviews (pyramids).
I'm sorry that doesn't answer the 'how to batch process' question, but I'm not sure whether you're using Windows/Mac/Linux. But it would be fairly easy to script this in Python or with a shell script (or whatever the Windows equivalent is these days!)
Best Answer
Based on the User's comment that they would be open to trying this in R, here's the solution: