Using the adjustbox package with the export option which extends graphicx with several new keys you can now simply use the max width key which scales the image down to that width if it is larger but doesn't scale it at all if it is smaller the equal to it. There are also the related keys max height, max size (for both width and height) as well as min width, min height and min size.
In your case simply use the following code:
\includegraphics[max width=\linewidth]{<image file name>}
Note that you can't set max width etc. in a global way using \setkeys{Gin}{max width=...}. It is implemented in a different way as width. However, I'm planning to add a possibility for this as well.
Please always show a complete document that generates the problem. You show some code and you show an error message, but not the code that generates the error.
TeX is a macro expansion language you can almost always put any sequence of commands in a macro and they work the same way. Here, where no catcode/verbatim trickery is used it is certainly possible.
\def\first{\includegraphics{#2}}
\def\first{\includegraphics{#3}}
\setlength{\gapspace}{#1}
That middle line should presumably be defining \second.
\setlength{\effwidth}{\textwidth-\gapspace}
would normally be a synax error but as you say it works, presumably you have loaded calc package? (Again it is much easier if you show a complete document with all relevant packages)
The syntax for \newcommand that you show is incorrect
\newcommand{\calcoptimalheight}{3}[0.5cm]{
Is not a syntax error but 3 is the entire body of the definition, you want
\newcommand{\calcoptimalheight}[3][0.5cm]{%
You probably want to add % at the end of every line as well.
Best Answer
You can mark the bases of lines as "labels" using
zref'ssaveposmodule. This allows you to do calculations on the (x and) y coordinates of the labels. The command\filltopageendgraphics[<options>]{<file>}below automates this process (assuming you'll only have one such image on any given page).