This is a question addressed to those familiar with category theory, since it concerns the preferability of two similar definitions, which I suppose is revealed only by further experience. In sum, the question is whether or not to mean functor as a property of couples of functions or as to couple of functions with built-in domain and codomain (which are categories). I point out that I myself include domains and codomains in the definition of (set-theoretic) functions.
The two definitions of functor are the following:
- according to the first one, a functor can be defined as a(n ordered) quadruplet in which the first two components are categories, called respectively domain and codomain of the functor, and the others are functions (possibly class functions) between the objects and the morphisms of the latter categories, satisfying appropriate conditions;
- according to the other, one says that a couple of functions is a functor from category $C$ to category $D$ if the first (second) one maps objects (morphisms) of $C$ to objects (morphisms) of $D$ satisfying the same conditions as case (1).
The two definitions are distinct, since in the second case a functor is a particular couple of function (so 'to be a functor from $C$ to $D$' is precisely a property of couples of functions), whereas it is not that case in definition 1.
As an other example, definition (2) allows a couple of functions to be a functor for multiple categories in ways (1) does not: e.g., a (2)-functor from $C$ to $D$ is also a (2)-functor from $C^\text{op}$ to $D^\text{op}$, whereas according to (1) it generally isn't (clearly enough, unless $C^\text{op} = C$ and $D^\text{op} = D$).
Best Answer
The answer you probably don't want is that these kinds of "implementation details" really don't matter at all. Pick whichever makes you happiest, and it will never impact the answers to category-theoretic questions.
First, let me give a simpler example to make sure that I understand what you're asking:
We have "two possible implementations" for a group homomorphism $(G, \cdot) \to (H, \star)$.
Now you worry that, under definition $2$, two different homomorphisms can have the same implementation. For instance, if we define $\cdot^\text{op} : G \times G \to G$ as $a \cdot^\text{op} b = b \cdot a$, then the same function $f$ can represent both a homomorphism $(G, \cdot) \to (H, \star)$ and to $(G,\cdot^\text{op}) \to (H,\star^\text{op})$ because these two homomorphisms have the same underlying function!
In definition $1$, we explicitly "tag" our function with its domain and codomain as groups so that the two homomorphisms $((G,\cdot), \ (H,\star), \ f)$ and $((G,\cdot^\text{op}), \ (H, \star^\text{op}), \ f)$ are encoded by different objects in the ambient set theory.
If I understand correctly, you're asking basically the same question, but in the context of categories and functors rather than groups and homomorphisms.
Now, in the group theoretic case and the category theoretic case, these implementation details don't matter. You will never be presented with a naked function out in the wild (or pair of functions in the category case) and asked "what is the group structure on the domain of this function?". Ironically, given the question, we can make this precise using category theory! Whenever we interact with a homomorphism $f$, we're really interacting with $f \in \text{Hom}((G,\cdot), (H,\star))$! We always have the bonus context on hand of what the domain and codomain should be. So, if you want${}^1$, you can absolutely use definition $1$ and tag your homomorphisms (functors) with the group structure (category structure) on their source and target.
Why does category theory make this precise? Because we can define categories of groups $\mathsf{Grp}_1$ and $\mathsf{Grp}_2$ using the two implementations of homsets. It's an easy exercise to show that these categories are equivalent (in fact, isomorphic!) so that, in this precise sense, you cannot distinguish the two definitions using only group theoretic questions. Similarly, you can build two categories $\mathsf{Cat}_1$ and $\mathsf{Cat}_2$ using the two definitions of functor, and you can show that these categories are also isomorphic${}^2$! So the answer to any "category theoretic question" you can ask won't depend on whether you choose definition 1 or 2!
I hope this helps ^_^
${}^1$: and it seems like you do want this, since you say you're doing this for your arrows in $\mathsf{Set}$
${}^2$: you do have to eat your tail a little bit since checking isomorphism of categories requires working in a category of categories. And in this category of categories you need a notion of functor, so you have to choose a definition before you can prove the two definitions agree... But this is really no different than any other time we have two equivalent definitions and we have to choose one "official" one to write down first.