The problem is due to unclear meaning of the sentence
I would have accomplished zero "net work".
In a colloquial sense, you did not do any useful work, so someone, perhaps your boss, would evaluate your trip as "you did no work". But in the sense word "WORK" is used in physics, the answer may be different. This is because the sentence "Body A did non-zero work on body B" in physics means that the body $B$ moved while $A$ exerted force on it which had non-zero component in the direction of motion of $B$.
Now, if we take your body as the object $A$ and the ground as the object $B$, there is indeed work done, because the ground moves to your back while you're pushing it there with your feet (you have to push to overcome air resistance). So in this sense, you did work on the ground, and your friend was mistaken.
The total work being zero when object returns to its original position is a situation that occurs when the work is taken as special work due to one special force, for example the gravity force. Then, because for the gravity force $\mathbf F_G$ the integral over every closed path $\gamma$
$$
\oint_\gamma \mathbf F_G \cdot d\mathbf s = 0,
$$
the gravity force did zero net work.
The first thing you need to understand: you are applying the creation of physics definitions backwards. You are asking, "why isn't work given by this equation?", but this question doesn't make sense if you think about it. It is not the case in physics where we think, "Hmm... I want to define something called "work". What should it's equation be?" This doesn't make sense, as the only use an equation has in physics is how useful it is in describing the world around us. So it is fine to ask "how is the concept of work that is defined in this way useful?", but a question of "why isn't work defined to be this instead?" is not a valid question.
So why is this definition of work useful? There are many reasons, but the simplest case, and the one where it is usually first introduced to students, is that the net work done on an object is equal to the change in the object's kinetic energy, which is another useful concept that has been defined in physics. In an equation, this is $$W_\text{net}=\Delta K$$ where $K$ is the kinetic energy $K=\frac12mv^2$ for an object with mass $m$ moving with a speed of $v$. A more general derivation can be made for variable acceleration in multiple dimensions, but as an introduction you can use your constant acceleration equations to arrive at this for when $W=Fd$ is valid.$^*$ You will probably encounter more instances where the concepts of "energy" and "work" become useful, but essentially whenever we are talking about changes in some type of energy work must be involved. The definition of work then is essential when thinking about energy.
What about your other expression? As described above, there is no reason to call this "work" over what we already have, but it can still be useful. $mat$ is, in fact, a change in momentum. In general for constant mass systems, $$\mathbf F=ma=m\frac{\text d\mathbf v}{\text dt}=\frac{\text d\mathbf p}{\text dt}$$ i.e. forces cause changes in momentum over time. For constant acceleration motion, we arrive at your expression: $F\Delta t=ma\Delta t=\Delta p$. This has a special name: impulse. So just like how work deals with a force being applied over a distance, impulse deals with a force being applied over some time period. They are both useful concepts, but to ask why one is called "work" over the other isn't a question worth asking here.
$^*$Just use your kinematic equation $v^2-v_0^2=2ad$ and Newton's second law $F=ma$ to easily show that $W=\Delta K$ for motion under constant acceleration in one direction.
It is worth noting that the equation $W=Fd$ is only valid when the force is constant in magnitude and along the direction of displacement during the entire path of interest. The more general definition of work takes this idea and breaks up a general path into small pieces where $W=Fd$ applies. Then all of the "little works" are "added" together in an integral
$$W=\int_C\mathbf F\cdot\text d\mathbf x$$
where $C$ is the path we are adding up the work on, $\mathbf F$ is the force we are interested in, and $\text d\mathbf x$ is the very small displacement for one of the "little works" we are adding up.
Best Answer
You must understand that work is firstly path dependent. Work = $\int F . dl $, this is a path integral. If displacement is 0, the work done need not necessarily be 0 if the direction of force is changing throughout the path. A simple example would be moving a block from A to B and back to A under the influence of friction. Even though the displacement is 0 as the direction of kinetic friction changes when you change the direction of motion and hence friction does indeed do work. Testimony to this is the heat dissipated during the process.
It is better to think of total work done as a sum of infinitesimal works $dW$. Hope this makes sense!