All the answers point to yes, but you need to be careful as what you mean by inverse (of course, mathematics always requires thinking). I will try not to get into set-theoretic issues and appeal to your intuition.
If $f : X \to Y$ is a map of sets which is injective, then for each $x \in X$, we have an element $y = f(x)$ uniquely determined by $x$, so we can define $g : Y \to X$ by sending those $y \in f(X)$ to that element $x$ for which $f(x) = y$, and the fact that $f$ is injective will show that $g$ will be well-defined ; for those $y \in Y \backslash f(X)$, just send them wherever you want (this would require this axiom of choice, but let's not worry about that). The function $g$ satisfies $g(f(x)) = g(y) = x$, so that $g \circ f$ is the identity map ; that is, $f$ admits a left inverse.
Conversely, suppose $f$ admits a left inverse $g$, and assume $f(x_1) = f(x_2)$. Then $x_1 = g(f(x_1)) = g(f(x_2)) = x_2$, so $f$ is injective. That is,
A function $f : X \to Y$ is injective if and only if it admits a left-inverse $g : Y \to X$ such that $g \circ f = \mathrm{id}_X$.
Similarly, it is not hard to show that $f$ is surjective if and only if it has a right inverse, that is, a function $g : Y \to X$ with $f \circ g = \mathrm{id}_Y$. I'll let you ponder on this one.
If you're looking for a little more fun, feel free to look at this ; it is a bit harder though, but again if you don't worry about the foundations of set theory you can still get some good intuition out of it.
Hope that helps,
Best Answer
$f$ need not to be surjective because $g\circ f$ is defined even if the image of $f$ is not the whole domain of $g$.
Example. Let $A\subset B$ non empty, and $f:A\to B$ the inclusion, that is to say $f(x)=x$. Define the map $g:B\to A$ as follows. Let $a_0\in A$ any point (which exists because $A\neq\emptyset$) then set:
$$g(x)=x\qquad\text{ for } x\in A$$ $$g(x)=a_0\qquad \text{ for } x\notin A$$
$g$ is a left inverse of $f$.