First, here is a relatively explicit homeomorphism from $(I^n, I^{n-1}\times \{0\})$ and $I^n, I^{n-1}\times \{0\} \cup \partial I^{n-1} \times I).$ Actually, I'm going to use $I = [-1,1]$ to make formulas a little bit nicer.
We will view $I^n$ as a union of concentric copies of $\partial I^n$ with a single point at the center. Concretely, for each $t\in [0,1]$, let $I_t:=\{(x_1,...,x_n)\in I^n: |x_i|\leq t$ for every $i$ and $|x_i| = t$ for at least one $i\}$. So, $I_1 = \partial I^n$ and $I_0$ is a single point.
We will define a homeomorphism $f$ of $I_1$ which maps $I^{n-1}\times \{-1\}$ to $I^{n-1}\times \{0\}\cup \partial I^{n-1}\times I$. Then we'll just copy this homeomorphism on each $I_t$.
To begin with, set $p:= (0,...,0,-1)\in I^{n-1}\times \{-1\}$. We set $f(p) = p$.
For every other point $x\in I^{n-1}\times \{-1\}$, there is a unique ray emanating from $p$ to $x$. We define $g(x)$ to be the point where this ray intersects $[-1/2,1/2]^{n-1}\times \{-1\}$ and we let $h(x)$ be where the ray intersects $\partial I^{n-1}\times \{-1\}$.
For $x\in [-1/2,1/2]^{n-1}\times \{-1\}$, we set $f(x) = \left(\frac{d(x,p)}{d(g(x), p)} h(x), -1\right)$, where $d$ is the usual Euclidean distance function. Intuitively, we are radially scaling the smaller cube $[-1/2,1/2]^{n-1}$ to fill the larger cute $I^{n-1}$.
For $x\in I^{n-1}\times \{-1\}$ but outside of $[-1/2,1/2]^{n-1}\times \{-1\}$, we define $f(x) = \left(h(x), \frac{d(x,g(x))}{d(x,h(x))}\right)$. This part surjects onto $\partial I^{n-1} \times I$.
All of this is just the definition of $f$ on the bottom face. So, far, we have a homeomorphism $f:I^{n-1}\times \{-1\}\rightarrow I^{n-1}\times {-1}\cup \partial I^{n-1}\times I$. We want to extend $f$ to $\partial I^{n-1}\times I \cup I^{n-1}\times \{1\}$. However, this domain is obviously homeomorphic to the range of $f|_{I^{n-1}\times \{-1\}}$, we can just use $f^{-1}$ (slightly modified) to extend $f$ to the rest of $I_1$. A little thought will show that this glues together where the two domains overlap.
$ \ $
How does this help with the connecting homomorphism?
Follow the proof as written until you get to the last step. From the above terrible formulas, $I^{n-1}\times \{1\}$ is homeomorphic to $I^{n-1}\times \{1\}\cup \partial I^{n-1}\times I$. Call such a homeomorphism $f$. Then, instead of declaring $\partial [\alpha] = \tilde{\alpha}(x_1,...,x_{n-1}, 1)$, define $\partial[\alpha] = \tilde{\alpha}(f(x_1,...,x_{n-1},1))$. The point is that $f$ maps the boundary $\partial I^{n-1}\times \{1\}$ onto $\partial I^{n-1}\times \{0\}$, and $\tilde{\alpha} $ has the value $\ast_E$ on that face.
I think all actions of $\pi_1(A,x_0)$ are the relative ones.
In particular, I think Hatcher sees the long exact sequence of $(X,A,x_0)$ as follows :
$\cdots\to\pi_n(A,x_0,x_0)\to\pi_n(X,x_0,x_0)\to\pi_n(X,A,x_0)\to\pi_{n-1}(A,x_0,x_0)\to\cdots$
This seems to be consistent with the proof of Theorem 4.3. Thus it makes sense to interpret all actions of $\pi_1(A,x_0)$ to any of the groups above by the right figure in your post.
The commutativity of the action of $\pi_1(A,x_0)$ should be clear if we stick with the relative version.
Best Answer
Your definition of $J^{n-1}$ seems to be this: Set $\bar I^{n-1} = \{t_1,\ldots,t_n) \in I^n \mid t_n = 1 \}$ and $J^{n-1} = \text{cl}(I^n \setminus \bar I^{n-1})$. This shows that $J^0 = \{0\}$ since $\bar I^{0} = \{1\}$. So your interpretation is correct. Note that some authors define $\bar I^{n-1} = \{t_1,\ldots,t_n) \in I^n \mid t_n = 0 \}$ which produces not the same $J^{n-1}$ as above. But that is irrelevant for the exact sequence.
The last three terms of your sequence are no groups, but only pointed sets. The concept of exactness for sequences of pointed sets is nevertheless introduced by the usual "$\operatorname{ker}-\operatorname{im}$"-definiton. To verify $\operatorname{ker} \partial \subset \operatorname{im} j_*$, consider $[f] \in \operatorname{ker} \partial$, where $f : (I,\{0,1\}, \{0\}) \to (X,A, x_0)$. This means that $[f \circ \iota]$ is the trivial element in $\pi_0(A)$, where $\iota : \{1\} \hookrightarrow I$. In other words, $f(1)$ lies in the path component of $x_0$ in $A$. Hence $f$ is homotopic via a homotopy of triples to a map $f' : (I,\{0,1\}, \{0\}) \to (X,A, x_0)$ such that $f'(1) = x_0$. Therefore $f' : (I,\{0,1\}) \to (X,x_0)$ and by definition $j_*([f']) = [f'] = [f]$.
$\pi_0(Y,y_0)$ is the set of homotopy classes of maps $g : \{1\} \to Y$ with basepoint being the homotopy classe of $c_{y_0} : \{1\} \to Y, 1 \mapsto y_0$. It can be naturally identified with the set of path compenents of $Y$.
$\operatorname{ker} i_*$ is the set of all $[g] \in \pi_0(A,x_0)$ such that $[i \circ g]$ represents the basepoint in $\pi_0(X,x_0)$. This means that $g(1)$ is contained in the path component of $x_0$ in $X$.
Let $[f] \in \pi_1(X,A,x_0)$, where $f : (I,\{0,1\}, \{0\}) \to (X,A, x_0)$. Then $i_*(\partial ([f])) = [i \circ f \circ \iota]$. But clearly $i \circ f \circ \iota(1)$ lies in the path component of $x_0$ in $X$. Thus $i_* \circ \partial$ maps $\pi_1(X,A,x_0)$ to the basepoint, i.e. we have $\operatorname{im} \partial \subset \operatorname{ker} i_*$. Next consider $[g] \in \operatorname{ker} i_*$, where $g : \{1\} \to A$ . This means that $g(1)$ lies in the path component of $x_0$ in $X$. Take any path $f : I \to X$ such that $f(0) = x_0$ and $f(1) = g(1)$. Clearly $f$ is a map of triples $(I,\{0,1\},\{0\}) \to (X,A,x_0)$ and by construction $f \circ \iota = g$. Thus $[g] \in \operatorname{im} \partial$-