First, here is a relatively explicit homeomorphism from $(I^n, I^{n-1}\times \{0\})$ and $I^n, I^{n-1}\times \{0\} \cup \partial I^{n-1} \times I).$ Actually, I'm going to use $I = [-1,1]$ to make formulas a little bit nicer.
We will view $I^n$ as a union of concentric copies of $\partial I^n$ with a single point at the center. Concretely, for each $t\in [0,1]$, let $I_t:=\{(x_1,...,x_n)\in I^n: |x_i|\leq t$ for every $i$ and $|x_i| = t$ for at least one $i\}$. So, $I_1 = \partial I^n$ and $I_0$ is a single point.
We will define a homeomorphism $f$ of $I_1$ which maps $I^{n-1}\times \{-1\}$ to $I^{n-1}\times \{0\}\cup \partial I^{n-1}\times I$. Then we'll just copy this homeomorphism on each $I_t$.
To begin with, set $p:= (0,...,0,-1)\in I^{n-1}\times \{-1\}$. We set $f(p) = p$.
For every other point $x\in I^{n-1}\times \{-1\}$, there is a unique ray emanating from $p$ to $x$. We define $g(x)$ to be the point where this ray intersects $[-1/2,1/2]^{n-1}\times \{-1\}$ and we let $h(x)$ be where the ray intersects $\partial I^{n-1}\times \{-1\}$.
For $x\in [-1/2,1/2]^{n-1}\times \{-1\}$, we set $f(x) = \left(\frac{d(x,p)}{d(g(x), p)} h(x), -1\right)$, where $d$ is the usual Euclidean distance function. Intuitively, we are radially scaling the smaller cube $[-1/2,1/2]^{n-1}$ to fill the larger cute $I^{n-1}$.
For $x\in I^{n-1}\times \{-1\}$ but outside of $[-1/2,1/2]^{n-1}\times \{-1\}$, we define $f(x) = \left(h(x), \frac{d(x,g(x))}{d(x,h(x))}\right)$. This part surjects onto $\partial I^{n-1} \times I$.
All of this is just the definition of $f$ on the bottom face. So, far, we have a homeomorphism $f:I^{n-1}\times \{-1\}\rightarrow I^{n-1}\times {-1}\cup \partial I^{n-1}\times I$. We want to extend $f$ to $\partial I^{n-1}\times I \cup I^{n-1}\times \{1\}$. However, this domain is obviously homeomorphic to the range of $f|_{I^{n-1}\times \{-1\}}$, we can just use $f^{-1}$ (slightly modified) to extend $f$ to the rest of $I_1$. A little thought will show that this glues together where the two domains overlap.
$ \ $
How does this help with the connecting homomorphism?
Follow the proof as written until you get to the last step. From the above terrible formulas, $I^{n-1}\times \{1\}$ is homeomorphic to $I^{n-1}\times \{1\}\cup \partial I^{n-1}\times I$. Call such a homeomorphism $f$. Then, instead of declaring $\partial [\alpha] = \tilde{\alpha}(x_1,...,x_{n-1}, 1)$, define $\partial[\alpha] = \tilde{\alpha}(f(x_1,...,x_{n-1},1))$. The point is that $f$ maps the boundary $\partial I^{n-1}\times \{1\}$ onto $\partial I^{n-1}\times \{0\}$, and $\tilde{\alpha} $ has the value $\ast_E$ on that face.
Best Answer
Since $p:E\to B$ is a weak fibration, it has homotopy lifting property with respect to all CW complexes. This is not immediately applicable as we are working with homotopy relative to subspace which requires additional constraints, but we can use the following result derived from HLP:
Continue from your argument. We are given $\alpha:I^n\to E$ with $\alpha(\partial I^n)=\{s(b_0)\}$ and a homotopy $H:I^n\times I\to B$ relative to $\partial I^n$ between $p\circ \alpha$ and $c_{b_0}$. Restricting to $\partial I^n\times I$, $H$ can be lifted to $\tilde h=c_{s(b_0)}$. By relative homotopy lifting property, we can find a homotopy $\tilde H:I^n\times I\to E$ such that $p\circ \tilde H=H$ and $\tilde H|_{I^n\times\{0\}}=\alpha$, and $\tilde{H}|_{\partial I^n\times I}=c_{s(b_0)}$, so $\tilde H$ gives a homotopy between $\alpha$ and $\gamma=\tilde H|_{I^n\times\{1\}}$ relative to $\partial I^n$.
Now, we have $p\circ \gamma=p\circ \tilde H|_{I^n\times\{1\}}=c_{b_0}$, meaning that $\gamma(I^n)\subseteq p^{-1}(b_0)=F$. Therefore $\gamma$ factors through $F=p^{-1}(b_0)$, that is, we can find a map $\beta:(I^n,\partial I^n)\to (F,s(b_0))$ such that $\gamma=i\circ \beta$, where $i:(F,s(b_0))\hookrightarrow (E,s(b_0))$ is the inclusion of fiber. This shows that $i_\ast([\beta])=[\gamma]=[\alpha]$, since $\alpha\simeq\gamma$ rel $\partial I^n$.
For a detailed proof of relative homotopy lifting property with respect to $(I^n,\partial I^n)$, see here. The proof uses a special homeomorphism of pairs $h:(I^n\times I, I^n\times\{0\}\cup\partial I^n\times I)\to (I^n\times I,I^n\times \{0\})$. For more information of this map, see here and here.