This question arose from the recent one, roots of a polynomial linked to mock theta function?. Let
$$
g(x):=\sum_{k=0}^\infty x^k\prod_{j=1}^{k-1}(1 + x^j)^2\\=1+x+x^2+3 x^3+4 x^4+6 x^5+10 x^6+15 x^7+21 x^8+30 x^9+43 x^{10}+59 x^{11}+…;
$$
the sequence $1,1,1,3,4,6,10,15,21,30,43,59,…$ with the generating function $g(x)$ is A059618 on OEIS, it is the sequence of numbers of strongly unimodal partitions.
Now let
$$
f(q):=g(q)\prod_{n=1}^\infty(1-q^n),
$$
and let $a_k$ be the $k$th coefficient in the Maclaurin series for $f$,
$$
f(x)=\sum_{k=0}^\infty a_kx^k\\=1-x^2+x^3+x^6+x^7-x^9+x^{10}-x^{14}+x^{18}-x^{20}+x^{21}+x^{25}+x^{26}-x^{27}\\+x^{28}-x^{30}+x^{33}-x^{35}+x^{36}-x^{39}-x^{40}+x^{42}-x^{44}+2x^{45}-x^{49}+x^{52}-x^{54}\\+x^{55}+x^{56}+x^{57}-x^{60}-x^{65}+…
$$
The sequence of $a_k$, starting with
1,0,-1,1,0,0,1,1,0,-1,1,0,0,0,-1,0,0,0,1,0,-1,1,0,0,0,1,1,-1,1,0,-1,0,0,1,0,-1,1,0,0,-1,-1,0,1,0,-1,2,0,0,…
is not on OEIS. Among the first 1000 terms of the sequence, there are 609 zeroes, 182 ones, 161 -1s, 19 of them are 2 ($a_{45},a_{150},a_{210},a_{221},a_{273},a_{300},…$), 22 are -2 ($a_{77},a_{90},a_{165},a_{225},…$), and two of them ($a_{525}$ and $a_{825}$) are 3; seems like $a_k$ are zero for $k=2^j$ ($j>0$), for $k=p$ or $k=2p$, with $p$ prime $>7$, $k=3p$ and $k=4p$ with $p$ prime $\geqslant23$, $k=5p$ with $p$ prime $>31$, $6p$ for $p>37$, $7p$ and $8p$ for $p>43$, $9p$ for $p>47$, $10p$ for $p>61$, $11p$ for $p>67$,…
What may (or may not) be relevant is another sequence obtained from introducing new variable in the way I learned from a paper by Rhoades linked to from the above OEIS page for $g$.
Let
$$
g_t(q):=\sum_{k=0}^\infty q^k\prod_{j=1}^{k-1}(1 + q^jt)(1+q^j/t),
$$
and let
$$
f_t(q)=g_t(q)\prod_{n=1}^\infty(1-q^n),
$$
so that $g_1(q)=g(q)$ and $f_1(q)=f(q)$. Then
$$
f_t(q)=1-q^2+\frac{1+t^3}{(1+t)t}q^3+\frac{1+t^5}{(1+t)t^2}q^6+q^7-\frac{1+t^3}{(1+t)t}q^9+\frac{1+t^7}{(1+t)t^3}q^{10}+…;
$$
most coefficients have form $\pm\frac{1+t^{2j+1}}{(1+t)t^j}$, except that I cannot figure out how $j$ depends on the number of the coefficient. Exceptions here start from the $15$th coefficient, which is $\frac{1+t^9}{(1+t)t^4}-1$ and the $45$th one which is $\frac{1+t^{17}}{(1+t)t^8}+\frac{1+t^3}{(1+t)t}$.
Despite all these clues, to my shame I've given up searching for an explicit formula for $a_k$. Is there one? I am pretty sure there is, but what is it?
Best Answer
The conjectured identity $$ f(q)=(q;q)_\infty\left(1+\sum_{k=1}^\infty q^k(-q;q)^2_{k-1}\right)=\sum_{\substack{m,n\geqslant0\\n\ne1}}(-1)^mq^{\frac{(m+n)(3m+n+1)}2},\tag{1} $$ using Euler's pentagonal number theorem $(q;q)_\infty=\sum _{m=-\infty}^\infty (-1)^m q^{\frac{1}{2} m (3 m+1)}$ can be brought to an equivalent form $$ (q;q)_\infty\sum_{k=1}^\infty q^k(-q;q)^2_{k-1}=\sum_{\substack{m\ge0}}\sum_{\substack{n\ge 1}}(-1)^mq^{\frac{(m+n)(3m+n+1)}2}.\tag{1a} $$ By identity $(4.1)$ in Rhoades' paper the sum on the LHS of $(1a)$ is $$ \sum_{k=1}^\infty q^k(-q;q)^2_{k-1}=\frac{1}{2(q;q)_{\infty }} \sum _{n=-\infty}^\infty \frac{q^{\frac{1}{2} n (n+1)}}{q^n+1}-\frac{1}{4}\sum _{k=0}^\infty \frac{q^{k^2}}{(-q;q)_k^2},\tag{2} $$ while the double sum in $(1a)$ \begin{align} \sum_{\substack{m\ge0}}\sum_{\substack{n\ge 1}}(-1)^mq^{\frac{(m+n)(3m+n+1)}2}&=\sum _{m=0}^\infty \sum _{k=m+1}^\infty (-1)^mq^{\frac{k(k+1+2m)}2}\\ &=\sum _{k=1}^\infty \sum _{m=0}^{k-1} (-1)^m q^{m k+\frac{1}{2} (k+1) k}\\ &=\sum _{k=1}^\infty q^{\frac{k^2}{2}+\frac{k}{2}} \frac{1-(-1)^k q^{k^2}}{1+q^k}.\tag{3} \end{align} Since $\sum _{n=-\infty}^\infty \frac{q^{\frac{1}{2} n (n+1)}}{q^n+1}=\frac{1}{2}+2 \sum _{n=1}^\infty \frac{q^{\frac{1}{2} n (n+1)}}{q^n+1}$ $(2)$ and $(3)$ contain the same sum $\sum _{n=1}^\infty \frac{q^{\frac{1}{2} n (n+1)}}{q^n+1}$. This sum cancels out after $(2)$ and $(3)$ are substituted in $(1a)$ resulting in $$ \frac14-\frac{(q;q)_{\infty }}{4}\sum _{k=0}^\infty \frac{q^{k^2}}{(-q;q)_k^2}=-\sum _{n=1}^\infty \frac{(-1)^n q^{\frac{3 n^2}{2}+\frac{n}{2}}}{1+q^n},\tag{1c} $$ and equivalently $$ \sum _{k=0}^\infty \frac{q^{k^2}}{(-q;q)_k^2}=\frac{2}{(q;q)_{\infty }}\sum _{n=-\infty}^\infty \frac{(-1)^n q^{\frac{3 n^2}{2}+\frac{n}{2}}}{1+q^n}.\tag{1d} $$ $(1d)$ corresponds to the special case $x=-1$ of the identity $$ \sum _{k=0}^\infty \frac{q^{k^2}}{(x q;q)_k(q/x;q)_k}=\frac{1-x}{(q;q)_{\infty }}\sum _{n=-\infty}^\infty \frac{(-1)^n q^{\frac{3 n^2}{2}+\frac{n}{2}}}{1-xq^n},\tag{4} $$ which can be obtained from Watson-Whipple transformation formula (see the paper "Modular transformations of Ramanujan’s fifth and seventh order mock theta functions", Ramanujan J. 7 (2003), 193–222. by Gordon and McIntosh). $(4)$ also can be proved directly by partial fractions expansion and the following limiting case of q-Gauss summation $$ \sum _{k=n}^\infty\frac{q^{k^2}}{(q;q)_{k-n}(q^{n+1};q)_{k}}=\frac{q^{n^2}}{(q^{n+1};q)_n}\sum _{k=0}^\infty\frac{q^{k^2+2kn}}{(q;q)_{k}(q^{2n+1};q)_{k}}=\frac{q^{n^2}}{(q^{n+1};q)_\infty}. $$