In calculating the electron dispersion you probably obtained the diagonalized Hamiltonian in the momentum space
$$
H=\sum_\mathbf{k}\left[c^{\dagger}_{\mathbf{k}A},c^{\dagger}_{\mathbf{k}B}\right]\left[\begin{array}{cc}0 & \Delta(\mathbf{k})\\ \Delta^{\dagger}(\mathbf{k}) &0\end{array}\right]\left[\begin{array}{c}c_{\mathbf{k}A} \\ c_{\mathbf{k}B}\end{array}\right].
$$
If you you chose your $x$ axis along the zigzag direction (arXiv:1004.3396), the two nonequivalent Dirac valleys are $\mathbf{K}_\kappa=\left(\kappa\frac{4\pi}{3\sqrt{3}a},0\right)$, $\kappa=\pm1$ and $\mathbf{K}_{-1}=\mathbf{K}^{\prime}$, where $a$ is the C-C distance. Then $\Delta(\mathbf{k})=-t\left(1+e^{-i\mathbf{k}\cdot\mathbf{a}_1}+e^{-i\mathbf{k}\cdot\mathbf{a}_2}\right)$, where $t$ is the hopping term, and $\mathbf{a}_1=\left(\sqrt{3}a/2,3a/2\right)$ and $\mathbf{a}_2=\left(-\sqrt{3}a/2,3a/2\right)$ are the lattice vectors.
Taylor expanding $\Delta(\mathbf{k})$ up to linear terms around those two points you obtain
$$
\Delta(\mathbf{k})=\kappa\frac{3ta}{2}q_x-i\frac{3ta}{2}q_y
$$
where $\mathbf{q}$ is the displacement momenta from the $\mathbf{K}_\kappa$ point. Promoting these displacement momenta to operators you obtain the Hamiltonian
$$
H=\hbar v_F\left[\begin{array}{cccc}0 & q_x-iq_y & 0 & 0\\q_x+iq_y & 0 & 0 & 0\\0 & 0 & 0 & -q_x-iq_y\\0 & 0 & -q_x+iq_y & 0\end{array}\right]
$$
where $v_F=\frac{3ta}{2\hbar}$ is the Fermi velocity. This is in $\left[\Psi_{A\mathbf{K}},\Psi_{B\mathbf{K}},\Psi_{A\mathbf{K}^{\prime}},\Psi_{B\mathbf{K}^{\prime}}\right]^T$ basis, if you rearrange your basis as $\left[\Psi_{A\mathbf{K}},\Psi_{B\mathbf{K}},\Psi_{B\mathbf{K}^{\prime}},\Psi_{A\mathbf{K}^{\prime}}\right]^T$ you get the compact form
$$
H=\hbar v_F\tau_z\otimes\boldsymbol{\sigma}\cdot\mathbf{k}
$$
where $\tau_z$ acts in the valley space. This is similar to the Dirac-Weyl equation for relativistic massless particles, where instead of $v_F$ you get the speed of light
$$
H=\pm\hbar c\boldsymbol{\sigma}\cdot\mathbf{k}
$$
where $+$ denotes right-handed antineutrions, and $-$ denotes left-handed neutrions. The differences are that $\boldsymbol{\sigma}=\left(\sigma_x,\sigma_y\right)$ for graphene acts in pseudospin space and $\boldsymbol{\sigma}=\left(\sigma_x,\sigma_y,\sigma_z\right)$ for neutrinos acts in real spin space.
The precise statement should be:
The third-component of the Berry curvature $\Omega_3=\boldsymbol{\Omega}\cdot\boldsymbol{e}_3$ vanishes everywhere except at the Dirac points where it is not well-defined (diverging).
Why do we need to care about the third-component of the Berry curvature? Because it is the only component that contributes to the Chern number $C$ of a 2D band structure
$$C=\frac{1}{2\pi}\int_\text{BZ}\mathrm{d}^2\boldsymbol{k}\; \Omega_3(\boldsymbol{k}).$$
Therefore $\Omega_3$ is also named as the Berry flux density or the Chern density. The Berry curvature near the Dirac point is indeed given by $\boldsymbol{\Omega}=\boldsymbol{k}/2k^3$, which is non-vanishing. But since the momentum $\boldsymbol{k}=(k_x,k_y,0)$ lies in the $xy$-plane and has no third component, so the Berry flux density $\Omega_3$ vanishes everywhere except at the origin.
To see what happens at the origin, we need to regularize the problem with a small mass. Consider
$$H=k_x\sigma^x+k_y\sigma^y+m\sigma^z.$$
One finds
$$\Omega_3(\boldsymbol{k})=\frac{m}{2(\boldsymbol{k}^2+m^2)^{3/2}}.$$
As $m\to0$, one can see that $\Omega_3\sim m/k^3\to0$ vanishes everywhere (as long as $k\neq0$). But at the Dirac point where $k=0$, $\Omega_3\sim \pm1/m^2\to\pm\infty$ diverges to either $+\infty$ or $-\infty$ depending on the sign of the mass $m$. In this case, since the band gap vanishes, the Chern number is not well defined, so usually, it is also not meaningful to talk about the Berry flux density at the Dirac point.
Best Answer
Your use of the no-crossing idea is correct - we do not expect level crossings in two dimension to appear unless protected by symmetry. The symmetries in this case are the symmetries of the honeycomb lattice and time reversal. The protection of level crossings by symmetry is ubiquitous in solid-state.
I should add that the existence of these Dirac point is actually slightly more robust than would be implied by simple symmetry arguments. The band structure will still have Dirac cones if one applies any perturbation that does not violate parity, time reversal and is not extremely strong[1]. This is because of the interplay of the Berry's curvature and the Dirac point, which I could find a reference for if you would like.
[1] Extremely strong means that if I imagined increasing the strength of this perturbation up from zero it would drag the Dirac cones from the $K$, $K'$ points into each other. This would mean a perturbation of energy about the bandwidth, which is several electron-volts.