We suppose that $X$ is a locally convex vector space. Before showing that the conjecture is true (under some conditions), we show a helpful lemma.
Lemma. If a subset $C$ of $X$ has the property that it's boundary can be covered by an open cover $\{U_\alpha\}_{\alpha\in A}$ with the property that each $U_\alpha\cap C$ is convex, then the triangle
$$
\Delta(x,y,z):=\{(1-t)((1-s)x+sy)+tz:s,t\in[0,1)\}
$$
is contained in $C$ whenever the lines $\ell_{x,y}:=\{(1-s)x+sy:s\in[0,1]\}$ and $\ell_{x,z}:=\{(1-t)x+tz:t\in[0,1]\}$ are contained in $C$.
Proof. Suppose that $\Delta(x,y,z)$ is not contained in $C$. In the following, the boundary and interior are considered relative to the triangle $\Delta(x,y,z)$.
Among $s\in[0,1)$ with the property that the line
$$
\{(1-t)((1-s)x+sy)+tz:t\in[0,1)\}\cap \partial(C\cap\Delta(x,y,z))\neq\varnothing
$$
there is a minimum $s_0$. Then there is also a minimum $t_0$ among the $t\in[0,1)$ with the property that
$$
u:=(1-t)((1-s_0)x+s_0y)+tz:t\in\partial(C\cap\Delta(x,y,z))
$$
Since $u$ is on the boundary of $C$, there exists an $\alpha\in A$ with the property that $u\in U_\alpha$. Then $U_\alpha\cap C\cap\Delta(x,y,z)$ is convex. But note that the set
$$
\ell_{x,y}\cup\ell_{x,z}\cup\Delta(x,(1-s_0)x+s_0y,z),
$$
where the latter triangle has a similar definition as $\Delta(x,y,z)$, is contained in $C$. The convexity of $U_\alpha\cap C\cap\Delta(x,y,z)$ then implies that there is a $\varepsilon>0$ such that the point
$$
(1-(t_0-\varepsilon))((1-s_0)x+s_0y)+(t_0-\varepsilon)z
$$
is on the boundary of $C\cap\Delta(x,y,z)$ unless $t_0=0$. The minimality of $t_0$ then gives us that $t_0=0$. A similar argument shows that $s_0=0$ and it follows once again from the convexity of the boundary of $C$ that $x,y,z$ are on a line. But then $\Delta(x,y,z)$ is (contained in) this line. QED.
We have the following corollary (in the open case, an easy adjustment of the above proof has to be given).
Corollary. If $X$ is either open or closed, the triangle
$$
\overline{\Delta(x,y,z)}=\{(1-t)((1-s)x+sy)+tz:s,t\in[0,1]\}
$$
is contained in $C$.
Now we are ready to answer positive to the conjecture in the case that $C$ is open or closed.
Proposition. Suppose that $X$ is a locally convex vector space and suppose that $C$ is a closed [open] subset of $X$. If there exists a cover $\{U_\alpha\}_{\alpha\in A}$ of $\partial C$ with the property that $U_\alpha\cap C$ is convex for each $\alpha\in A$, then $C$ is convex.
Proof.
Suppose $C$ is a subset of a locally convex vector space $X$ and that $\{U_\alpha\}_{\alpha\in A}$ is a cover of the boundary of $C$ with the property that $U_\alpha\cap C$ is convex for each $\alpha\in A$.
First, let $B$ be a maximal among the subsets of $A$ with the property that the convex hull of $\bigcup_{\alpha\in B}U_\alpha$ is contained in $C$. For example, such a maximal $B$ can be found with Zorn's lemma, using the poset
$$
P:=\big\{B\subseteq A:\mathrm{Hull}\big(\textstyle\bigcup_{\alpha\in B}U_\alpha\cap C\big)\subseteq C\big\}
$$
ordered with inclusion. Here, $\mathrm{Hull}$ denotes the operation of taking the convex hull.
Now let $C^\prime$ be a maximal convex subset of $C$ which contains $U_\alpha$ for each $\alpha\in B$. Here's an outline of the rest of the proof.
- First we will show that if for $\alpha\in A$ there is a $\beta\in B$ such that $U_\alpha\cap U_\beta\cap C\neq\varnothing$, then $\alpha$ is in $B$.
- Then we will show that no boundary points of $C^\prime$ are interior points of $C$. (Here we will use the assumption that $X$ is locally convex.)
With these two observations the proof is easily finished. Since boundary points of $C^\prime$ are boundary points of $C$, it follows that $C^\prime$ is closed in $C$. The set
$$
U:=\textstyle\bigcup_{\alpha\in B} U_\alpha\cup\mathrm{int}\,C^\prime
$$
is an open set with the property that $x\in C^\prime$ whenever $x\in U\cap C$, therefore $C^\prime$ is also open in $C$. The connectedness of $C$ together with the non-emptiness of $C^\prime$ (whenever $C$ is non-empty) now implies that $C^\prime= C$.
Proof of 1. Suppose that $x\in U_\alpha\cap U_\beta\cap C$. Note that for any $y\in U_\alpha$, for any $\gamma\in B$ and for any $z\in U_\gamma$, the line
$$
\ell_{y,z}:=\{(1-t)y+tz:t\in[0,1]\}
$$
is contained in $C$, which is a consequence of the corollary because the we have $\ell_{x,y}\subseteq U_\alpha\cap C\subseteq C$ and $\ell_{x,z}\subseteq C^\prime\subseteq C$. By the maximality of $B$, it follows that $\alpha\in B$.
Proof of 2. Suppose that $x^\prime$ is in $\partial C^\prime\cap\mathrm{int}\,C$. Let $V$ be a convex open neighborhood of $x^\prime$ which is contained in $C$, let $y$ be a point of $V\setminus \bar{C^\prime}$ and let $x$ be a point of $V\cap C^\prime$. It follows by the corollary that for every $z\in C^\prime$ the line $\ell_{y,z}$ is contained in $C$, since the lines $\ell_{x,y}$ and $\ell_{x,z}$ are. Therefore, $C^\prime$ can be extended to include $y$, which contradicts the maximality of $C^\prime$. Thus we may conclude that no boundary points of $C^\prime$ are interior points of $C$. QED.
It turns out that twice-differentiability implies that the Hessian is symmetric even without convexity and with no reference to whether the second-order partial derivatives are continuous! The proof below is based on Theorem 8.12.2 in the book Foundations of Modern Analysis by Dieudonné (1969, p. 180).
Claim: Let $U\subseteq\mathbb R^n$ be an open set and $f:U\to\mathbb R$ a function. Suppose that $f$ is (Fréchet) differentiable on $U$ and that it is twice (Fréchet) differentiable at $\mathbf x_0\in U$. Then, the Hessian matrix $\mathbf H(\mathbf x_0)$ at $\mathbf x_0$ is symmetric.
Proof: Let $\mathbf D:U\to\mathbb R^n$ denote the gradient function of $f$. Fix $\varepsilon>0$. Since $\mathbf D$ is Fréchet differentiable at $\mathbf x_0$ by assumption, it follows that there exists some $\delta>0$ such that $\|\mathbf v\|<4\delta$ implies that
$$\left\|\mathbf D(\mathbf x_0+\mathbf v)-\mathbf D(\mathbf x_0)-\mathbf H(\mathbf x_0)\cdot\mathbf v\right\|\leq\varepsilon\|\mathbf v\|.$$
There is no loss of generality in taking $\delta$ to be so small that the open ball $B(4\delta,\mathbf x_0)$ is contained in the open set $U$.
For any $i,j\in\{1,\ldots,n\}$, let $\mathbf e_i$ and $\mathbf e_j$ be the corresponding standard basis vectors of unit length. Let $\mathbf s\equiv\delta\mathbf e_i$ and $\mathbf t\equiv\delta\mathbf e_j$. It is clear that $\mathbf x_0+\xi\mathbf s+\mathbf t$ and $\mathbf x_0+\xi\mathbf s$ are both in $U$ whenever $\xi\in[0,1]$; this is because $\|\xi\mathbf s+\mathbf t\|<4\delta$ and $\|\xi\mathbf s\|<4\delta$. Define the following function $g:[0,1]\to\mathbb R$:
$$g(\xi)\equiv f(\mathbf x_0+\xi\mathbf s+\mathbf t)-f(\mathbf x_0+\xi\mathbf s)\quad\forall\xi\in[0,1].$$
Clearly, $g$ is continuous on $[0,1]$ and differentiable on $(0,1)$. Lagrange's mean-value theorem, in turn, implies that there exists some $\xi\in(0,1)$ such that
$$g(1)-g(0)=g'(\xi)=\mathbf s\cdot\left[\mathbf D(\mathbf x_0+\xi\mathbf s+\mathbf t)-\mathbf D(\mathbf x_0+\xi\mathbf s)\right],$$
using the chain rule.
Next, one can derive the following chain of inequalities (the first one uses the Cauchy–Schwarz inequality):
\begin{align*}
&\left|g(1)-g(0)-\mathbf s\cdot\mathbf H(\mathbf x_0)\cdot\mathbf t\right|\leq\underbrace{\|\mathbf s\|}_{=\delta}\left\|[\mathbf D(\mathbf x_0+\xi\mathbf s+\mathbf t)-\mathbf D(\mathbf x_0)]-[\mathbf D(\mathbf x_0+\xi\mathbf s)-\mathbf D(\mathbf x_0)]-\mathbf H(\mathbf x_0)\cdot\mathbf t\right\|\\
=&\,\delta\left\|[\mathbf D(\mathbf x_0+\xi\mathbf s+\mathbf t)-\mathbf D(\mathbf x_0)-\mathbf H(\mathbf x_0)\cdot(\xi\mathbf s+\mathbf t)]-[\mathbf D(\mathbf x_0+\xi\mathbf s)-\mathbf D(\mathbf x_0)-\mathbf H(\mathbf x_0)\cdot(\xi\mathbf s)]\right\|\\
\leq&\,\delta\varepsilon\left(\|\xi\mathbf s+\mathbf t\|+\|\xi\mathbf s\|\right)<8\delta^2\varepsilon.
\end{align*}
That is, one has that
$$|f(\mathbf x_0+\mathbf s+\mathbf t)-f(\mathbf x_0+\mathbf s)-f(\mathbf x_0+\mathbf t)+f(\mathbf x_0)-\delta^2\mathbf e_i\cdot\mathbf H(\mathbf x_0)\cdot\mathbf e_j|<8\delta^2\varepsilon,$$
and, by a completely analogous and symmetric reasoning in which $\mathbf s$ and $\mathbf t$ are interchanged,
$$|f(\mathbf x_0+\mathbf s+\mathbf t)-f(\mathbf x_0+\mathbf s)-f(\mathbf x_0+\mathbf t)+f(\mathbf x_0)-\delta^2\mathbf e_j\cdot\mathbf H(\mathbf x_0)\cdot\mathbf e_i|<8\delta^2\varepsilon.$$
Given that $\mathbf e_i\cdot\mathbf H(\mathbf x_0)\cdot\mathbf e_j=h_{ij}(\mathbf x_0)\equiv\partial^2 f/(\partial x_i\partial x_j)(\mathbf x_0)$, the preceding two inequalities imply that
$$\left|h_{ij}(\mathbf x_0)-h_{ji}(\mathbf x_0)\right|<16\varepsilon.$$
Taking $\varepsilon$ to be arbitrarily small, one sees that $h_{ij}(\mathbf x_0)=h_{ji}(\mathbf x_0)$. $\blacksquare$
Best Answer
Here is proof if $f$ is assumed $C^2$. This is not necessary but simplifies the proof significantly.
It is sufficient to show the $f''(x) \ge 0$.
Pick some $x$, then there is a neighbourhood $U$ of $x$ such that $f(x+h) -f(x) \ge f'(x) h$ for $x+h \in U$.
Since $f$ is $C^2$, Taylor gives (for $h$ sufficiently small) that $f(x+h) = f(x) + f'(x)h + {1 \over 2} h^T f''(\xi_h)h$, where $\xi_h \in [x,x+h]$. This gives $h^T f''(\xi_h)h \ge 0$ for $h$ such that $x+h \in U$. If $h \neq 0$ then ${h^T \over \|h\|} f''(\xi_h) {h \over \|h\|} \ge 0$, of course.
Pick some unit vector $v$, and let $h = t v$ for small $t$, then we have $ v^T f''(\xi_{tv}) v \ge 0$, and letting $t \to 0$ and using continuity of $f''$ we get $ v^T f''(x) v \ge 0$.