I believe that the sum of the lengths of diagonals of a regular polygon ($n$-gon) is always greater than or equal to any other irregular polygon ($n$-gon) inscribed in a circle.
For example for a 4-gon, if we consider a regular $4$-gon, then the two diagonals pass through the centre of a circle, resulting in the sum of the diagonals' lengths twice the diameter. Considering any other $4$-gon (irregular), the sum of the lengths of the diagonals is not greater than $2\times$ the diameter.
Case : 5-gon
I proved that the sum of the lengths of diagonals in a regular $5$-gon is maximum compared to any $5$-gon (irregular polygon). I used Lagrange's multiplier to prove it.
Case : $n$-gon
I tried proving the same for an $n$-gon using Lagrange's multiplier, where we need to maximize the sum of the lengths of all diagonals with respect to $$\theta_1+ \theta_2 + \dots + \theta_n =2\pi,$$ but unable to do the calculation as the number of variables increases. Here $\theta_i$ is the angle subtended to the centre of a circle by the side $a_i$ of the polygon.
I believe that the sum of the lengths of diagonals is maximum if $$\theta_1 = \theta_2 = \dots = \theta_n,$$ which is possible only in a regular polygon.
Edit: same question posted in mathoverflow, https://mathoverflow.net/questions/398248/sum-of-the-lengths-of-all-diagonals-in-a-regular-polygon
Best Answer
For simplicity, let us consider the index $i$ of $P_i$ as the member of $\mathbb{Z}_n$.
We aim to show a stronger criterion : given points $P_1,P_2,...,P_m$ on the unit circle $\Omega$ lying in the counter-clockwise order, for each $k<m$ such that $gcd(k,m)=1$, we have $$\sum_{j=1}^m \overline{P_j P_{j+k}}\le \sum_{j=1}^m\overline{Q_j Q_{j+k}}$$ , where $Q_1,...,Q_m$ are vertices of a regular $(m)$-gon inscribed in $\Omega$.
Assuming this, we can easily show the original conjecture observing that the collection of all diagonal lines of the convex $n$-gon $P_1 P_2...P_n$ is partitioned into the cycles of the form above.
(Explicitly speaking, we can write
$$\sum_{|i-j|\ge 2}\overline{P_iP_j}=\frac{1}{2}\sum_{k=2}^{n-1}\sum_{j=1}^{gcd(k,n)}\left(\sum_{i=1}^{n/gcd(k,n)}\overline{P_{ki+j}P_{k(i+1)+j}}\right)$$ You can check this holds since each segment of LHS is counted at least twice in RHS, and the total count of segments of RHS is $(n-1)n$, which is exactly twice of the total number of terms in LHS. )
For better intuition, I post an example of $n=8$.
The collection of the diagonal segments of the octagon above is the disjoint union of the red, green, blue-colored chains plus the four main diagonals. Assuming our criterion is correct, we can argue that the total length of the segments of each color (including the main diagonals) is maximized when the polygon is regular, which concludes our proof.
Now, to show the criterion, we first exclude the trivial case $m=2$, and allow some of $P_j$'s to be the same. (but not in a switched order)
Writing $P_j=e^{i\theta_j}$, one can easily observe that the set of such $(\theta_1,...,\theta_m)\in \mathbb{T}^m$ is compact.
(Here, $\mathbb{T}^m$ is the $m$-dimensional torus $\mathbb{R}^m/\mathbb{Z}^m$ with the natural metric)
Thus, there exists a pair $(\theta_1,...,\theta_m)$ that maximizes the sum.
Now, observe that for each $j$, $P_j$ contributes only to two distinct segments : $\overline{P_jP_{j+k}}$ and $\overline{P_jP_{j-k}}$.
By the sine addition formula $\sin \alpha+\sin \beta=2\sin(\frac{\alpha+\beta}{2})\cos(\frac{\alpha-\beta}{2})$, this can be maximized only if $\overline{P_jP_{j+k}}=\overline{P_jP_{j-k}}$.
It follows that the lengths $\overline{P_jP_{j+k}}$ are all equal, which requires $P_1...P_m$ to be vertices of a regular $(m)$-gon, ending the proof of the desired criterion.
P.S. This is a formalization of the great idea suggested by Toni Mhax.