Let $x$ be the length of small n-sided regular polygon in the star & $a$ be the distance between two adjacent vertices, then the angle of spike of regular star polygon is given as
$$\alpha=\frac{\pi}{\text{number of vertices (points) in the star}}=\frac{\pi}{n}$$
Now, draw a perpendicular from one vertex of star to the side of the small regular polygon to obtain a right triangle .
Using geometry of right triangle, the length of perpendicular drawn to the side of small regular polygon can be obtained
$$=\frac{a}{2}\csc\frac{\pi}{n}-\frac{x}{2}\cot\frac{\pi}{n}$$
Hence, in right triangle, one should have
$$\tan\frac{\pi}{2n}=\frac{\frac{x}{2}}{\frac{a}{2}\csc\frac{\pi}{n}-\frac{x}{2}\cot\frac{\pi}{n}}$$
$$x=\frac{a\tan\frac{\pi}{2n}\csc\frac{\pi}{n}}{1+\tan\frac{\pi}{2n}\cot\frac{\pi}{n}}$$
$$x=\frac{a\sin\frac{\pi}{2n}}{\sin\frac{\pi}{n}\cos\frac{\pi}{2n}+\cos\frac{\pi}{n}\sin\frac{\pi}{2n}}$$
$$x=\frac{a\sin\frac{\pi}{2n}}{\sin\left(\frac{\pi}{n}+\frac{\pi}{2n}\right)}$$
$$\bbox[5pt, border:2.5pt solid #FF0000]{\color{blue}{x=\frac{a\sin\frac{\pi}{2n}}{\sin\frac{3\pi}{2n}}}}$$
$$\forall \ \ n=2k+1\ \ (k\in N)$$
Hence for a regular pentagon in the star, setting $n=5$, the side of regular pentagon
$$x=\frac{a\sin\frac{\pi}{10}}{\sin\frac{3\pi}{10}}=a\frac{\sin18^\circ}{\cos36^\circ}=a\frac{\frac{\sqrt 5-1}{4}}{\frac{\sqrt 5+1}{4}}=\color{red}{\frac{a}{2}(3-\sqrt 5)}$$
Edited details: If the star regular polygon has $2n$ no. of vertices which is obtained by placing two congruent $n$-sided regular polygons one on the other in symmetrical staggered manner similar to a hexagram then a generalized formula for calculating side $x$ of $2n$-sided regular polygon in the star (having $2n$ no. of vertices & $a$ is the distance between two adjacent vertices) can be derived as follows (see figure below)
The angle of spike of regular star polygon is given as
$$\alpha=\text{interior angle of a n-sided regular polygon}=\frac{(n-2)\pi}{n}$$
Now, draw a perpendicular from one vertex of star to the side of small regular polygon to obtain a right triangle.
Using geometry of right triangle, the length of perpendicular drawn to the side of small regular polygon can be obtained
$$=\frac{a}{2}\csc\frac{\pi}{2n}-\frac{x}{2}\cot\frac{\pi}{2n}$$
Hence, in right triangle, one should have
$$\tan\frac{(n-2)\pi}{2n}=\frac{\frac{x}{2}}{\frac{a}{2}\csc\frac{\pi}{2n}-\frac{x}{2}\cot\frac{\pi}{2n}}$$
$$\cot\frac{\pi}{n}=\frac{x}{a\csc\frac{\pi}{2n}-x\cot\frac{\pi}{2n}}$$
$$x=\frac{a\csc\frac{\pi}{2n}\cot\frac{\pi}{n}}{1+\cot\frac{\pi}{n}\cot\frac{\pi}{2n}}$$
$$x=\frac{a\cos\frac{\pi}{2n}}{\cos\frac{\pi}{n}\cos\frac{\pi}{2n}+\sin\frac{\pi}{n}\sin\frac{\pi}{2n}}$$
$$x=\frac{a\cos\frac{\pi}{n}}{\cos\left(\frac{\pi}{n}-\frac{\pi}{2n}\right)}$$
$$\bbox[5px, border:2px solid #C0A000]{\color{blue}{x=\frac{a\cos\frac{\pi}{n}}{\cos\frac{\pi}{2n}}}}$$
$$\forall \ \ \ \ n\ge 3\ \ (n\in N)$$
Hence for a regular hexagon in the hexagram (see in the above diagram) , setting $2n=6$ or $n=3$, the side of regular hexagon
$$x=\frac{a\cos\frac{\pi}{3}}{\cos\frac{\pi}{6}}=a\frac{\frac{1}{2}}{\frac{\sqrt 3}{2}}=\color{red}{\frac{a}{\sqrt3}}$$
Call the $(2n+1)$-gon $P$, and label an arbitrary vertex $A$. We assume without loss of generality that $A$ is one of the chosen vertices. Let the other two chosen vertices in clockwise order be $B$ and $C$, respectively, and suppose there are $k$ edges of $P$ contained within minor arc $BAC$. Because $\triangle ABC$ contains the center of $P$, we have $\angle BAC < 90^\circ$. This implies $k\cdot \frac{180}{2n+1} < 90$; hence $1\le k \le n$.
For a fixed $k$ between $1$ and $n$ inclusive, it is easy to verify that there are $k$ choices of $B$ and $C$ such that exactly $k$ edges of $P$ are contained within minor arc $BAC$ . This gives us a total of $\sum_{k=1}^n k = n(n+1)/2$ valid choices for $B$ and $C$. Because we have $\binom{2n}{2}$ ways to choose $B$ and $C$ from all the remaining vertices of $P$, the probability that a randomly chosen triangle contains the center of $P$ is $$ \frac{\frac{n(n+1)}{2}}{\binom{2n}{2}} = \frac{n+1}{2(2n-1)}. $$
Note: This probability implies there are a total of $\frac{n(n+1)(2n+1)}{6} = \sum_{k=1}^n k^2$ triangles that contain the center of $P$. Is there also a bijective proof that counts this directly?
Best Answer
We are going to prove that provided the lengths of $n$ segments ($n\ge3$) satisfy the condition: $$ 0<a_1\le a_2\le\dots\le a_n;\quad a_n<\sum_{i=1}^{n-1} a_i $$ it is possible to construct out of these segments a $n$-sided polygon.
I will assume that for triangles ($n=3$) the statement is established (let me know if you have troubles with this and I will provide a proof).
Assume now that the statement is valid for $n-1$ segments. We will demonstrate that this implies that it is valid for $n$ segments as well (observe that $n$ is at least $4$).
Consider the following cases:
$a_n-a_1>a_{n-1}: \max(a_n-a_1,a_{n-1},\dots,a_2)=a_n-a_1;\quad a_n-a_1< \sum_{i=2}^{n-1}a_i$;
$a_n-a_1\le a_{n-1}: \max(a_n-a_1,a_{n-1},\dots,a_2)=a_{n-1};\quad a_{n-1}< a_n-a_1+\sum_{i=2}^{n-2}a_i$.
Thus if we decrement the length of the longest segment by that of the shortest one and remove the shortest segment from the set, the longest of the remaining segments will be less than the sum of the rest $n-2$ segments.
Let $AB=a_{n}$ be the longest segment. Construct a point $F$ on the segment $AB$ such that $AF=a_1$ (see the figure below). By the induction hypothesis from the remaining $\color{red}{n-1}$ segments we can construct a polygon $BC\dots EF$. Now reflect the segments $AF$ and $FE$ about the line $AE$ to obtain the polygon $ABC\dots EF'$ with $\color{red}n$ sides.
PS. One can note that the above proof fails in the case 2. if (and only if) $n=4$ and the equalities $$a_1=a_2,\quad a_3=a_4$$ hold. In this case the explicit construction of the quadrilateral (parallelogram or kite) helps.