Consider the space of planes in $ \mathbb{R}^3 $. This space is known as the Grassmannian of affine planes in $ \mathbb{R}^3 $. It is a line bundle over $ \mathbb{R}P^2 $
$$
\mathbb{R} \to \text{Graff}_2(\mathbb{R}^3) \to \mathbb{R}P^2
$$
where the map $ \text{Graff}_2(\mathbb{R}^3) \to \mathbb{R}P^2 $ associates an affine plane to the unique line through the origin that is orthogonal to it (equivalently the unique plane through the origin which is parallel to it).
In general the affine Grassmannian $ \text{Graff} $ is a vector bundle over the (regular/linear) Grassmannian $ \text{Gr} $
$$
\mathbb{R}^k \to \text{Graff}_{n-k}(\mathbb{R}^n) \to \text{Gr}_{n-k}(\mathbb{R}^n)
$$
where $ \text{Graff}_{n-k}(\mathbb{R}^n) $ is the manifold of all
$ n-k $ dimensional affine subspaces of $ \mathbb{R}^n $. And the map $ \text{Graff}_{n-k}(\mathbb{R}^n) \to \text{Gr}_{n-k}(\mathbb{R}^n) $ is given by sending an affine $ n-k $ dimensional subspace to the unique $ n-k $ dimensional linear subspace which is parallel to it. It turns out that $ \text{Graff}_{n-k}(\mathbb{R}^n) $ is diffeomorphic to the total space of the tautological bundle over $ \text{Gr}_k(\mathbb{R}^n) $ see Affine Grassmannian and the tautological bundle
A familiar example of this is $ \text{Graff}_1(\mathbb{R}^2) $
$$
\mathbb{R} \to \text{Graff}_1(\mathbb{R}^2) \to \mathbb{R}P^1
$$
which is just the Moebius strip viewed as the manifold of all lines in the plane (these are affine lines so they don't need to go through the origin). An interesting structure here is that the Euclidean group $ E_n $ acts by isometries and so takes any affine $ n-k $ dimensional subspace to another affine $ n-k $ dimensional subspace. Indeed $ \text{Graff}_{n-k}(\mathbb{R}^n) $ has a transitive action by $ E_n $ and can be written
$$
\text{Graff}_{n-k}(\mathbb{R}^n) \cong E_n/(E_{n-k}\times O_k)
$$
where $ E_{n-k} $ is the part of the stabilizer of a fixed $ n-k $ dimensional affine subspace consisting of isometries of the subspace itself while $ O_k $ is the part of the stabilizer just consisting of rotations of the orthogonal complement. Note that this is all analogous to expressing a Grassmannian as
$$
\text{Gr}_{n-k}(\mathbb{R}^n)\cong O_n/(O_{n-k}\times O_k)
$$
One difference is that while taking orthogonal complements gives a canonical isomorphism $ \text{Gr}_{n-k} \cong \text{Gr}_k $ this is not quite true for $ \text{Graff}_{n-k} $ and $ \text{Graff}_{k} $ since they are vector bundles of different dimensions, $ n-k $ versus $ k $, over the same base space $ \text{Gr}_{n-k} \cong \text{Gr}_k $ (the affine Grassmannians only coincide when $ n-k=k $).
At this point it is unclear how all this relates to the title of the question $ \mathbb{R}P^3 \# \mathbb{R}P^3 $. To see the connection, first lets consider $ \mathbb{R}P^2 \# \mathbb{R}P^2 $, in other words the Klein bottle. In the same way that the Moebius strip can be viewed as the space of all affine lines in the plane we can sort of "roll up" or "compactify" the Moebius trip by identifying all parallel lines which are an integer distance apart. The action of $ E_2 $ on the set of all collections of parallel lines in the plane that are an integer distance apart is transitive (again $ E_2 $ acts by isometries so take integer distances to integer distances). So the Klein bottle can be expressed as
$$
\mathbb{R}P^2 \# \mathbb{R}P^2 \cong E_2/(E_1 \times O_1 \ltimes \mathbb{Z})
$$
where the $ \mathbb{Z} $ factor represents an integer shift from one parallel line to the next. To see this done explicitly in terms of matrices see
Is it possible to realize the Klein bottle as a linear group orbit?
and
https://mathoverflow.net/questions/414402/is-it-possible-to-realize-the-moebius-strip-as-a-linear-group-orbit
EDIT: Ok here is the explicit way to do the 2d stuff with matrices:
The Moebius strip is homogeneous for the special Euclidean group of the plane
$$
\operatorname{SE}_2= \left \{ \
\begin{bmatrix}
a & b & x \\
-b & a & y \\
0 & 0 & 1
\end{bmatrix} : a^2+b^2=1 \right \}.
$$
There is a connected group $ V $ of translations up each vertical line
$$
V= \left \{ \
\begin{bmatrix}
1 & 0 & 0 \\
0 & 1 & y \\
0 & 0 & 1
\end{bmatrix} : y \in \mathbb{R} \right \}.
$$
and
$$
SE_2/V
$$
is a cylinder.
Now if we include the rotation by 180 degrees
$$
\tau:=\begin{bmatrix}
-1 & 0 & 0 \\
0 & -1 & 0 \\
0 & 0 & 1
\end{bmatrix}
$$
then $ \langle V, \tau \rangle$ has two connected components and
$$
\operatorname{SE}_2/\langle V, \tau \rangle
$$
is the Moebius strip. The subgroup we are modding out by here is the stabilizer of the $ y $-axis.
Similarly for the compact case, there is an element that shifts horizontally by one unit
$$
b:=\begin{bmatrix}
1 & 0 & 1 \\
0 & 1 & 0 \\
0 & 0 & 1
\end{bmatrix}
$$
If we mod out $ SE_2 $ by the group generated by $ \langle V,b \rangle \cong \mathbb{R} \times \mathbb{Z} $ we just get a torus. Now if we include the rotation by 180 degrees $ \tau $ then
$$
\operatorname{SE}_2/\langle V,b, \tau \rangle
$$
is a Klein bottle. The subgroup we are modding out by here is the stabilizer of all vertical lines with integer $ x $ intercepts.
Note that we can take either of these constructions and and do it in terms of $ E_2 $, we just have to expand the stabilizer we mod out by to include $ diag(-1,1,1) $ or $ diag(1,-1,1) $ as a generator.
Essentially what we did here is take the vector bundle for the affine Grassmannian and identify it periodically to roll up the fiber into a torus. Indeed one can do this in general to get what I will call $ \text{cGraff} $ the compact affine Grassmanian, a $ k $-torus bundle over $ \text{Gr}_{n-k}(\mathbb{R}^n) $
$$
T^k \to \text{cGraff}_{n-k}(\mathbb{R}^n) \to \text{Gr}_{n-k}(\mathbb{R}^n)
$$
This space still has the interesting structure of a transitive action by $ E_n $
$$
\text{cGraff}_{n-k}(\mathbb{R}^n) \cong E_n/(E_{n-k}\times O_k \ltimes \mathbb{Z}^k)
$$
where $ \mathbb{Z}^k $ corresponds to the discrete subgroup of isometries translating by integer amounts along the orthogonal complement. (I feel pretty confident about the codimension 1 case $ \text{cGraff}_{n-k}(\mathbb{R}^n) $ but something about $ O_k \ltimes \mathbb{Z}^k $ doesn't seem right as a stabilizer in general for example $ \text{cGraff}_{1}(\mathbb{R}^3) $ the stabilizer of the collection of all lines parallel to z axis with $ x,y$ plane intercept with integer coordinates seems like $ O_2 $ could not stabilize ) So for example
$$
\text{cGraff}_{1}(\mathbb{R}^2) \cong \mathbb{R}P^2 \# \mathbb{R}P^2
$$ is the Klein bottle.
Consider the 3-manifold $ M= \text{cGraff}_{2}(\mathbb{R}^3) $
$$
S^1 \to \text{cGraff}_{2}(\mathbb{R}^3) \to \mathbb{R}P^2
$$
Then $ M $ admits a transitive action by $ E_3 $. This paper
https://www.researchgate.net/publication/242949481_Three-dimensional_homogeneous_spaces
seems to strongly imply that the 3-manifold that is a nontrivial circle bundle over $ \mathbb{R}P^2 $ and admits a transitive action by $ E_3 $ is in fact $ \mathbb{R}P^3 \# \mathbb{R}P^3 $. In other words
$$
\text{cGraff}_{2}(\mathbb{R}^3) \cong \mathbb{R}P^3 \# \mathbb{R}P^3
$$
If that really is true, it seems likely that in general the circle bundle over projective space
$$
S^1 \to \text{cGraff}_{n-1}(\mathbb{R}^n) \to \mathbb{R}P^{n-1}
$$
is diffeomorphic to a connected sum
$$
\text{cGraff}_{n-1}(\mathbb{R}^n) \cong \mathbb{R}P^n \# \mathbb{R}P^n
$$
But while I am fairly comfortable with fiber bundles and and smooth homogeneous spaces (manifolds admitting a transitive Lie group action) working with connected sums is a bit out of my wheel house.
Could someone help me understand why $ \text{cGraff}_{2}(\mathbb{R}^3) \cong \mathbb{R}P^3 \# \mathbb{R}P^3 $?
Best Answer
To begin with, I'm going to set $X = \{\text{oriented $(n-1)$-dimensional planes in }\mathbb{R}^n\}$.
There is a function $f:X\rightarrow S^{n-1}$ defined as follows. Given a plane $P\in X$, because it is oriented we can define $f(P)$ to be the unique unit normal to $P$ for which $\{P,f(P)\}$ gives the usual orientation on $\mathbb{R}^n$. It's not too hard to convince yourself that $f$ gives $X$ the structure of an $\mathbb{R}$-bundle over $S^{n-1}$. Moreover, this bundle is principal: $\mathbb{R}$ acts on $X$ by translating $P$ in the direction of $f(P)$.
By quotient by the restriction of this action to $\mathbb{Z}\subseteq \mathbb{R}$, we obtain a principal $S^1$-bundle $S^1\rightarrow X/\mathbb{Z}\rightarrow S^{n-1}$. I'll write $Y$ for $X/\mathbb{Z}$.
Proposition 1: The space $Y$ is diffeomorphic to $S^1\times S^{n-1}$.
Proof: Such a diffeomorphism can just be written down: Given $P\in X$, the function $\phi:X/\mathbb{Z}\rightarrow S^1\times S^{n-1}$ given by $\phi([P]) = (d(P,0)\mod{1}, f(P))$ (where $d(P,0)$ is the signed Euclidean distance from the origin $0\in \mathbb{R}^n$) is a diffeomorphism. $\square$
For later, note that if $-P$ denotes $P$ with the opposite orientation, then $d(-P,0) = - d(P,0)$, and that taken $\mod{1}$, this corresponds to complex conjugation on $S^1\cong \mathbb{R}/\mathbb{Z}$.
There is a natural $\mathbb{Z}/2\mathbb{Z}$ action on $Y$ which maps a class $[P]$ to $[-P]$, where $-P$ refers to $P$ with opposite orientation.
(To save on typing, I'll write $\mathbb{Z}_2$ instead of $\mathbb{Z}/2\mathbb{Z}$ for the rest of this post.) This is well defined because all the planes $P\in [P]$ are translates of each other, so have the same orientation. This action is clearly free, so $Z:= Y/\mathbb{Z}_2$ is a manifold.
Proposition 2: The space $Z$ is diffeomorphic to both $\mathbb{R}P^n\sharp \mathbb{R}P^n$ and also to $\operatorname{cGraff}_{n-1}(\mathbb{R}^n)$.
The fact that $Z$ is diffeomorphic to $\operatorname{cGraff}_{n-1}(\mathbb{R}^n)$ is almost tautological: a point in $Z$ is an infinite family of parallel planes with integer distance between them.
The fact that $Z$ is diffeomorphic to $\mathbb{R}P^n\sharp \mathbb{R}P^n$ will take a bit of work.
For starters, suppose $\mathbb{Z}_2$ acts on $S^1\times S^{n-1}$ diagonally via complex conjugation on the $S^1$ factor and the antipodal map on the $S^{n-1}$ factor. Then it's not too hard to verify that the above diffeomorphism $\phi$ is $\mathbb{Z}_2$-equivariant.
So, to complete the proof, we only need to show ...
Proposition 3: The space $(S^1\times S^{n-1})/\mathbb{Z}_2$ is diffeomorphic to $\mathbb{R}P^n\sharp \mathbb{R}P^n$.
Proof: Observe that there is a decomposition $S^1 = L\cup R$ where $L$ is the left hemisphere and $R$ is the right hemisphere (more rigorously, $L$ consists of the points on $S^1$ with non-positive real part, and $R$ those of non-negative real part.) Both $L$ and $R$ are preserved by complex conjugation, so it follows that we can view $(S^1\times S^{n-1})/\mathbb{Z}_2$ as a union $(L\times S^{n-1})/\mathbb{Z}_2 \cup (R\times S^{n-1})/\mathbb{Z}_2$.
From Proposition 4 below, both "halves" are diffeomorphic to $\mathbb{R}P^n$ with a ball removed, so $(S^1\times S^{n-1})/\mathbb{Z}_2$ is obtained by gluing two copies of $\mathbb{R}P^n$ with open balls deleted. That is, $(S^1\times S^{n-1})/\mathbb{Z}_2$ is diffeomorphic to either $\mathbb{R}P^n\sharp \mathbb{R}P^n$ or $\mathbb{R}P^n\sharp -\mathbb{R}P^n$. However, when $n$ is even, $\mathbb{R}P^n$ is non-orientable, and when $n$ is odd, $\mathbb{R}P^n$ admits an orientation reversing diffeomorphism. So, up to diffeomorphism, we get $\mathbb{R}P^n\sharp \mathbb{R}P^n$. $\square$
So, we need only prove Proposition 4.
Proposition 4: Both $(L\times S^{n-1})/\mathbb{Z}_2$ and $(R\times S^{n-1})/\mathbb{Z}_2$ are diffeomorphic to $\mathbb{R}P^n$ with a ball removed.
Proof: I'll just do the proof for $L$, with the answer for $R$ being similar.
I will identify $L$ with $[-1,1]$ and think of $\mathbb{R}P^n$ as the quotient of $S^n$ by the antipodal action. Using the usual projective coordinates, we map $L\times S^{n-1}$ to $\mathbb{R}P^n$ via $(t,x)\mapsto [\sin(t), \cos(t)x]$. Note that $(t,x)$ and $(t',x')$ map to the same point if and only if $(t,x) = \pm (t',x')$. In particular, this map descends to smooth embedding of $(L\times S^{n-1})/\mathbb{Z}_2$ onto the complement of the ball of radius $1-\sin(1)$ about $[1:0:...:0]$. $\square$