[Tex/LaTex] How to make page numbers skip note slides

Question

I am putting together a beamer presentation. I have notes for each page so that when I send the slides to my co-authors they have an idea of what I am going to say on each slide, what I am going to emphasize, etc. so they can give me feedback. Some prefer the notes, some do not, so I send them out according to the preference. However, when I compile with notes on and \setbeamertemplate{footline}[page number] in the preamble, the notes are numbered along with the slides, so that when I get notes back saying "fix the typo in slide 6", or when I want to discuss the slides with more than one person, the slide numbers are different.

How can I set the template such that the page/slide numbers only show up on the slides and skip the notes?

MWE:

\documentclass{beamer}
\usetheme{Pittsburgh}
\setbeameroption{show notes}
\setbeamertemplate{footline}[frame number]

\begin{document}
\frame{
\frametitle{Title}
\framesubtitle{Subtitle}
This is some text
\note{This is a note}
}
\end{document}

The result I would like is for the notes to not be numbered, but to keep the style.

If I use samcarter's solution below, the note template (defined in beamerouterthemedefault.sty) gets redefined and all of the formatting gets wiped out. Alternatively, I can go into beamerouterthemdefault.sty and copy/paste the entire definition into my preamble, adding in the page counter line. I can also edit the default style file to include this line, which will get overwritten the next time beamer is updated.

It may be that what I want is not possible with beamer, in which case I'll stick with copying the entire definition into my preamble and adding the one line, or else writing a new theme that incorporates these changes when I am not against a deadline.

Last edit:
The above actually gets me very close to what I want. Taking out the re-definition of the notes page keeps the style, takes the page numbers off the notes, and only increments each frame. The only minor quibble is that now if I have multiple parts within a frame (using only or uncover, for instance) they all show up as the same frame number. I can work with this, but would still like to know if there is a way to do this with page numbers.

Answer 1

\documentclass{beamer}
\setbeamertemplate{footline}[frame number]
\setbeameroption{show notes} 

\setbeamertemplate{note page}{%
    \addtocounter{page}{-1}
    \begin{minipage}{2.1\textwidth}
        \normalfont 
        \fontsize{20}{34}
        \selectfont 
        \insertnote
    \end{minipage}

}
\begin{document}

    \begin{frame}
        test
        \note{testnote}
    \end{frame}

    \begin{frame}
        test2
    \end{frame}

\end{document}