Here's a counter-example.
Let $\mu$ be counting measure supported on $\mathbb Z$; so $\int f(x) \ d\mu(x) = \sum_{m\in\mathbb Z} f(m)$ for $f$ continuous with compact support.
Choose a very rapidly decreasing sequence of positive reals $\delta_n$.
Let $\psi$ be the piecewise linear function which is $1$ at $n+1/n$ for $n\geq 10$ (say), and is $0$ at $n+1/n \pm \delta_n$ (and is zero at all $x<10+1/10-\delta_{10}$). By definition, $\psi$ is continuous (and could be made smooth by the use of a bump function) and $\psi$ is Lebesgue integrable (if $\delta_n$ decreases fast enough).
Then set
\[ \alpha(x) = (\psi*\mu)(x) = \int \psi(x-y) \ d\mu(y) = \sum_{m\in\mathbb Z} \psi(x+m). \]
A priori, this sum might diverge, but only to $+\infty$. Clearly $\alpha$ is periodic in that $\alpha(x+k)=\alpha(x)$ for any $k\in\mathbb Z$.
Fix $x\in[-1/2,1/2)$, and consider which $m\in\mathbb Z$ are such that $\psi(x+m)>0$. This occurs iff there is $n\geq 10$ with $1/n-\delta_n < x+m-n < 1/n+\delta_n$. As $m-n\in\mathbb Z$ and $n\geq 10$ and $|x|\leq 1/2$, this can only occur if $m=n$. So $1/n-\delta_n < x < 1/n+\delta_n$. Choosing $(\delta_n)$ suitably, we can arrange that $1/(k+1)+\delta_{k+1} < 1/k-\delta_k$ for all $k$, and then $n$ is unique for any given $x$.
We conclude that for every $x$, there is at most one $m\in\mathbb Z$ with $\psi(x+m)>0$. In particular, $\alpha(x)\in[0,1]$ for all $x$.
However, clearly $\alpha(1/n)\geq 1$ for all $n\geq 10$, but $\alpha(0) = 0$ as if $\psi(m)>0$ for some $m\in\mathbb Z$, then there is $n\geq 10$ with $-\delta_n < m-n-1/n < \delta_n$ which forces $m=n$, but then $-\delta_n <-1/n<\delta_n$, which we can avoid by a suitable choice of $(\delta_n)$. So $\alpha = \psi*\mu$ is not continuous at $0$.
For instance let $f:[0,1]\to[0,1]$ be the Cantor function and define $g(x):=x+f(x)$. Then $g:[0,1]\to[0,2]$ is a homeomorphism that maps the complement of the Cantor set $C$ onto a measure one open set of $[0,2]$ (just because $g'(x)=1$ on $[0,1]\setminus C$). So $g_{|C}:C\to g(C)$ is a homeomorphism of the Cantor set onto a compact set of measure one, and if $W$ is any non-measurable subset of $g(C)$, $g$ is also a homeomorphism between the Lebesgue measurable null-set $g^{-1}(W)$ and $W$.
edit. As to the issue of finding a non-measurable subset within a Lebesgue measurable set of positive measure $S$, there is such a set of the form $S\cap (V+q)$, the trace on $S$ of a suitable translation of the Vitali set $V$. Indeed, the Vitali set $V$ does not contain any measurable subset of positive measure (reason: if $E\subset V$ then $E-E \subset V-V\subset \big(\mathbb{R}\setminus\mathbb{Q}\big)\cup \{0 \}$, while $E-E$ is always a nbd of zero for any measurable set of positive measure $E$). On the other hand, the sets $S\cap (V+q)$ for $q\in\mathbb{Q}$ are a countable cover of $S$, so one of them has positive exterior measure.
Best Answer
The statement in the question is not true. Given any two enumerable and dense sets in open intervals of the reals, there is a (complex) analytic1 function giving a bijection between them. See the following paper: Analytic Transformations of Everywhere Dense Point Sets, Philip Franklin, Transactions of the American Mathematical Society, Vol. 27, No. 1 (Jan., 1925), pp. 91-100.
The analytic functions can be constructed along similar lines to the method outlined in this MO answer. Is a real power series that maps rationals to rationals defined by a rational function? Also see this mathforum.org discussion on the subject with plenty of links. A question on real-analytic functions
1 The analytic function can be chosen to be entire on the complex plane except for the obvious case where either of the ends of the real interval in the domain is bounded but the corresponding end in the range is unbounded, where the function must have a singularity.