There is no contradiction because $f$ is not Riemann-integrable on $[0,1]$. The fact that the limit $\lim_{c \to 0}\int_c ^1f(x)dx$ exists does not mean that $f$ is Riemann-integrable on $[0,1]$. Note that Rudin says in the exercise that we can define the symbol $\int_0^1f(x)dx$ to mean said limit in the case where we have a function on $(0,1]$. It does not mean that $f$ is Riemann-integrable on $[0,1]$. (Indeed, it isn't in your example. It isn't even bounded.) It is just an assignment of a value to a symbol.
PS: Note that part of the exercise is even to show that the two (a priori possibly conflicting) definitions of the symbol $\int_ 0^1f(x)dx$ agree when $f$ is Riemann-integrable.
A) Suppose $f \in \mathcal{R}$ on $[0,1]$. Let $\epsilon >0$. Let $M= \sup\{ |f(x)|: 0 \leq x \leq 1\}$. Fix $c \in (0, \frac{\epsilon}{4M})$, and consider any partition of $[0,1]$ that contains $c$, where the lower and upper Riemann sums are $\sum M_j (t_j - t_{j-1})$ and $\sum m_j (t_j - t_{j-1})$ of $f$ differ by less than $\frac \epsilon 4$. Then the partition of $[c,1]$ formed by the points of this partition that lie in the interval surely has the property that its upper and lower Riemann sum $\sum M'_j(t_j - t_{j-1}) $ and $\sum m'_j (t_j - t_{j-1})$ differ by less than $\frac \epsilon 4$. From above we have
$$\sum M_j (t_j - t_{j-1}) \frac \epsilon 4 < \int_0^1 f(x) dx < \sum m_j (t_j - t_{j-1}) + \frac \epsilon 4$$
and
$$\sum M'_j (t_j - t_{j-1}) \frac \epsilon 4 < \int_c^1 f(x) dx < \sum m'_j (t_j - t_{j-1}) + \frac \epsilon 4$$
Moreover, we have
$$\left|\sum M_j (t_j - t_{j-1}) - \sum M'_j (t_j - t_{j-1}) \right| < \frac \epsilon 4$$
and
$$\left|\sum m_j (t_j - t_{j-1}) - \sum m'_j (t_j - t_{j-1}) \right| < \frac \epsilon 4.$$
From these two inequalities you can see that
$$\left|\int_0^1 f(x) dx - \int_c^1 f(x)dx \right|< \epsilon.$$
B)- let
$$f(x) = (-1)^n (n+1).$$
For $\frac{1}{n+1} < x\leq \frac{1}{n}, n=1,2,...$ Then if $\frac{1}{N+1} \leq \leq \frac{1}{N}$ we have
$$\int_c^1 f(x) dx = (-1)^N (N+1) (\frac 1 N -c) + \sum_{k=1}^{k-1} \frac{(-1)^k}{k}.$$
Since $0\leq \frac{1}{N} -c \leq \frac{1}{N} - \frac{1}{N+1} = \frac{1}{N (N+1)}$, the first term on the right-hand side tends to zero as c goes to zero, while the sum converges to $\ln 2$. However,
$$\int_c^1 |f(x)| dx = (N+1) (\frac{1}{N}) + \sum_{k=1}^{N-1} \frac{1}{k},$$
although in this case the first term in the right-hand side goes to zero, in contrast, the summation goes to infinity.
Best Answer
$b$) Using the well known integral $$ \int_{1}^\infty \frac{\sin x}{x}\mathrm dx $$ which converges conditionally, we reflect reflect everything to near $0$ by sending $$ x\to 1/x $$ and find $$ \int_{1}^\infty\frac{\sin x}{x}\mathrm dx= \int_0^1\frac{\sin(1/y)}{y}\mathrm dy $$
For part $a$, you must prove using the definition of the Riemann integral that for a function which is integrable on $[0,1]$, the number $$ \int_0^1 f(x)\mathrm dx $$ is the same as the number $$ \lim_{c\to 0^+}\int_c^1f(x)\mathrm dx $$