Seeing the code would help to understand the points where it could be (or not) better than it is now, but initially there is a basic point you mention that could be enhanced, in your last step:
"So iterate from 1 to n, check if number has any prime factor less than Y, if yes, dont add it, otherwise add it."
- You would not need to start from $1$ but from $Y+1$, because a number under or equal to $Y$ has all the divisors under or equal to $Y$, so:$$n \ge Y+1$$(only the special case $n=1$ will be in your addition by default).So any $n \in [2,Y]$ can be discarded.
Now supposing that in a first step you sieved all the primes, then:
Now for the interval $[Y+1,Y^2]$ you would just need to add the prime numbers inside the interval that you already sieved, not the composite numbers. This is because any composite $n$ has at least a divisor under $\sqrt{n}$, so any composite number in the interval $[Y,Y^2]$ at least has a divisor in the interval $[2,Y]$. So if you had made in a first step the sieving of the primes, you will not need to verify again any number in $[Y,Y^2]$, just directly add the primes in that interval.
And finally you would need to test all composite numbers in the interval $[(Y+1)^2,X]$, because some of them could have all the divisors (except $1$) over $Y$, and in the other hand you can directly add all the primes in that interval.
I hope it helps, if you add the code I could have a look, just let me know in the comments. Probably is fine as it is, specially if the interviewer did not explain any details about how to make it better.
I'm going to post the method involving trig functions. As an example, we will be using $0=x^3-3x^2+3$.
To use this method, you must know that
$$\cos(3\theta)=4\cos^3(\theta)-3\cos(\theta)$$
which is easily derived from the sum of angles formulas and the Pythagorean theorem.
We manipulate this to give us
$$\cos(3\arccos(r))=4r^3-3r\tag0$$
Anyways, we start with
$$0=ax^3+bx^2+cx+d$$
$$0=x^3-3x^2+3$$
We always start with the substitution $x=y-\frac{b}{3a}$, which for our example is $x=y+1$
$$0=(y+1)^3-3(y+1)^2+3\\=y^3-3y+1$$
In general, we get something along the lines of
$$0=a\left(y-\frac{b}{3a}\right)^3+b\left(y-\frac{b}{3a}\right)^2+c\left(y-\frac{b}{3a}\right)+d\\=ay^3+py+q$$
Where $p$ and $q$ are constants.
Make the substitution $y=uz$ and multiply both sides by $v$.
$$0=v(uz)^3-3v(uz)+3v\\=vu^3z^3-3vuz+v$$
And have $vu^3=4$ and $-3vu=-3$ so that it comes in our $(0)$ form. Solving this system of equations gives $u=2$ and $v=\frac12$.
$$0=4z^3-3z+\frac12$$
$$0=\cos(3\arccos(z))+\frac12$$
Solving for $z$ and recalling the period of cosine, we get
$$z=\cos\left(\frac{2\pi k}3+\frac13\arccos\left(\frac{-1}2\right)\right)$$
$$y=2z=2\cos\left(\frac{2\pi k}3+\frac13\arccos\left(\frac{-1}2\right)\right)$$
$$x=1+y=1+2\cos\left(\frac{2\pi k}3+\frac13\arccos\left(\frac{-1}2\right)\right)$$
$$x=1+2\cos\left(\frac{2\pi k}3+\frac{2\pi}9\right)\qquad k=1,2,3$$
This is not always possible, but under those cases, we can just switch to different trig functions and use their corresponding formulas. It's pretty easy to see which trig function you should use at the "solve for $u$ and $v$ step".
An advantage to this method is that it avoids casus irreducibilis, which is a fancy way of saying you can't factor a cubic polynomial using only real numbers, radicals, and basic arithmetic operations. This does not include trig functions, however, which can be used to obtain nice forms of the solution.
To compare, the radical solution for the above polynomial is given as
$$x_1=1-\frac12(1-i\sqrt3)\sqrt[3]{\frac12(-1+i\sqrt3)}-\frac{1+i\sqrt3}{\sqrt[3]{4(-1+i\sqrt3}}$$
$$x_2=1-\frac{1-i\sqrt3}{\sqrt[3]{4(-1+i\sqrt3)}}-\frac12(1+i\sqrt3)\sqrt[3]{\frac12(-1+i\sqrt3)}$$
$$x_3=1+\frac1{\sqrt[3]{\frac12(-1+i\sqrt3)}}+\sqrt[3]{\frac12(-1+i\sqrt3)}$$
Compared side by side, you might find one form much nicer.
Best Answer
That's called multiplicative partitions, and there is a generating function discovered by Oppenheim and McMahon. You could use it. The list of the number of multiplicative partitions is on http://oeis.org/A001055