Let $k$ be a field and consider the polynomial ring $A = k[x_1,...,x_n]$.
Claim: Given $f \in A \setminus \{0\}$, $(f)$ is radical if and only if $f$ factors into a product of irreducible polynomials of multiplicity $1$.
Proof:
$\Leftarrow$: We know $A$ is a UFD. So, let $f = f_1\cdots f_m$ be a product of $f$ into irreducible factors such that for all $i \neq j$, $(f_i) \neq (f_j)$. Then $$(f) = (f_1\cdots f_m) = (f_1) \cap \cdots \cap (f_m)$$ (I am using unique factorization for the second equality). Thus, $(f)$ is an intersection of prime ideals of $A$ and hence radical.
$\Rightarrow$: Suppose $(f)$ is radical. Again, let $f = {f_1}^{e_1}\cdots {f_m}^{e_m}$ be a product of $f$ into irreducibles where $i \neq j$ $\Rightarrow$ $(f_i) \neq (f_j)$.
Our goal is to show that each $e_i = 1$. Well, suppose not. Then there exists $e_i$ such that $e_i > 1$. Then $({f_1}^{e_1}\cdots {f_i}^1\cdots {f_m}^{e_m}) \subseteq (f) \subseteq ({f_1}^{e_1}\cdots {f_i}^1\cdots {f_m}^{e_m})$. The first inclusion is because ${f_1}^{e_1}\cdots {f_i}^1\cdots {f_m}^{e_m} \in \mathrm{Rad}((f)) = (f)$, and the second inclusion follows from the fact that ${f_1}^{e_1}\cdots {f_i}^1\cdots {f_m}^{e_m}\mid f$.
But this means that there is some $u \in A^\times$ such that ${f_1}^{e_1}\cdots {f_i}^1...{f_m}^{e_m} = u{f_1}^{e_1}\cdots {f_i}^{e_i}\cdots {f_m}^{e_m}$, which contradicts unique factorization.
If there exists a set of $n$ distinct maximal ideals intersecting to $J(R)$, then the Chinese remainder theorem says that $R/J(R)$ is isomorphic to $n$ fields, and such a ring has exactly $n$ maximal ideals.
So if just one set of $n$ exists, they are precisely the entire set of maximal ideals.
Best Answer
There are indeed algorithms for the polynomial rings, based on the notion of Groebner bases —you can learn about this from David Cox et al.'s book Varieties, Ideals and Alrgorithms.
Those algorithms are actually implemented in several computar algebra systems. I usually use Macaulay2, which google can find for you.
Your ideal $I=(x^2y,xy^3)$ is what's called a monomial ideal, because it is generated by monomials; it has the nice property that a polynomial belongs to $I$ if and only if each of its monomials belongs to $I$, and you can probably prove this easily. It is not hard to see that its radical is also generated by monomials, and then you can probably see which monomials you need in this specific case. I'll let you try your hand at this before giving you the answer...