Unfortunately the question I am asking isnt very well-defined. But I will try to make it as precise as possible. Supposed I am given a mod-p representation of $G_Q$ into $Gl_2(F_p)$. I want to check for arithmetic invariants so that I can conclude that the representation comes from a modular form but not an elliptic curve. The whole point of this exercise is to understand the difference between the representations coming from elliptic curves and cusp forms in general. I hope I was able to make the question precise. A few things that one can look at is the conductor of an elliptic curve (i.e. the exponent of 2 in the level of modular form is too high then it cant come from an elliptic curve) or one can look at the Hasse bound for $a_l$ for different primes. But I want to know some non-trivial arithmetic constraints attached to such invariants. Also if such a representation doesnt come from an elliptic curve then it must come from an abelian variety of $GL_2$ type. Can anything be said about that abelian variety in general.
[Math] Galois representation attached to elliptic curves
arithmetic-geometrynt.number-theory
Best Answer
Since your representation $\overline{\rho}$ is defined over $\mathbb F_p$, you can't do things like the Hasse bounds, since the traces $a_{\ell}$ of Frobenius elements at unramified primes are just integers mod $p$, and so don't have a well-defined absolute value.
One thing you can do is check the determinant; this should be the mod $p$ cyclotomic character if $\overline{\rho}$ is to come from an elliptic curve. In general (or more precisely, if $p$ is at least 7), that condition is not sufficient (although it is sufficient if $p = 2,3$ or 5); see the various results discussed in this paper of Frank Calegari, for example. In particular, the proof of Theorem 3.3 in that paper should give you a feel for what can happen in the mod $p$ Galois representation attached to weight 2 modular forms that are not defined over $\mathbb Q$, while the proof of Theorem 3.4 should give you a sense of the ramification constraints on a mod $p$ representation imposed by coming from an elliptic curve.