The question here is really about the best way to select lag length for a VAR, as I noted in this answer. Granger causality doesn't even enter into it until your model for the time series is selected, which is why you may not see many papers specifically concerned with lag order for Granger causality tests. It's more about lag order selection for vector autoregressive models. I'd take a look at this paper for a relatively recent reference on which criteria (AIC, BIC, SIC, HQC) are most appropriate, though they may largely agree for your application.
The trade-off is between bias and power. Too few lags, you have a biased test because of the residual auto-correlation. Too many, you allow for potentially spurious rejections of the null - some random correlation might make it look like $X$ helps predict $Y$. Whether or not that's a practical concern depends on your data, my guess would be to lean higher, but lag length can always be determined as follows:
Granger causality always has to be tested in the context of some model. In the specific case of the granger.test function in R, the model has p past values of each of the two variables in the bivariate test. So the model it uses is:
$$
y_{i,t}=\alpha+\sum_{l=1}^p \beta_ly_{i,t-l} + \gamma_lx_{i,t-l}+\epsilon_{i,t}
$$
A conventional way to choose $p$ for this model would be to try this regression with various values of $p$ and use keep track of the AIC or BIC for each lag length. Then run the test again using the value of $p$ which had the lowest IC in your regressions.
In general the number of lag in the model can be different for $x$ and $y$ and a Granger test will still be appropriate. It's in the specific case of the implementation of granger.test that your constrained to the same number of lags for both. This is a matter of convenience not a theoretical necessity. With different lag lengths for the two variables, you can still use the AIC or BIC to select your model, you'll just have to compare many combinations $n$ lags of $x$ and $m$ lags of $y$. See this.
Just an extra word - because the Granger test is model dependent, omitted variables bias may be a problem for Granger causality. You may want to include all the variables in your model, and then use Granger causality to exclude blocks of them instead of using the granger.test function which only does pair-wise tests.
Best Answer
Partially answered in comments:
Have you read Dave Gile's blog posts on testing Granger causality? (They will not help you with coding but with lag order selection.) – Richard Hardy
https://davegiles.blogspot.com/2011/04/testing-for-granger-causality.html
You can have a look at this document: http://www.stat.wisc.edu/courses/st333-larget/aic.pdf It shows how to do the implementation in R, but the idea should be easily portable to Python. – Matthias Schmidtblaicher