Stuart et al. mention the R package Zelig, which seemlessly works for post-matching analysis after matching with MatchIt. It is mentioned quite often that you should NOT simply compare the means after matching, although this is quite common practice. You can make optimal use of the matching process by using regression models. When matching has been performed, further (parametric) statistical analysis need to be
performed. Matching is just a first step. It can be seen as a non-parametric method for pre-processing the data in order to create a quasi-randomized study and thus decrease or eliminate the dependence of the outcome variables on the confounding covariates.
1) If your goal is to make a causal inference, balance is paramount. Although you may have improved balance, if it is not good then your causal inference may still be invalid (/your estimate will still be biased). If you have untreated units that fall outside the range of your treated units, your causal inferences will not be valid for them unless you can justify extrapolation. You may want to delete these cases and limit your inferences to the region of overlap. (Overlap can be conceptualized as the overlap between the covariate distributions, e.g., the convex hull, or common support on the estimated propensity scores.)
2) You could randomly simulate, but I think a better approach would be to find the matched groups that yield the best balance and move forward with that single sample.
3) I'm not sure what your outcome is, but an "alternative to matching" is regression (to many, matching is an alternative to regression). If you are willing to make parametric assumptions and use regression of some kind (e.g., logistic, linear, count), you can use regression instead of or in addition to matching. With only 7 covariates that you want to control for, it shouldn't be too hard to run a regression and account for potentially relevant interactions and curviliearities.
4) Hypothesis tests are NOT appropriate for assessing balance. Balance is a sample property, so there is no sense in which a p-value will be more helpful than an effect size measure. Also, typically, balance is achieved by comparing your balance statistics to a chosen threshold, not by looking at reduction in imbalance from your original sample. The fact that you've achieved balance slightly better than you started with doesn't mean you can move forward; you need to arrive at balance that permits an unbiased estimate of the treatment effect.
I recommend you try various methods of conditioning on the propensity scores. You've chosen matching, but there doesn't seem to be reason not to try weighting or full matching (really a form of weighting). Weighting using CBPS or by entropy balancing can be very effective. If you want to compare balance across multiple methods of conditioning, you can use the cobalt package which interfaces with some of the other packages and offers some additional tools for balance assessment. I also recommend you combine propensity score conditioning with regression on the treatment and your covariates. That technique is preferred in the literature and can reduce the remaining imbalance in your adjusted sample.
Best Answer
In order to perform moderation, you need to be able to validly estimate subgroups effects, which means confounding needs to be removed within subgroups of the moderating variable. In the context of matching, this means you must exactly match on the moderator, or equivalently, match within subgroups of the moderator (i.e., performing a separate matching routine within each subgroup).
To estimate the treatment effect, you can fit a model that include an interaction between the moderator and all other variables in the model (including the treatment and any treatment-by-covariate interactions), then perform a marginal effects procedure within subgroups. To assess whether moderation is present, you can test whether the subgroup treatment effects differ from each other.
Some useful resources on moderation analysis: Green and Stuart (2014), Griffin et al. (2022), the
MatchItvignette section on moderation analysis.