In order to verifiy if Peirce's law is sufficient, when added to Deduction Theorem and modus ponens, we can try to verify if the (complete) axiom system for propositional logic of Elliott Mendelson, Introduction to Mathematical Logic (4th ed - 1997) [page 35] can be derived under these assumptions.
(A1) $\mathcal{B} \rightarrow ( \mathcal{C} \rightarrow \mathcal{B})$
We have that :
$B$ 1 --- assumption
$C$ 2 --- assumption
$B$ 3 --- assumption
$C \rightarrow B$ 4 --- deduction theorem from 2 and 3
$B \rightarrow (C \rightarrow B)$ 5 --- deduction theorem from 1 and 4.
(A2) $(\mathcal{B} \rightarrow ( \mathcal{C} \rightarrow \mathcal{D})) \rightarrow ((\mathcal{B} \rightarrow \mathcal{C}) \rightarrow (\mathcal{B} \rightarrow \mathcal{D}))$
We have that :
$B \rightarrow (C \rightarrow D)$ 1 --- assumption
$B \rightarrow C$ 2 --- assumption
$B$ 3 --- assumption
$C$ 4 --- modus ponens from 2 and 3
$C \rightarrow D$ 5 --- modus ponens from 1 and 3
$D$ 6 --- modus ponens from 4 and 5
$B \rightarrow D$ 7 --- deduction theorem from 3 and 6
$(B \rightarrow C) \rightarrow (B \rightarrow D)$ 8 --- deduction theorem from 2 and 7
$(B \rightarrow (C \rightarrow D)) \rightarrow ((B \rightarrow C) \rightarrow (B \rightarrow D))$ 9 --- deduction theorem from 1 and 8.
We still have to derive
(A3) $(\lnot \mathcal{C} \rightarrow \lnot \mathcal{B}) \rightarrow ((\lnot \mathcal{C} \rightarrow \mathcal{B}) \rightarrow \mathcal{C})$.
But, according to Implicational propositional calculus
the axiom system formed by (A1), (A2) and Peirce's law with the rule of inference modus ponens is semantically complete with respect to the usual two-valued semantics of classical propositional logic.
Note. Peirce's law is necessary, due to the fact that $\nvdash_{(A1)(A2)}(Peirce)$. The addition of Peirce’s law is sufficient, due to some results of Tarski and Bernays.
In order to have "full" propositional calculus, we have to add the falsum symbol ($\bot$) and an additional axiom : Ex Falso Quodlibet ($\bot \rightarrow \mathcal{A}$).
There are indeed many presentations of classical propositional logic1.
If complexity was measured purely in terms of number of axioms, then yes, Mendelson's axiom system would be more "economical". We could be even more "economical" with Meredith's system: $$((((A\to B)\to(\neg C\to\neg D))\to C)\to E)\to((E\to A)\to(D\to A))$$ Just rolls right off the tongue. While minimality is often a driver, people do still need to discover more minimal systems. (Indeed, the above is not the most minimal system if we include the complexity of the axiom and not just the number.) There's also the question of accepting the axioms. Ideally, we want the axioms to be "self-evident" or at least easy to understand intuitively. Maybe it's just me, but Meredith's axiom does not leap out at me as something that should obviously be true, let alone sufficient to prove all other classical tautologies.
Minimality is, however, not the "whole point" of axiomatic systems. You mention another reason: sometimes you actually do want to prove things at which point it is better to have a richer and more intuitive axiomatic system. You may argue that we can just derive any theorems we want to use from some minimal basis, and then forget about that basis. This is true but unnecessary complexity if we have no other reason for considering this minimal basis. When we compare different styles of proof system (e.g. Hilbert v. Sequent Calculus v. Natural Deduction), translations between them (especially into Hilbert-style systems) can involve a lot of mechanical complexity. That complexity can sometimes be significantly reduced by a careful choice of axioms.
For the Laws of the Excluded Middle (LEM) and Non-contradiction, the first thing you'd need to do is define the connectives. You can't prove $\neg(P\land\neg P)$ in a system that doesn't have $\land$. Given $\neg$ and $\to$ as primitives, standard definitions of $\land$ and $\lor$ are $P\land Q:\equiv \neg(P\to\neg Q)$ and $P\lor Q:\equiv\neg P\to Q$. With these definitions (or others), then yes, the LEM and Non-contradiction can both be proven in the systems you mention and any other proof system for classical propositional logic. Your concern here is an illustration that we often care about which axioms we have and not just that they're short and effective.
This also leads to another reason why we might want a certain presentation. We may want that presentation to line up with other, related logics. As you're starting to realize, it is ill-defined to say something like "intuitionistic propositional logic (IPL) is classical propositional logic (CPL) minus the LEM". When people say things like this, they are being sloppy. However, "CPL is IPL plus LEM" is unambiguous. Any presentation of IPL to which we add LEM is a presentation of CPL. For that presentation, it makes sense to talk about removing LEM. It doesn't make sense to talk about removing an axiom without a presentation of axioms that contains that axiom. It is also quite possible to have a presentation of CPL containing LEM that becomes much weaker than IPL when LEM is removed. In fact, you'd expect this because a presentation of IPL with LEM added is likely to be redundant because things which are intuitionistically distinct become identifiable when LEM is added. The story is the same for paraconsistent logics.
While it isn't as much of a driver for Hilbert-style proof systems, for natural deduction and sequent calculi the concerns of structural proof theory often push for more axioms (or rules of inference, rather). For example, many axioms in a Hilbert-style proof system mix together connectives, e.g. as Meredith's does above. This means we can't understand a connective on its own terms, but only by how it interacts with other connectives. A driving force in typical structural presentations is to characterize connectives by rules that don't reference any other connectives. This better reveals the "true" nature of the connectives, and it makes the system more modular. It becomes meaningful to talk about adding or removing a single connective allowing you to build a logic à la carte. In fact, structural proof theory motivates many constraints on what a proof system should look like such as logical harmony.
1 And that link only considers Hilbert-style proof systems.
Best Answer
Axiom scheme 1 with $\lnot B$ replacing $B:$ $$ A \to (\lnot B \to A). $$ New axiom scheme with letters $A,B$ interchanged: $$(\lnot B \to A) \to (B \to \lnot A).$$ Modus ponens on last two lines: $$A \to (B \to \lnot A).$$ Axiom scheme 3 with $C=\lnot A$ and modus ponens applied to this: $$ (A \to B) \to (A \to \lnot A).$$ In this replace $B$ by $A$: $$(A \to A) \to (A \to \lnot A)$$ At this point I need to invoke $A \to A.$ [I did not get that from Hilbert's axioms, but surely it must follow from them] If I can do that, next apply modus ponens and get to: $$A \to \lnot A.$$ This is a contradiction, since it implies every proposition is false, for example we could replace $A$ here by the entire axiom scheme 1, call that $U,$ then modus ponens, and arrive at a proof that $U$ and $\lnot U$ both hold.