Usually the notation $\mathbb{Z}_p$ or $\mathbb{Z}/p\mathbb{Z}$ mean the integers modulo $p$, that is $\{0,\ldots,p-1\}$ where you add and multiply as usually and then take the reminder modulo $p$.
The notation of $\mathbb{Z}_p^\times$ is for those numbers which have a multiplicative inverse modulo $p$, namely all $n$ such that exists $m$ such that $n\cdot m$ is $1$ modulo $p$. These are the numbers coprime to $p$, the greatest common divisor of them and $p$ is $1$.
A subgroup of $\mathbb{Z}_p^\times$ is a subset of these numbers which is closed under multiplication (but not necessarily addition), and every number in this subset also has its multiplicative inverse in there.
For example,
$\mathbb{Z}_5 = \{0,1,2,3,4\}$
$\mathbb{Z}_5^\times = \{1,2,3,4\}$
$\{1,4\}$ is a subgroup of $\mathbb{Z}_5^\times$. Can you see why?
Big $\mathcal{O}$ notation is a device useful in asymptotic analysis that helps us bound error terms in the approximations that we derive. More specifically, when you see an equation of the form:
$$\text{blah}=\text{stuff}+\mathcal{O}(\text{error})$$
the meaning is that $|\text{blah}-\text{stuff}|\le C\cdot\text{error}$ for some constant $C$. We have to be careful because, as this is asymptotic analysis, we're only claiming this inequality (for some constant) holds in some neighborhood of a region of interest. For example, if our analysis is as $x\to\infty$, we intend the error statement to hold in some neighborhood of infinity, or $(a,\infty)$ for some $a$, or if we look at $\to0^+$ we would mean that the error statement holds in an interval $(0,\epsilon)$ for some $\epsilon>0$.
Take the function $f(x)=1/(1-x)$ for example. This has the following Maclaurin series:
$$\color{Red}{\frac{1}{1-x}}= 1+x+x^2+x^3+\cdots=\color{Blue}{1+x+x^2}+\color{Purple}{x^3}\color{LimeGreen}{\big(1+x+\cdots\big)}.$$
Say we want to do analysis about the point $x=0$. In the interval $(-\frac{1}{2},\frac{1}{2})$ the part in green is no more than $f(1/2)=2$, so we can say that $\big|\color{Red}{f(x)}-\color{Blue}{1+x+x^2}\big|\le \color{LimeGreen}2\color{Purple}{x^3}$, or in Big-$\mathcal{O}$ notation,
$$\frac{1}{1-x}=1+x+x^2+\mathcal{O}(x^3). $$
Keep in mind this analysis applies only sufficiently close to the point $x=0$, so it doesn't apply to the singularity $x=1$ for example. As you hopefully can see, this is a tidy way to convey the asymptotic size of a remainder or leftover term in an approximation, such as a series expansion.
In these easier analyses (exercises with relatively simple and elementary functions), to find the expansion up to $\mathcal{O}(x^n)$ we simply write out the usual series expansion about the appropriate point, but truncate it at the $a_nx^n$ term and replace the deleted terms with a simple $\mathcal{O}(x^n)$.
Mathematicians love to abuse notation when it's convenient, so on top of what I've said so far you need to understand what it means to do operations with Big-$\mathcal{O}$ terms. One mentally helpful interpretation of any $\mathcal{O}(g(x))$ term is "some function bounded by $g$ up to a constant, but what it is specifically isn't important for my purposes or just at the moment." So when you see two $\mathcal{O}$ terms in an equation, you can just imagine the same interpretation for each separately, so there are two unknown error/leftover functions that are independent of each other but at least you're keeping track of how each is bounded in the region under consideration.
Consider, for example, $x^n\mathcal{O}(x^m)$: if $\text{error}(x)\le C x^m$ (our big-$\mathcal{O}$ is a placeholder for $\text{error}$), then we have $x^n\cdot\text{error}(x)\le C x^{n+m}$, which tells us we can just replace $x^n\mathcal{O}(x^m)$ with $\mathcal{O}(x^{n+m})$. Similar reasoning applies to the general term $f(x)\mathcal{O}(g(x))$. Exercise: roughly describe what the product of two $\mathcal{O}$ terms, i.e. $\mathcal{O}(f(x))\mathcal{O}(g(x))$, would mean to us, and find a single $\mathcal{O}$ term for $\mathcal{O}(x^n)\mathcal{O}(x^m)$.
Now if you were able to follow my answer you should be capable of following the explicit derivation given for $\rm5(a)$.
Two notes: in order to evaluate the series expansion of the function composition $f\circ g$, first write the expansion of $f(g)$ in the letter g, i.e. $a_0+a_1g+a_2g^2+\cdots$, and then replacing $g$ with its power series and taking powers where needed, then gathering up all the terms. You do not need to do this with the general, symbolic form of the terms, but rather truncate the expansions of $f$ and $g$ appropriately and work with a finite number of power and $\mathcal{O}$ terms (Exercise: figure out what "appropriately" means here).
Best Answer
‘Best’ here would be ‘smallest’, so as Henning said, in all likelihood you’re being asked to write $e^{\tan x} = 1 + x + \frac{x^2}{2} + o(x^n)$ as $x\to 0$ for the smallest integer $n$ that makes the statement true.
Added: As Brugerfugl points out, that should be ‘largest’, since were’ looking at $x\to 0$.