A good guide on what factors to consider when mixing fonts is Thierry Bouche's Diversity in math fonts article in TUGboat, Volume 19 (1998), No. 2.
The most important aspect is to use the same font for text and math letters (as well as letter-like symbols as \partial or \infty). This has drawbacks as some letters will suffer from spacing problems, but compared to the other option (using totally different math letters), it's really a lesser evil. Of course, if this is not acceptable to you, then you should first choose the math font and then use the same font for text, but that limits your font choices dramatically.
Once you've assigned the text font to the math letters, the remaining choices you face is for the geometric symbols, the delimiters and the big operators (\sum, \int, \bigcup etc.). The main consideration is color (how bold the symbols are) and the shape of the symbols (mainly the shape of sum or integral symbols, especially if you use them often). Compared to Libertine, XITS and Asana are a bit too bold (especially true for the sum symbol), Latin Modern is a bit too light (especially +, \otimes, etc.), and Cambria has a very huge \sum symbol, huge \otimes and \oplus as well as very bold \bigcup. Thus, which font will look better will depend on what type of math you're typing, and none will be perfect.
Here's a sample to show the results of this font mixing with Libertine. Notice the spacing problems around the f in f(r_k) and \Sigma_c f(r) due to the fact that's it's a text font we're using for math. I've not set all letter-like symbols to come from Libertine (only \infty), so there's still room for improvement. (Note also the missing parenthesis in one of the formulas with Latin Modern Math.)
\documentclass{article}
\usepackage{amsmath}
\usepackage{fontspec}
\usepackage{unicode-math}
\setmainfont{Linux Libertine O}
\newcommand{\setlibertinemath}{%
% use Libertine for the letters
\setmathfont[range=\mathit/{latin,Latin,num,Greek,greek}]{Linux Libertine O Italic}
\setmathfont[range=\mathup/{latin,Latin,num,Greek,greek}]{Linux Libertine O}
\setmathfont[range=\mathbfup/{latin,Latin,num,Greek,greek}]{Linux Libertine O Bold}
%\setmathfont[range={"2202}]{Linux Libertine O}% "02202 = \partial % doesn't work
\setmathfont[range={"221E}]{Linux Libertine O}% "0221E = \infty
% etc. (list should be completed depending on needs)
}
\newcommand{\sample}{%
When computing the sums $\sum_{k=0}^{+\infty}{f(r_k)}$ of $f$ the integral representation of $K_0(x)$ may be used.
\[
\eta(r)\frac{\partial f}{\partial r} + 2\Sigma_cf(r)
= \sum_{k=0}^{+\infty}{K_0\mathopen{}\left(\frac{\lvert r - r_k \rvert}{L}\right)}
= \int_{0}^{\infty}{e^{-\left(z+\frac{r^2}{4L^2\pi}\right)} \frac{dz}{2z}}.
\]
We then use
\[
\bigcup_{\lambda \in \Lambda}{U_\lambda} \cap \bigsqcup_{\delta > 0}{G_\delta} = \bigcap_{i \in I}{\mathbf{A}_i}
\quad \text{so that} \quad
u \otimes w \oplus v = 0.
\]
}
\pagestyle{empty}
\begin{document}
\section{Libertine + Latin Modern}
\setmathfont{Latin Modern Math}\setlibertinemath
\sample
\section{Libertine + Cambria}
\setmathfont{Cambria Math}\setlibertinemath
\sample
\section{Libertine + XITS}
\setmathfont{XITS Math}\setlibertinemath
\sample
\section{Libertine + Asana}
\setmathfont{Asana Math}\setlibertinemath
\sample
\end{document}
I use the following code to help with supporting xetex and pdftex with Linux Libertine and Latin Modern Mono along with Biolinum. You need to use fontspec, and you don't need to load the libertine package at all with xetex unless you want some of the libertine-specific glyph commands, so you can remove it.
\usepackage{ifxetex}
\usepackage{ifthen}
\ifxetex
\usepackage{libertine}
\usepackage[log-declarations=false]{xparse} % same as below
\usepackage[quiet]{fontspec} % remove quiet option to check for errors
\setromanfont[Ligatures={Common,TeX}]{Linux Libertine O}
\setmainfont[Ligatures={Common,TeX}]{Linux Libertine O}
\setmonofont[SmallCapsFont={Latin Modern Mono Caps}]{Latin Modern Mono Light}
\setsansfont{Linux Biolinum O}
\usepackage{xunicode}
\else
\usepackage{sansmath}
%\usepackage{lmodern}
\usepackage{libertine}
\usepackage{DejaVuSansMono}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{textcomp}
\usepackage{epstopdf}
\usepackage[final,expansion=true,protrusion=true,spacing=true,kerning=true]{microtype}
\fi
TeX Live should have the fonts installed already, but I use miktex, so I can't help with distribution specific troubles. Remember to rehash your db.
Update: disregard DejaVuSansMono and update the pdftex code accordingly.
Best Answer
You will have to use a font that you have on your computer (just as I had to download Vrinda to mine for your code). Linux Libertine O is available for free download, as are the others I mentioned in my answer to your original question. If you have Microsoft products on your computer, you may have some fonts that have a few IPA symbols (Calibri, Cambria, Times New Roman, etc.), but they might not work well with diacritic stacking or overbars.
There's no IPA font that is perfect in every respect (e.g., Linux Libertine O is missing a few superscript IPA characters and has clashing overbars, Charis SIL doesn't have ligatures or true small caps, Gentium doesn't have ligatures or true bold or true small caps). See this answer and this answer for more discussion.
You can also use
\usepackage{libertine}to get Linux Libertine O but that doesn't generalize to other IPA fonts like the\setmainfontwith a system font does.