As you already have found out the problem occurs every time the hyperref package will be loaded prior to the minted package. This is happening here:
The hyperref package will be loaded. It does not detect the float package (because the float package is not loaded yet), and therefore it does not patch the float package code.
The minted package will be loaded which loads the float package, and does a \newfloat.
The caption package will be loaded. It does detect the float and hyperref packages, and assumes that the float package code was patched by hyperref. But this assumtion is false in this case.
So as solution you could load the float package prior to the hyperref package and the minted package after the hyperref package - which BTW is the correct load order anyway, see hyperref's README for details.
\documentclass{report}
% it's all in the order of the imports
% this one is the correct one:
% 1. Load the float package
\usepackage{float}
% 2. Load the hyperref package; hyperref will patch the float package
\usepackage{hyperref}
% 3. Load the minted package which will use the patched \newfloat now
\usepackage{minted}
% The load order of the caption package is (or should be) irrelevant now
\usepackage{caption}
\begin{document}
\chapter{test}
\clearpage
\begin{listing}[H] \mint{cl}/(car (cons 1 2))/
\caption{Example of a listing.}
\label{lst:example}
\end{listing}
\clearpage
Listing \ref{lst:example} contains an example of a listing.
\end{document}
You need to set the environment up to accept an optional argument if you wish to use one. (This problem has nothing to do with the direction of typesetting.) There is an example on page 42 of the manual for listings.
However, to get the caption typeset correctly, you need to look at page 18 of the manual for bidi which explains the problem and its solution specifically for the case you are interested in i.e. RTL caption (such that generic name -> number -> colon -> specific name) with LTR code box. For this, bidi provides an additional key for listings, captiondirection which you may set to one of three values: RTL, LTR or textdirection. Since you need to switch to LTR to typeset the code, but want to retain RTL for the captions, you want to use RTL to override the current direction of typesetting.
\documentclass[a4paper,10pt]{article}
\usepackage{listings}
\usepackage[usenames,dvipsnames]{xcolor}
\usepackage{xepersian}
\settextfont{Noto Kufi Arabic}%{PakType Naqsh} http://code.google.com/p/noto/
\DefaultMathsDigits
\definecolor{customblue}{RGB}{235,241,245}
\definecolor{light-gray}{gray}{0.95}
\lstdefinestyle{C++Style}{%
backgroundcolor=\color{customblue},
breaklines=true,
basicstyle=\footnotesize\ttfamily,
keywordstyle=\color{blue},
commentstyle=\color{OliveGreen}\textit,
stringstyle=\color{red},
numbers=left,
numberstyle={\tiny\lr},
showspaces = false,
showstringspaces = false,
tabsize = 2,
frame=single,
xleftmargin=5pt,
xrightmargin=3pt,
language = C++,
aboveskip = 20pt,
rulecolor=\color{black},
captiondirection=RTL,
}
\lstnewenvironment{C++Code}[1][]
{%
\lstset{style=C++Style, #1}%
}{%
}
\def\lstlistingname{برنامه}
\begin{document}
%case 1
\begin{LTR}
\begin{C++Code}[caption={تابع فلان}]
int main()
{
return 0;
}
\end{C++Code}
\end{LTR}
%case 2
\begin{LTR}
\begin{lstlisting}[style=C++Style,caption=\rl{تابع فلان}]
int main()
{
return 0;
}
\end{lstlisting}
\end{LTR}
\end{document}
Note that I don't have a font on this computer which supports the right script and includes a colon, so the boxes below are actually colons.
Since this uses caption, it should do all the normal things e.g. get added to the list of listings etc.
Best Answer
I figured it out, thanks to this question.
When I use
\unskipafter the frame, the caption will be positioned right after it.\vspacealso works now, but is not needed anymore, at least for my case.