[Tex/LaTex] Putting shade in part of a figure in `picture` environment

Question

I was trying to draw the some diagram by using the picture environment. I want to put shade certain part of the figure, namely inside of the rectangle (with curved corners) created in the left hand side of the figure. The code is given below.

\documentclass[a4paper,12pt]{amsart}
\usepackage{xspace}
\usepackage{amsmath,amscd,amsfonts,amssymb, color}
\usepackage{latexsym, graphicx, rotating,subfig,framed,xcolor,wrapfig}
\begin{document}
\definecolor{shadecolor}{gray}{0.9}
\begin{figure}[htb]
\setlength{\unitlength}{5mm}
\begin{picture}(20,16)(-10,-5)
\put(-5,0){\line(1,0){10}}
\multiput(-5,0)(-0.3,0){10}{\line(-1,0){0.1}}
\multiput(5,0)(0.3,0){10}{\line(1,0){0.1}} %the baselines
\put(-8,0){\circle*{0.3}}\put(8,0){\circle*{0.3}}
\put(-5,0){\circle*{0.5}}\put(5,0){\circle*{0.5}}
%the verticle line
\multiput(0,-5)(0,0.5){15}{\line(0,1){0.2}}
%the box and the write up
\linethickness{0.25mm}
\put(-1,2.3){\framebox(2,1.2){$x$}}
%remaining verticle line
\linethickness{0.1mm}
\multiput(0,3.5)(0,0.5){10}{\line(0,1){0.2}}
%% This is the intended shaded area which I am hiding by %  
   %\begin{shaded}
    \put(-9,-4.5){\line(0,1){10}}
    \put(-4.75,-4.5){\oval(8.5,1.5)[b]}
    \put(-0.5,-4.5){\line(0,1){10}}
    \put(-4.75,5.5){\oval(8.5,1.5)[t]}
    %\end{shaded}
put(-3,-4){Alice}\put(2,-4){Bob}
\put(-3.8,-2){$\mathcal{H}_A=\mathbb{C}^3$}
\put(1.2,-2){$\mathcal{H}_B=\mathbb{C}^3$}
\put(-8,-2){$\mathcal{H}_A^{aux}$}
\put(7,-2){$\mathcal{H}_B^{aux}$}
\put(-7.5,1.5){$|0\rangle_A\langle0|\quad \otimes$}
\put(3.8,1.5){$\otimes\quad|0\rangle_B\langle0|$}
    \end{picture}
    \caption{Schematic diagram} 
    \end{figure}
    \end{document}

If I put the shaded section outside the picture environment, it works and covers the whole figure in a gray shade. As mentioned, I just want a part of the area defined by any contour to be shaded. Is there a way to make this work? This question is partly related to this earlier entry.

NOTE: I am using pdflatex. I wanted to write a few mathematical formulas inside the figure, which is the reason for using picture environment.

Answer 1

I would suggest you using TikZ instead (the code contains some explanatory comments):

\documentclass[a4paper,12pt]{amsart}
\usepackage{amsmath,amscd,amsfonts,amssymb}
\usepackage{braket}
\usepackage{tikz}
\usetikzlibrary{calc,positioning,fit,backgrounds}

\pgfdeclarelayer{background}
\pgfsetlayers{background,main}

\begin{document}

\begin{figure}
\begin{tikzpicture}
% the black circles and their connecting lines
\node[fill,circle,inner sep=1.5pt] (sc1) {};
\node[fill,circle,inner sep=2.5pt,right=1.4cm of sc1] (bc1) {};
\node[fill,circle,inner sep=2.5pt,right=4.5cm of bc1] (bc2) {};
\node[fill,circle,inner sep=1.5pt,right=1.4cm of bc2] (sc2) {};
\path[draw,dashed] (sc1) to (bc1) (sc2) to (bc2) ;
\draw (bc1) -- (bc2);

% labels above
\node[above=of bc1,anchor=east,inner xsep=0pt] {$\ket{0}_A\bra{0}$};
\node[above=of bc1,anchor=west,xshift=10pt] {$\otimes$};
\node[above=of bc2,anchor=west,inner xsep=0pt] {$\ket{0}_B\bra{0}$};
\node[above=of bc2,anchor=east,xshift=-10pt] {$\otimes$};

% labels below
\node[below=of sc1,anchor=west,inner sep=0pt] (ha) {$\mathcal{H}_A^{aux}$};
\node[right=of ha,anchor=west,inner sep=0pt] (hac) {$\mathcal{H}_A=\mathbb{C}^3$};
\node[below=of sc2,anchor=east,inner sep=0pt] (hb) {$\mathcal{H}_B^{aux}$};
\node[left=of hb,anchor=east,inner sep=0pt] (hbc) {$\mathcal{H}_B=\mathbb{C}^3$};
\node[below=0.8cm of hac] (ali) {Alice};
\node[below=0.8cm of hbc] (bob) {Bob};

% the shaded rounded corner rectangle
\coordinate[above=3.5cm of hac] (aux1);
\begin{pgfonlayer}{background}
\node[draw,inner sep=10pt,fill=gray!60,fit=(ali) (sc1) (aux1),rounded corners=10pt] (rect) {};
\end{pgfonlayer}

% the dahed line and its label
\draw[dashed] 
  let \p1=(bc1), \p2=(bc2) in
  (0.5*\x2+0.5*\x1,0|-rect.south) -- +(0,7cm) coordinate[pos=0.65] (aux2);
\node[draw,fill=white,text width=2em,align=center] at (aux2) {$x$};
\end{tikzpicture}
\caption{Schematic diagram} 
\end{figure}

\end{document}

Notice also that I used the braket package.