Consider a $9\times9$ array, where the row represents the event and the column represents the round. For each house, arrange the $9$ copies numbers from $1$ to $9$ in the array, so that each number appears once in each row and each column. That is, make a $9\times9$ Latin square. Call these arrays $(a_{ij}),(b_{ij}),(c_{ij}),(d_{ij})$ for groups A,B,C,D, respectively.
If $a_ij=m$ for example, it means that team $A_m$ will play event $i$ in round $j$. Now suppose that arrays $(a_{ij}),(b_{ij})$ are orthogonal, that is if $$a_{ij}=a_{mn}=x$$ and $$b_{ij}=b_{mn}=y$$ then $$x=y.$$ Then $A_x$ meets team $B_y$ at most once, and since $A_x$ must meets $9$ different teams, it meets team $B_y$ exactly once.
We see that the conditions can be fulfilled if and only if there exist $4$ pairwise orthogonal Latin squares. Since $9=3^2$ is a prime power, it is known that there exist $8$ pairwise orthogonal Latin squares, and the answer is "yes."
The construction is described here.
EDIT
On reflection, I think I should have included a specific construction. Here is a set of $8$ mutually orthogonal $9\times9$ Latin squares, the maximum possible.
$$\begin{array}{|c|c c c|c c c|c c c|}\hline&1&2&3&4&5&6&7&8&9\\ \hline 1 &11111111&56489723&98765432&69358247&72634859&24973568&85296374&37542986&43827695\\
2 &22222222&64597831&79846513&47169358&83415967&35781649&96374185&18653794&51938476\\
3 &33333333&45678912&87954621&58247169&91526748&16892457&74185296&29461875&62719584\\
\hline4 &44444444&89723156&32198765&93682571&15967283&57316892&28539617&61875329&76251938\\
5 &55555555&97831264&13279846&71493682&26748391&68124973&39617428&42986137&84362719\\
6 &66666666&78912345&21387954&82571493&34859172&49235781&17428539&53794218&95143827\\
\hline7 &77777777&23156489&65432198&36925814&48391526&81649235&52863941&94218653&19584362\\
8 &88888888&31264597&46513279&14736925&59172634&92457316&63941752&75329461&27695143\\
9 &99999999&12345678&54621387&25814736&67283415&73568124&41752863&86137542&38476251\\
\hline\end{array}$$
The rows represent rounds, and the columns represent events. The table is for $8$ houses, say $A,B,C,D,E,F,G,H.$ The sequence indicates which teams from each house will compete in a particular event in a given round. For example, at row $3$, column $4$ we find $58247169$. The interpretation of this is that in round $3$, the teams that compete in event $4$ are $$A_5,B_8,C_2,D_4,E_7,F_1,G_6,\text{ and }H_9$$
In the instant case, where there are only $4$ houses, the last $4$ digits or each table entry can simply be ignored.
Best Answer
Assuming that during every shift all 12 people are involved in 4 groups of 3, it obviously takes at least 4 shifts before everyone will have worked at some point with every from all the other groups, because for everyone there are 8 such 'other' people, and during any shift they can only get to meet 2 people.
In fact, here is a schedule that does it in exactly 4 shifts:
\begin{array}{c|cccccccccccc} Shift&a&b&c&d&e&f&g&h&i&j&k&l\\ \hline 1&1&2&3&4&1&2&3&4&1&2&3&4\\ 2&1&2&3&4&2&1&4&3&3&4&1&2\\ 3&1&2&3&4&3&4&1&2&4&3&2&1\\ 4&1&2&3&4&4&3&2&1&2&1&4&3\\ \end{array}
(so, during every shift, group together the people with the same number)