Concept: separate shirts for men and women, one showing a lone woman among men (the wider conference), another showing a lone man among women (the miniconference)
Please note: The colours used here are indicative only, I'm sure these specific colours would be awful for printing. Also, I don't anticipate that the shirt would be white. But this should give you the idea.
Shirt for women
Version 1: highlighted woman among men
Version 2: highlighted women with sole man
Shirt for men
Version 1: highlighted man among women