It sounds as though you brushed the swarm into the ice box, closed it up and went home right away. It's best to give them some time to cluster up in the container which would indicate the queen is in there with them before going home. The queen may have been left behind.
A frame of eggs and larva would confirm or time will tell.
A frame of eggs and larva would confirm or time will tell.