I would put a frame with eggs into that hive and check it a week later for queen cells. This would give you the answer as to whether they are queenless or not. It's sounding like they are queenless and in possible danger of turning laying worker to me, from here.
