Dirt Dauber (aka mud wasp)- they catch spiders, lay eggs in them, and then wall them up in those little chambers to hatch. They don't hurt anything, they don't normally sting. They are quite annoying due to their habit of building those mud nests in places you don't want them, like up in gas burners.
Thanks for the help! I didn't even think to check the solitary wasps. I'll put 'em back out and let them be.