I may be a newbee, but it seems like your queen is giving you your answer. If you want the bees to lay in comb and they aren't laying in it, but are laying in fresh comb, then replace it....if it's not completely empty you could probably wait. If the comb is 100 years old and she's still laying in it, then leave it unless you see some problem emerging (poor bee health, etc.)
As for chemicals, I understand the issues around frames from hives that have made the pollination circuit and been around orchards, fields, etc. that have had mass amounts of chemical treatment, but I also know that as a backyard beek, not all of my neighbors are as organic and environmentally conscious as I am when they maintain their yards, not to mention the dept. of transportation's annual spraying they do on the highway. That means that I have chemicals coming into my hives too (probably trace amounts of roundup, weed n' feed, DEET, Miracle Gro, car exhaust, cigarette components, industrial pollutants, coal dust, whatever else people put on their lawns and gardens, etc.). Given what transits the planet in dust, etc (we get pollution from Asia here on the US West Coast), unless you're beekeeping on the moon, you're not going to have a "pure" hive.