Can I Use A Frame Feeder In An Empty Box Above Brood Box?

You can but watch it or the next thing you will have is comb hanging off the inner cover. Instead I would add frames to the box. So if they do start to build you can control it.
David

If it turns cold, the bees might not be able to get up there to the feed.

I am trying this with two hives now and so far so good. I did not add frames in the top box.

Put in your frame feeder and some empty frames foundation or comb. Or get a top hive feeder.

I had an empty medium stacked on top of the inner cover (over a deep) all winter. I fed the bees on top of the inner cover. I also had them inside my barn so they would stay a little bit warmer and be able to come up to feed, which they did. As soon as we get reliably warm temps, I will return to a more standard hive configuration. But this worked nicely for my situation.