My wife and I really enjoy Artisanal Fromagerie & Bistro (
http://artisanalbistro.com ). It's only about 3 blocks from you, on East 32nd Street between Madison and Park.
Aside from the cheeses, their specialty, we've never been disappointed by other items on their menu.
Oh - if you're tempted to try a fondue, skip the "100 Cheese Fondue" if it's on offer that day. It's the one and only thing that didn't live up to they hype. It was well made, but so many cheeses comprise it that it actually loses any individual character and comes out monochromatic and rather dull. We thought so, anyway.
Hope you have a successful trip, and the weather is nice for you!