New York is surrounded by escapable distances in every direction. New Jersey is the most accessible: cross the Hudson or take NJ Transit from Penn Station and you're out of the city in under an hour.
New Jersey shore
Atlantic City is about two and a half hours south by car or NJ Transit bus from Port Authority — a strange, entertaining place to spend a day, with a free beach, the historic Boardwalk, and the surreal déjà vu of walking past Monopoly street names in real life. Cape May sits at the very tip of New Jersey, an hour further south: a Victorian seaside town with landmark gingerbread cottages, wide Atlantic beaches, and one of the best birding spots on the East Coast. Both are doable as long day trips; Cape May rewards an overnight if you can swing it.
Within the boroughs
You don't have to leave the city for a change of scene. Head up to the Bronx for a day combining the New York Botanical Garden and Bronx Zoo, finishing with dinner on Arthur Avenue — the real Little Italy.