Yarmouth is actually three villages folded into one Cape Cod town: Yarmouthport on the bay side, and West Yarmouth and South Yarmouth along the south shore. They have almost nothing in common in feel, which is part of what makes the place interesting.
Yarmouthport, on the quieter north shore, is old Cape Cod — tree-lined Route 6A, a handful of antique dealers, weathered sea captains' houses, and a working general store. This is the village worth slowing down for. The south shore along Route 28 is a different world: mini-golf, go-karts, shops selling salt-water taffy, and the kind of summer-resort bustle that families with young kids tend to love and everyone else either tolerates or avoids.
The town has 15 beaches between them — salt marshes, kettle ponds, and open Atlantic — plus some good conservation trails through the wooded interior. The Bass Hole Boardwalk in Yarmouthport stretches out over a tidal marsh toward the bay and is worth the short walk, particularly around sunset.
Yarmouth's cultural highlight is the Edward Gorey House — the former home and studio of the idiosyncratic illustrator, now a museum. The Whydah Pirate Museum on the south shore is the other attraction worth driving for. Yarmouth sits between Hyannis to the west and Dennis to the east, making it a convenient base if you're working your way along the Cape rather than a destination in itself. That said, Yarmouthport alone is worth a stop for anyone who wants to see what Cape Cod looked like before the strip malls arrived.