Port Perry Downtown is Scugog's main reason to stop: a compact strip of brick storefronts, independent shops, cafes, galleries, and small-town Ontario street life a short walk from the lake. It has been used often as a film stand-in, which tells you something about its tidy, period look.
Do not rush it. The point is to browse, eat, and wander down to the waterfront rather than hunt for one grand monument.