Narva Castle, also called Hermann Castle, is the city's essential sight. The Livonian Order fortress rises directly above the Narva River, with the Russian fortress of Ivangorod almost close enough to shout at. The setting is dramatic in a way that no museum display can improve: two castles, one river, one border.
Inside, the Narva Museum covers the city's medieval, Swedish, Russian, and Estonian layers. The restored Hermann Tower gives the best view in town, especially down the river and across to Ivangorod. Allow at least ninety minutes, more if you want to read the exhibits properly.