The Narva River Promenade is the easiest way to feel the city's border geography. The path runs below the castle and bastions, with the river on one side and Russia immediately opposite. There are few places in the EU where another country feels so close and so administratively distant.
Walk it slowly, especially around sunset. The views back to Hermann Tower and across to Ivangorod Fortress are excellent, and the promenade ties together the castle, bastions, and lower riverbank better than the street grid above.