Tavush Fortress, also called Tslik Amram's Fortress, is the ruined stronghold above Berd. Its surviving walls are fragmentary, but the site still explains why this river valley mattered: from the ridge you can read the folds of the hills and the old routes through them.
The fortress is more atmospheric than polished. Wear decent shoes, expect uneven ground, and treat it as a short walk into local history rather than a restored monument.