The Priory Church of St Mary began as part of a Benedictine nunnery founded in the 12th century. Much has changed since the Dissolution, but the church still keeps medieval fabric, a quiet churchyard, and the sense that Usk's religious history was once bigger than the town's present size suggests.
Look for the old priory gatehouse nearby and the details inside the church if it is open. It is a calm, worthwhile counterpoint to the castle ruins on the other side of the centre.