The Old Baptist Chapel in Church Street is a Grade II* listed building with a history stretching back to at least the early seventeenth century. The building is plain and modest — deliberately so, in keeping with the Dissenting tradition that rejected the ornament of Anglican churches — but the interior retains its period fittings and the scale gives a sense of how a congregation worshipped in an era when Nonconformity was actively persecuted.
The chapel is occasionally open to visitors and can be viewed by arrangement. It sits in a narrow courtyard off Church Street, a few minutes' walk from the abbey, and the contrast between the two buildings — the great Norman church and this deliberately humble meeting house — captures something important about English religious history.