St Peter's, on Marefair, is a Grade I listed Norman church of exceptional quality that is now redundant — no longer used for regular worship — and in the care of the Churches Conservation Trust. The interior is a serious piece of Norman architecture: round arches with elaborate carved decoration, a rich chancel arch, and carved capitals that show the skill of 12th-century Northampton stonemasons.
The church is notable in the context of the town because it survives from before the Great Fire of 1675 that destroyed most of medieval Northampton; much of what you see dates to around 1150. The interior is open on certain days; check the Churches Conservation Trust website. The location, on a busy road near the railway station, means it is easily missed, but it is worth seeking out.