The remains of Chipiona's old castle — built by the dukes of Medina Sidonia in the fifteenth century — stand at the corner of the Plaza del Castillo, a short walk from the sanctuary. Not much survives: a stretch of curtain wall, a thick tower, and the outline of the inner courtyard, now grassed over. Restoration work in the 2010s opened the site to visitors with information panels in Spanish and English.
It takes twenty minutes to walk around. Worth combining with a visit to the parish church of Nuestra Señora de la O nearby, the small main square, and a coffee at one of the cafes along the way.