Nautholsvík Thermal Beach is Reykjavik's oddest swim: a golden-sand beach where geothermal water is piped into a sheltered section of the Atlantic. The sea is still bracing, but the warm pool and hot tubs make it a very Icelandic way to test the North Atlantic.
It is close enough to the centre to work as a short outing, especially in summer when locals come to sunbathe, soak, and swim.