In the Kingdom of Ice: The Grand and Terrible Polar Voyage of the USS Jeannette by Hampton Sides — not about Svaneti, but if you are going to spend time in the mountains, packing a book about extremity is appropriate. For Svaneti specifically, the literature is thin in English.
The Knight in the Panther's Skin by Shota Rustaveli — The 12th-century Georgian epic poem, written during the reign of Queen Tamar, is the foundational text of Georgian literature. The poet is named in Svaneti at the Shota Rustaveli Peak in the Bezengi Wall. Reading even a portion gives context for why medieval Georgian culture produced the level of artistry visible in the Svaneti museum collections.
A Journey to the Western Caucasus by Douglas Freshfield (1869) — Freshfield was a British mountaineer who made some of the first ascents in the Caucasus, including in the Svaneti area. His account is out of print but available digitally; it describes the region almost exactly as it appears now, in terms of landscape and village structure.