Egil's Saga (anonymous, 13th century) — The Old Norse sagas are the primary literary record of Viking-age Vestfold, and Egil's Saga is among the most gripping: a poet-warrior's life spanning raids, feuds, and the political turbulence of early Norway. Several scenes involve the coast around Tønsberg. Reading it before visiting sharpens the landscape considerably.
The King's Mirror (Konungs skuggsjá, anonymous, c. 1250) — A Norwegian didactic text structured as a father's advice to his son, covering commerce, the sea, court life, and the natural world. It was likely composed in the Vestfold region and gives a vivid picture of the concerns and worldview of medieval Norway at its peak.
Knut Hamsun, Hunger (1890) — Hamsun is Norway's great difficult genius — Nobel laureate and later Nazi collaborator — and Hunger is his most urgent book: a starving writer adrift in a Norwegian city, half-mad with pride and desperation. It is not set in Tønsberg, but it captures something true about the pressures of small-country Norwegian life that formed the world around this coast.
Edvard Munch: Behind the Scream by Sue Prideaux (2005) — Munch lived and worked for years at Åsgårdstrand, a short drive from Tønsberg, and this biography is the best in English: rigorous, readable, and genuinely illuminating about how the fjord landscapes around Vestfold fed into his painting. Worth reading before a visit to his house at Åsgårdstrand.