Minstrelsy of the Scottish Border by Walter Scott — The defining collection of Border ballads, compiled by Scott in the early 19th century while he was living nearby at Abbotsford. The ballads deal with the history, feuds, and legends of the Borders; this is the primary literary document of the region.
The Thirty-Nine Steps by John Buchan — Buchan was a Borders man, born in Perth but deeply rooted in the Border landscape, and it shows in the pacing and setting of his fiction. The Thirty-Nine Steps is his most famous novel — a tight, fast-moving thriller that remains highly readable.
Mary Queen of Scots by Antonia Fraser — The definitive biography, detailed and fair-minded. Essential reading if the Mary Queen of Scots House piqued your interest; Fraser covers the Jedburgh visit and the Borders period in full.
Walking the Border — Several walking guides cover the Borders landscape around Jedburgh, including routes along Jed Water and across the hills to the other abbeys. Any current edition from a publisher such as Cicerone will do.