Akhalkalaki is a small city in the Samtskhe-Javakheti region of southern Georgia, sitting on a high plateau near the Armenian border. It is not a tourist destination in the conventional sense, but it earns its place on any itinerary through southern Georgia as a transit hub. Buses and minibuses connect it to Tbilisi and Akhaltsikhe, and from here you can cross into Armenia.
If you are trying to reach the cave monastery of Vardzia, Akhalkalaki is not the most convenient base — Akhaltsikhe serves that better — but it is possible. Take a marshrutka minibus towards Khertvisi and try to pick up an onward connection to Vardzia from there.
The border crossing into Armenia is straightforward. Minibuses and shared taxis run to Gyumri, Armenia's second city; a private taxi should cost around $20.