Las Bóvedas, the vaulted chambers built into the city walls, houses a long arcade of craft shops selling hammocks, emeralds, mochilas (woven bags), and Colombian souvenirs. Prices are tourist-level but the setting is worth seeing. Compendium on Plaza Bolívar has a better-curated selection of artesanías. Ábaco Libros y Café is the place for Colombian and Latin American books.
For emeralds, the Museo de la Esmeralda has a small museum explaining what you're buying before you enter the adjacent showroom. The Mercado de Bazurto is where locals shop and is not aimed at tourists — but it's the most authentic market experience in the city.