The Bagan Archaeological Museum sits in Old Bagan and holds a collection of artefacts removed from the temples over the past century: glazed Jataka tiles, Buddha images, stone inscriptions, and items from the royal palace. It is not a large museum and most visitors skip it in favour of the temples themselves, but it provides useful context for what you are seeing outdoors.
The most interesting exhibits are the inscribed stone slabs, which record land grants and donations to temples in Old Burmese script, and the glazed Jataka tiles recovered from the Mingalazedi and other pagodas. Admission is included in the Bagan archaeological zone entrance fee. The museum is usually cool and uncrowded — a good midday retreat from the heat.