The Arlington Historical Museum is run by the Arlington Historical Society in the Hume School, the oldest surviving school building in the county. Exhibits cover Arlington's development from rural Virginia through the Civil War, the growth of the federal capital, and the modern suburban county.
It is modest, but useful if you want local context beyond the national memorials. The museum is easiest to combine with a visit to Crystal City, Pentagon City, or the nearby Pentagon Memorial.