Across from the race course, this is the definitive museum of American thoroughbred horse racing. The Hall of Fame section honors the sport's great horses, jockeys, and trainers in a proper institution with good curatorial standards. Interactive exhibits let you try handicapping and experience the view from a jockey's perspective; they're more engaging than you'd expect.
The collection of racing memorabilia is impressive — silks, trophies, paintings, and photographs going back to the 19th century. A scale model of the Saratoga Race Course shows how little it has changed since the Victorian era. The museum is open year-round and is worth a couple of hours whether or not you have any particular interest in horse racing.