Madison Barracks grew from Sackets Harbor's frontier military role and remained an active US Army post until after the Second World War. Many of the stone and brick buildings have been restored, making the former base one of the strongest visual reminders of the village's long military life.