The Siskiyou County Museum is the best indoor stop in Yreka. Its collections cover Native history, mining, pioneer settlement, and the county's long relationship with the mountains, forests, and border routes around it.
The outdoor exhibits are especially useful if you like local history in physical form: old buildings, tools, and everyday objects that make the Gold Rush and ranching stories less abstract. Check hours before going, as they are limited.