The BYU-Idaho campus is open to visitors and free to walk through. The grounds are genuinely pleasant — well-landscaped and maintained to a standard you don't always find at institutions of this size. The Eliza R. Snow Performing Arts Center is the architectural standout, and the university puts on a regular programme of concerts, theatre, and lectures, some of which are open to the public.
Even if you have no interest in the university itself, the campus is worth a short walk for the context it provides. Rexburg exists in its current form largely because BYU-Idaho is here — without the university, this would be a much smaller agricultural settlement. The relationship between the church, the university, and the town is visible everywhere you look, and the campus is the clearest expression of it. You don't need to share the faith to find it interesting.