Almost everything worth seeing in Rakvere is the castle on the hill and the streets just below it. You can do the lot in an afternoon. Stay overnight if you want to combine it with a quieter, evening walk through the old town and a proper dinner at one of the cellar restaurants.