St James's Palace is the older, more reserved royal neighbour to Buckingham Palace. Built for Henry VIII in the 1530s, it is still the official seat of the monarch, which is why ambassadors are appointed to the Court of St James's.
You cannot tour the palace interior, but the Tudor gatehouse and red-brick frontage are worth a look while walking between St James's Park, Pall Mall, and The Mall. It is best treated as a quick historic stop rather than a destination in itself.