Tanjung Pinang itself is a working port with no real beach to speak of. The waterfront is great for atmosphere -- stilted houses, fishing boats, the smell of drying fish -- but not for swimming.
The nearest good beach is Trikora, about 50 km east along Bintan's coast. It is a long stretch of white sand backed by coconut palms, with a handful of simple guesthouses and warungs. The water is calm and clear, and on weekdays you may have it largely to yourself.
For island hopping, boats from the harbour run to Soreh and Terkulai, both about an hour away. Terkulai has exactly one permanent resident -- the lighthouse keeper -- and if you ask nicely he may let you climb the tower. Tour operators in town offer day trips covering two or three islands with lunch included.