The mud volcanoes near Gobustan are about 7 km south of the petroglyphs site, near the road to Alat, and they are worth the detour. The landscape around them is genuinely surreal — grey mud cones of various sizes rising from a flat plain, some gently bubbling, others still. The mud is cold (unlike volcanic lava) and the whole area has an eerie, lunar quality.
There are dozens of volcanoes in the Gobustan field, ranging from small puddles to cones several metres high. You can walk among them freely; wear shoes you do not mind getting muddy. No entrance fee, no facilities. Combine with the petroglyphs into a half-day trip from Baku, or a full day if you want to take your time at both sites.