Late spring and early autumn are the best compromise: long enough days for walking, fewer crowds than summer, and moorland that still has colour. Summer weekends can be busy but sociable.
Rain is part of the Calder Valley story. Check flood warnings in severe weather, especially if staying near the river or canal, and treat winter walks on the tops with proper respect.