Wizard's Staff is bait.
In the base game, I hate drawing Wizard's Staff on turn 1. Every gamer instinct says to play it right away for maximum value before it becomes unplayable. However, waiting an entire extra hunt to have the potential for Gandalf the White and his action die is so limiting.
When can you afford to not sacrifice Gandalf? Perhaps if you got Aragorn turn 1 because you had lots of movement and a Will of the West AND there were 4+ eyes in the hunt. Seems pretty uncommon.