OotMm randomizer Gerudo Fortress jail

I feel like a massive idiot but I cannot for the life of me figure out how to get into the jail they throw you in. I checked my spoiler log and have confirmed it was expected to have the gerudo card before coming here and I only have the hookshot not the longshot. How do I get back in there?

EDIT: thanks everyone for the suggestions. I ended up doing a mega flip at the top near the scarecrow. Only took a couple hundred tries to get it right for the first time