Azkaban is not Nurmengard
In Harry Potter: The Order of the Phoenix, Dumbledore tells Fudge that he could break out of Azkaban if he was imprisoned there. If Grindelwald was only a shade weaker than Dumbledore, then why couldn't he also break out of Nurmengard?
I don’t believe for a second that Grindelwald wasn’t capable of wandless magic — it’s not that exceptional a feat. It was only exceptional for little orphan Tom because he was a child, and this all took place before he got a formal understanding of his powers. In reality, whilst wandless magic is a more difficult and consequently rarer feat, it’s performed by a number of adult wizards and witches. Even Tom the landlord of the Leaky Cauldron was capable of it:
“Tom clicked his fingers, a fire burst into life in the grate, and he bowed himself out of the room.” Harry Potter and the Prisoner of Azkaban
In fact, we even see Grindelwald disguised as Graves use wandless magic to fling an entire car:
As you said, Dumbledore even stated that Grindelwald was only a “shade less skilled”, so it’s not a lack there. Nor can anyone accuse him of lacking power, so it can’t be that either.
Overall, he firmly cemented himself as being one of the “big players” alongside Voldemort and Dumbledore.
So why? Because breaking out of Azkaban is not that exceptional. In fact, we even know there’s been a number of breakouts in the past, by lesser magic folk. Azkaban overall is an OK prison, but it ain’t all that difficult to escape if you’re even a half-decent wizard — as Sirius’ solo breakout proves.
Nurmengard though was of Grindelwald’s own design.
The enchantments and magical fortifications were his doing, and so we can expect them to be high-level magic, and what’s more, very probably enhanced by the power of the Elder Wand.
That’s the difference.
Don’t be expecting even Dumbledore and Voldemort to handle complex magic without their wands, because it won’t happen.