I don't think Vader would even give him a chance to use the wand for example and just break it like a twig with the force and then force choke his ass or just do him in with the lightsaber or his bare hands.
It's hard to explain but I think I get it...... Gravitas doesn't come with popularity either...... It kind of just comes.....
Maybe there is something to be said about that itself and the imposing command that Darth Vader brings in it of itself...... Or him being such a badass extrudes such gravitas