I think the knee-shaking comes from trying to forcefully break the habits, as opposed to "fearing" the opponent per se.

In the current fight, Killua will run and dodge as he finds out more about his opponent. He's also careful not to overpersue so that he won't fall into a trap. As Shoot pointed out, those are very good reactions and fighting skills. There's nothing wrong with that.

In trying to overcome his habits (and by extension: his thoughts - sometimes it's hard to say which follows the other), Killua is trying to do things he previously considered dangerous and suicidal (overpursuing, not backing out after taking damage) - and that scares him.

As for the stinger-girl before, I thought he overestimated the girl - it just turned out to do no damage. He really did screw up there. If we look at the beginning of that fight, he was dodging and sussing her out.

I get what Kraco's saying, and I think David's point about the current fear having to do with Nen may have something to do with it.