They don't interfere because the pacing has been dragged out alot compared to the Manga, where these events flowed much much faster than they have here. Additionally, everything that is happening in the war isn't always being show on scene. Like for example, what ever happened to Crocodile and Don Flamingo's fight? Obviously they did fight each other, but they simply chose to not show it. Same can be said about alot of stuff potentially going on. You never know who/what could have been holding up the Admirals, or if they would have even had a chance to interfere.