The shichibukai might have already made the deal with the government and teamed up to attack him