I am unsure aswell sadly...my first thought, the logical one, is either you think that fighting off the grappler would require you to stop the charge, or you could reason it like football, and fighting off a tackle. In the second's case you are more expecting it than in the first.
Hmm...I need to add an AoO with the failed grapple.