Got 3rd party? Bastard & Bloodlines has "Lost Tradition", a 1st lvl only feat which says "Pick a class, your casting stat changes to something else." It doesn't exclude physical stats, though I personally always rule it has to be another mental stat. You can have a Str based wizard, or a Wisdom based bard.... You don't need to take said casting class at 1st level, just need to take the feat then.