I think it'd be easier if Pats fans just said "yeah Brady is an asshole off the field and does shady shit but I like it when the Pats win so I don't give a shit beyond that"
I think this is somewhat how most Pats fans feel. He is like the Tom Cruise of football. Cruise may be a crazy nutcase about some things, but damn does he make good and entertaining movies.