I've never heard someone express this but after my first few weeks of the Sunday League season I think you're spot on. Had two tough games involving three teams with reputations for being unpleasant enough and tough to control when things don't go their way, and ultimately I've decided to knock it on the head, switch to youth on Sundays, and try to find a Saturday league to middle/AR in open age.
However, I've been struck how for the most part no matter how much of a sh1thjouse a player/players/coaches have been throughout the game almost every single one comes up to you at full-time as if the last 90 minutes plus didn't happen.
I think those of us on here clearly take what we do seriously regardless of the level, beat ourselves up when we get or think we got something wrong, and are probably better refs than the standard we ref at. But I agree that rhe vast majority of Sunday league players have forgotten about it if not at full-time, or back in the changing room, but by the time they've got to the pub.