Give them one warning if you feel like it, then caution. If he's stupid enough to carry on then he goes.
The second one can be argued a yellow, especially if he's thrown his hands up. I'd not do much otherwise but the hands in the air is a visual sign of dissent. If you let one get away with it, they'll all try it.
I infrequently do Sunday morning games, but have done a few this season. I had a game with a player much like the OP.
Quiet word with him after 4 minutes as he's already arguing 2 straightforward decisions.
After 7 minutes he does it again so I have him in with the captain and give him the clear and loud talking to, to which his captain tells him to "shut the f**k up and get on with his game."
13 minutes in to the match, off he goes again, I have him straight over and in the book.
Not one peep from him or the rest of either team for the remainder of the game. At the end of the match both captains and managers came up to me and thanked me telling me it was one of the most enjoyable games they've had this season.
It is a really difficult balancing act. On the one hand, if you ignore it, or don't sanction effectively, it becomes like a plague spreading to the other players and before long you have potential for there to be more than one "problem player." On the other hand if you go for the sanction early, then you have to be consistent and follow it up if other players are doing the same and you can end up with a whole load of cards.
I do use the public warning with players, I make sure I speak to them in a loud enough voice that others can hear what i'm saying so its clear what I will and won't stand for. That way its no great surprise to anyone if you then get a card out for a subsequent offence.
I look at it much you like deal with children, you have to be consistently clear on where the boundaries are and then not be afraid to take appropriate action when they overstep.