Always tricky situation and damned if you do damned if you don't - had a similar situation in a league match last season where the ball went out for the corner just after the added time on my watch had expired. I let them take the corner, it got knocked about a bit, then was semi cleared to an attacking team player about 20 yards outside the penalty area, when I blew for full-time. There was still uproar bizarrely.
In terms of timekeeping and communication, I will also try to inform coaches (if game is tight) at some point in the last 5 minutes of the game how much added time I currently have, and also say "plus any added from now". In terms of the constant "how long" questions, once it gets under 5 minutes to go I won't be specific, and will say "one or two minutes" and then inside the last minute or so I will only say "not that long" or something similar. I said "last play" once and it went out for a corner and could've caused me the same issue as you, so have never done that again!