I don't think you would need to go back to the previous restart. Logically, the player is just being shown the caution later than he should have been - that doesn't mean the restart was incorrect.
Imagine if R missed the flag, the goalkick is taken, and play continues until a foul is...