If you've not blown, you haven't stopped the game. In which case I'm giving the foul and the DFK (and probably a card). I'd justify to myself on the basis of penalising the most serious simultaneous offence. You probably are going with expectation here, but there will be some grief from the defenders so I might have a word with the captain to explain that the foul takes precedence.
If I have blown, that's more tricky - a bit like a foul after the ball has gone out of play (as discussed on a recent thread). I'm giving the card in this case (as it must have been late from your description) but the restart will be the IDFK for the offside. Tougher to manage though - and I'd probably speak to both captains if/when the outcry started.