I have once removed myself from the list for an u13s national final, because one team was a local school and I had coached half of the lads the season before.
Ordinarily I wouldn't though. Coaching last season, the home team had to provide the ref for junior games, and often it was a brother or father of one of the players. I got my players to ALWAYS call them 'Ref' or 'Sir' during the game, even if it was their Dad. Whoever was reffing us did the same to them, 'player' or '#4'. More than anything it gives the impression of impartiality and 'learns the kids' how to approach officials.
I also ref my wife's team regularly, and once I had a coach huffing about 'bet your wife gets all the decisions', so I just said, "look, after the game, if you can point my wife out to me I'll buy you two beers". He got three wrong before pointing at the missus lol!
If you feel really uncomfortable with it, then talk to whoever gives you the appointments, I'm sure it's not the first time they have had the situation and there may well be a recognised policy.
Otherwise, talk to the players the way you would normally, call by number etc. You set the tone and they should follow.