Definitely not too old.
I didn't pick up a whistle until I was 47, exactly like you. I'm in to my 7th season and just got my level 3 in the mid-season promotions - that includes 2 Covid curtailed seasons with no promotion opportunity, so you can get to a decent standard reasonably quickly if it goes well and you have a bit of luck along the way.
You do need to be fit and I have to work harder at that than younger colleagues. But if you have a background playing, you will also have knowledge of the game and how to work with some of the characters you find in it, that younger referees have to learn. Incidentally, that will probably mean you have to learn how to think like a referee who understands players, rather than refereeing thinking like a player. That's a whole other subject and links to my final point of encouragement which is that at an age where you can't realistically get better as a player, there is still masses you can learn as a referee, whilst being involved right at the heart of the game.
In short you are not too old - go for it !