When I first got onto the Conference list I thought that 2.5 to 3 hours before kick off was massively overkill, but in most cases it passes in no time at all. You have safety briefings to go through, and in some cases police briefings if there is a big crowd expected. There's the pre-match instructions, interaction with the observer, the referee often has their coach there, briefing with the captain and member of coaching team, etc. The only time I found it to be a problems was the few occasions where I didn't get on with the referee, but thankfully there were only a couple of those. Often you'd be listening to people like Dermot Gallagher, Paul Taylor, Jarnail Singh, Steve Bennett, recount stories from their refereeing careers and this passed the time really quickly.
At level 4, if you are in the middle you decide when to get there but it will be a minimum of an hour before. Personally I would always aim for 90 minutes before as that way if there are problems you have a bit of contingency. The difficulty here is the assistants aren't "in the system", so you can ask them to turn up early but they can effectively say no as long as they are there an hour before.
When you are on the line as a 4 the rule is still an hour before, but if the referee asks you to be there earlier you are expected to do so. I once had an assistant say he could only get there bang on an hour before, and once he'd told me where he was coming from and when he was leaving I got him taken off the game. There was zero contingency, and the slightest delay would have made him late, and anyone living in the South East of England knows that the slightest of delays is almost inevitable. The general expectation on contrib games is anything between 75 and 120 minutes before kick off in my experience.