@Hazzy this sounds awful. From what I understand, reffing is more of a vocation, a passion, rather than about the money, so if things don't improve then it might be best to give up on it. I certainly wouldn't want to take relentless abuse for anything, whether paid or not.
Hope the situation...