It's all gags. Magic is so useful that its use should have been becoming more and more common over the centuries, not more rare. It's like suggesting that, in RL, most of humanity would gradually forget how to use electricity, while only a small minority would still keep using it. That's just not going to happen. With magic, the only plausible explanation for such would be that there's genetic factor in being able to become a magician, and it's recessive. That just might result in a situation where only pure-blood families would maintain it, otherwise random lucky individuals might be able to, but possibly they would never realise it. Though you'd think at the dawn of the time it would have been so über beneficial that it would have become extremely common, thanks to population genetics.
There have been some "magic goes away" stories over the years, but that doesn't seem to be the case in this series. Magic is as strong as ever. Awfully oppressive religion doesn't explain it either, since religion in this work doesn't hunt down all and any magicians. There simply are no explanations given to why pretty much everybody is not trying to rely on it. So, it's all artificially forced for the sake of the story.