In France we have specific classes called "Classes Préparatoires" for high level Education in Maths and Physics. You go there in order to prepare for Top level "Engineer Schools". Been there, entered one of those schools.
Thing is, the difficulty are those "Classes Préparatoires" and of course being selected after that, but the selection is a small -yet extremely intense- period of time.

There, I had topology, and all sorts of exotic maths I didn't even know exist. I remember being astonished we even retraced all the process for math research dating something less than 20 years.
Same in Physics.
Honestly, it was Hell on earth. Literally.

And now, 15 years after that, I think I forgot almost everything I learned back then. Maybe I still have some thought process and the like, maybe reading maths and physics vulgarization papers is easier (I still do that everyday after all). But that's it.
For some reason, I often feel like I wasted those years, yet at the same time I often understand things far ahead of everyone, understand why things are as they are... but it does not give me the edge in anything. Maybe because I clearly lack other qualities and I'm not at the right place right now who knows...

What I want to tell, other than speaking about myself, is that no matter what you do, it might not be as important as you think it is in the long run. Yet, it might be important to experience them. I feel like I'm paraphrasing Gandhi here (I really think I did that... spank me )

My understanding of society is that if you have qualities you need to work on, they mostly are social... How can you have people do what you want them to do for you... and they do it like it's natural and they even are happy doing so.... for example.