Yes, it all boils down to hardwork in the end. You could be the smartest person in the world, yet accomplish less than an average person if you don't work hard with your abilities. Another way to look at bloodlines, limits, and geniuses is at a point scale.

Lets say an average person trained for 5 hours a day. He might improve his abilities by a hundreth (.01). I'm trying to be realistic.
A person who is considered a genius merely has the capabiltiy to understand or comprehend things easier, thus learn faster. So a genius might improve by a hundreth (.01) after just 3 or 4 hours of training. But if the person who is considered a genius only trains an hour a day he will not be as strong as the average person who trains much harder.The reason why geniuses are looked upon as strong is because they most likely know this. They know that with hardwork they can become greater than anyone else can be.

I'd also like to stress the difference between being a genius and having talent. A ninja could be horrible at taijustsu and ninjutsu, but incredible with genjutsu. That does not necessarily make him or her a genius. Like stated before, being a genius is only the ability to understand and comprehend things easier. Technically you could be an idiot but still be a great ninja because of a talent in a certain area.