keep in mind that shikamaru was the one who led him there, and we don't know how far they may have traveled away from the others. he chose the place, they didn't just magically end up there