salam
thats the same point i was making. The prophet (saw) was not pointing to a specific spot but rather the sky and the "heavens" above it as a generality
if somebody asks you to point to where the international space station is, or the moon, or some other structure that exists beyond the sky, and at that point in time it was orbitting the opposite side of the earth, you would have to point downwards at the ground to be accurate.
ws