Native speakers! just one question about english grammar?

Down and up often mean the same thing in that context. They imply a difference in the speaker's perspective but don't reveal anything about the physical world. You can hear both "down the street" and "up the street". They both mean you have to GO somewhere, but whether it's down or up is totally arbitrary.