As a native American English speaker, I can definitely hear the difference between /o/~/wo/, /u/~/wu/, etc. I think most native English speakers can.
As for learning to distinguish them better, looking at the speaker's mouth while they talk may help. The lips are closer together and more rounded for /w/ than for /o/ and /u/, so when a person pronounces /wo/ and /wu/ you should see their lips begin close together and very rounded, then separate somewhat. Paying attention to the visual difference could help you pick up on the auditory difference. (This should also be easier to do with /wo/ than /wu/, since /w/ is more acoustically similar to /u/ than to /o/).