Despite the fact that there exist many varieties of English, the vast majority of spoken media is in the General American or in the Received Pronunciation of British English. Written media is pretty dialect-free (again the great majority).
So your difficulty is probably understanding spoken language in a casual atmosphere. It could be that the people you hear come from a variety of backgrounds, but they will most likely be aspiring (if ESL speakers) to one of American or British pronunciation.
So the first step is to pick one of those. The general point is that no choice is wrong but making a choice now will solidify skills in one, to make learning the other one easier later.
As you say, movies (and TV) will teach you good pronunciation as far as accent. Yes, too casual speech (slang) might be confusing, and you don't want to learn profanity inappropriately (which is almost always). But one can often find online transcripts so that you can read along. Actors in TV and movies do tend to articulate well and in the standard. Some movies might use a 'bad' variety (mumble too much or be too casual/slangy/taboo), but most are not. Comedies might be hard because of word play, but then that would be a good learning lesson too.
News reports are good too because their pronunciation will be very articulate. Start with TV news rather than radio, because it will give semantic context with the pictures, which will help disambiguate unsure vocabulary.
A growing, very easy resource is youtube videos. A lot of it is amateur, meaning, not produced with expectations of high quality and experienced actors. It will be more casual and conversational, more likely to have dialect/pronunciation variations, but the real challenge is the natural slurring, mumbling, rushed lack of distinct articulation. I'd suggest sticking with the more professionally produced youtube videos before graduating to the real-life speech of the amateur (which is eventually what you really want to learn).
But frankly, don't leave something out if its easy to do. That is, listen, listen, listen. (Talking too is very important, in fact terribly important, the more you try to talk the more practice in your head you'll be doing and you'll be able to understand more that you hear)