I'm also confused in many cases by how to use prepositions. I guess a good way to understand these subtle differences, as in your example, is to read more. At some point you will naturally feel the differences like native speakers do. At least that's what I'm trying to do. I don't think most native speakers actually pay attention to the explanation of these differences, they just feel it right or wrong.
As for your example, I think "for" sounds better. You could also add "to" after it, like:
It is difficult for me to understand
the difference between "to" and "for".
"to" only makes sense if you say:
It sounds difficult to me.
When to use "to" and when "for"? has some good tips.
Just my two cents. Take my suggestions cautiously, since English is my second language as well. :)