The vowel y has three predominant sounds, and they mimic the long and short sounds of the vowel i, and the long sound of the vowel e.
Examples of the long i sound: cry, sty, dye, type, pylon, hyphen, cycle, hyperbole, xylophone.
Examples of the short i sound: gym, hymn, cynic, lynx, crystal, typical, syllable, homonym.
When the y is pronounced with the long e sound, you typically find that at the end of a word, or the end of a prefix to a word.
Examples of the long e sound: happy, bevy, candy, dizzy, polygraph, and almost any word ending with the suffix -ly.
John is correct when he says that the pronunciation rules are too complex to summarize easily. One good example is the word cycle, which has the long i sound – but, for some reason, after adding a prefix (i.e., bicycle, tricycle) the y is typically pronounced with the short i sound!
And Malvolio's point is well-taken, too. I'm classifying these according to what you'd find in the pronunciation guide of a dictionary. What you'd actually hear might vary according to regional and local accents. Remember:
"There is no 'correct' pronunciation of anything." (Barrie England)