I believe the following are reasonable guidelines (although there are numerous exceptions, like cuckoo).
For words that came from Middle English, in standard British English you pronounce long u as "oo" if it follows an "l", "r", "sh", "ch", or "j" sound; and "yoo" otherwise.
In standard American, you can pronounce long u as "oo" if it follows an "l", "r", "sh" "ch", "j", "n", "t", "d", "s", "z" or "th" sound; and "yoo" otherwise.
This phenomenon is called "yod dropping" and exactly what consonants trigger it varies widely with the specific dialect of English (and maybe even the speaker).
See Wikipedia.
For foreign words, like kudos, sushi, and puma, all bets are off. All of these were pronounced "oo" in their original languages, but English speakers may (fairly randomly) decide to pronounce some of them with the spelling pronunciation.