You can interpret "feared drowned" as meaning "feared to have drowned". i.e.,
Three men feared drowned = Three men are feared to have drowned.
The drowned is parasitic or dependent on the 'feared', and so feared is the main verb.
Now what is "drowned"? It appears not to be a verb, but rather a verbal adjective (i.e., participle). This is because you cannot use a full verb expression in that kind of construction. For example, you cannot say
Three men feared sold nuclear material to North Korea*
Three men are feared to have sold nuclear material to North Korea.
For this reason I think it is better to see "drowned" and its equivalents as verbal adjectives, and not proper verbs.
However, I do not think the original construction is as uncommon or as inelegant as Chris suggests in his answer. For instance the following is a perfectly good sentence:
Four girls feared dead have been found in safety in the forest.
This is much clearer and simpler than saying
Four girls that were feared to have died have been found in safety in the forest.
Finally, note that instead of 'feared', the main verb could be 'thought', 'considered', 'believed,' 'assumed,' and some other verbs of attitude.