If I say "I am annoyed", is that in the passive voice?
It depends on context — annoyed can function in two different ways, either as a participle or as an adjective. (Or, arguably, there’s a spectrum between those two, along which it can vary a fair bit.)
If you mean Something annoys me, then yes — annoyed is functioning as a participle, and it’s the passive voice. This is parallel to a case like Help! I am attacked! A clear diagnostic test showing that it could be passive is that you can add by to incorporate an agent: I am annoyed by the self-reference in this sentence.
If you mean The way I feel is: annoyed, then no — it’s not the passive voice. Like many participles, annoyed has made the transition to sometimes acting as an adjective. The standard diagnostic test for this is that you can talk about, for instance, a very annoyed pedant. (By contrast, you can’t talk about *a very attacked castle.)
Some participles have gone further, and changed almost entirely into adjectives. If I say My computer was broken, I’m probably describing a state my computer was in, not the act of something breaking it. In extreme cases, the original verb has been lost entirely: you can be renowned, but you can’t *renown anything any more. So Lynne Truss is renowned for… is not the passive voice.
Since there are currently two answers, one "yes" and one "no", and both are correct, I'll try to add some explanation :)
The passive is formed using the verb "to be" plus a past participle; "to annoy" is a verb, so the sentence
is of the right form to be a passive, and as kiamlaluno notes may be extended with "by" to show the agent who is causing the annoyance.
However, most speakers would consider that this sentence is instead of the same form as
or, in other words, that "annoyed" is being used as an adjective rather than a participle. To illustrate the difference consider the sentences:
There is no doubt that one can be "seen" or "hit" to a great extent, and yet adding "very" does not work with those, which function only as participles and not as modifiable adjectives.
uses the passive voice.
To use the active voice, you have to have an agent for the annoyance.
uses active voice.
Short answer: if you can replace "am" by "get" or add the word "generally" to the sentence, then it's probably a passive; if you can replace it with "seem", then it's probably not.
The longer answer is that the passive isn't just any case of be followed by a past participle: it also has certain constraints:
In principle, the form "I am annoyed" can have a passive and a non-passive reading. The passive reading would imply a general action ("am" in the simple present can be stative, but as in the river example, passives tend not to have stative interpretations, so only the general interpretation of the simple present is left). And as a passive, 'get' can be subtituted:
On the other hand, if the sentence isn't a "true" passive, then it can have a stative interpretation (or put another way, mean "I am annoyed now" rather than stating a generality). In this case, replacing 'am' with 'get' would change the interpretation, but verbs like 'seem', 'appear' would not change the sentence from a present observation to a generality:
(Notice how changing to "I get annoyed" changes it to a generality.)
On a related note, notice how with verbs denoting an obvious action, to get a present passive interpretation (rather than being interpreted as a generality), the continuous form is used, so that the following sentence is not a contradiction:
It is the passive voice in I am annoyed by you, where the active voice is you annoy me.
I am annoyed is not the passive voice, as annoyed is an adjective.