EDIT: I have received a lot of commentary on this, enough legitimate commentary that I decided to take a closer look at my answer. The results were mildly surprising to me. See below.
Either one has the right meaning and does not really carry bad connotations in my own intuition, although others have commented that questioner connotes an interrogation and that they much preferred asker.
I found this unusual, since questioner is by far the more common word in general English use:
A Google Ngram query for 'asker, questioner' reveals that 'questioner' is found overwhelmingly more often than 'asker'. In addition to the Google Ngram, a query against the BYU Corpus of Contemporary American English gives the following relative rankings:
WORD 1 (W1): ASKER (0.05)
WORD 2 (W2): QUESTIONER (19.53)
While the British National Corpus query gives:
WORD 1 (W1): ASKER (0.01) NONE
WORD 2 (W2): QUESTIONER (90.00)
At this point, I would've recommended you stick with questioner in most cases. And that answer was accepted, but it was not popular, and I wondered why. Dictionaries and thesauri did not show any evidence that either was more or less used, or had any particular connotation. I investigated whether it was a UK-US difference, but found no evidence of that either; the commenters did not have a geographic bias that I could discern.
Then I realized I had the perfect corpus right at hand: Stack Exchange itself! So I ran a search query against all SE sites. And to my surprise:
site:stackexchange.com asker About 24,600 results
site:stackexchange.com questioner About 3,260 results
And there you have it. In general English usage across all contexts, questioner is favored. But in Internet-based interactive context like that found on this -- a question and answer site -- asker is used more often by far.
Why is this so? I suspect that is because a query limited to StackExchange also limited the context to a more neutral or descriptive use. Alternatively, it may be that SE users come from a subset of the population -- generational? gender? education? -- that favors asker as more neutral.
Since you specifically say, "The context can be taken as referring to the people who ask questions on this site" then I must say that, contra my earlier thoughts, asker is the answer.