I want to write phrase "Capital of A is B, or I'm mistaken?", but I'm not sure that the second part of thus phrase is correct. Should I write "... or I've made a mistake?", or "... or I'm wrong?"? What is the best?
As the question stands, the OP asks if (forms of) mistake can be used as other than a noun (Should I write "... or I've made a mistake?").
In that sense, the "Capital of A is B, or I'm mistaken?" is correct in the use of mistaken as an adjective.
Incidentally, minor errors of grammar need correction:
What you’re looking for is “...if I’m not mistaken,” but it's rather a cliché. “...if I remember correctly” is not much better. “...I believe” is a bit more honest.
Expressing a little doubt in these clichéd expressions actually indicates certainty: you know you’re not mistaken, and it’s a false modesty.