As an inhabitant of India by birth, I would like to share my viewpoint with the community. Actually, IMHO, the problem arises because Asia is a vast continent (the biggest AFAIK) and therefore a different number of cultures, casts and creeds from different religions reside here. It's literally so vast and full of diversity that even from a North Indian you can easily differentiate a North-East Indian who is significantly different from a proper East Indian who in turn differs from a west Indian in terms different human traits. This is just an example of only the country India. So now consider how big, vast and diverse the whole Asia would be. So I definitely in accordance with Prash that using different terms would be more efficient to clearly make out about which part of Asia you are talking, like, Middle East, Far East, Central, South East and South Asians etc.
So technically speaking you are right in calling a Japanese or an Indian or a Saudi Arabic an Asian, but it would be not fruitful at all. Just so that you feel it, consider a French, a Spaniard and a Scott come here and we call all of them European. It would be too general to say, categorization would be more efficient here.
Now calling somebody Asian would readily mean different to an American from a British. For historical reasons (explained nicely in other answers), a British would expect an Asian to be from Indian Subcontinent (a census result is given by somebody from Britain in some comment to another answer proving a lot of people from Indian subcontinent living there) whereas an American would expect a Chinese most likely. In fact AFAIK, some Indian languages dominate parts of Britain as too many people from Indian sub-continent live there!
Also, as a side note, "Middle East" does not necessarily mean countries who have more Muslims in their population. See this:
62% of the world's Muslims live in Asia, with Indonesia, Pakistan, India and Bangladesh having the four largest Muslim populations in the world (Wikipedia)
Middle east is those countries which literally are situated in middle East.