Is "But" allowed in the beginning of a sentence as synonym to "Though" or "However"?
You can afford it. But think twice before buying it.
Won't using "But" instead of "However" mark me as non-native speaker?
If anything it would mark you as a native speaker, because native and non-native speakers alike are taught not to do it, but native speakers grow up hearing everyone else doing so, and ignore those teachers.
It's quite likely that the condemnation of starting with a conjunction is because some children have a habit of overusing such beginnings:
Such overuse is certainly not good, and banning children from ever starting a sentence with a conjunction is easier than telling them to be moderate with it, absolute prohibitions being easier to judge after all. (Though if as an adult writer you find that you are starting a very large number of sentences with conjunctions, then banning yourself from doing so might be a good exercise!)
Of course, as adult writers we should be better able to judge what is wrong with the example I gave than just proscribing all such use.
Interestingly, while you speak of however as another possibility, it is starting a sentence with however that is the more controversial. One common style-guide only changed to permit it in the last few years (alas I forget which) and many still recommend but over however in those cases where one could reasonably use either. (There was an editor of the New Yorker who was forever changing initial but into however, but he was an exception, and even he changed his ways).
Now, I'll disagree with those people and say that however is just as good as but at the start of a sentence too, but it's worth noting that the opinion does exist.
(It's also worth noting, that if you start with a conjunctive however you must followed by a comma while but would not. Other senses of however have different requirements regarding commas, see http://english.stackexchange.com/a/104877/15770 for more).
But at the beginning of a sentence is fine, and in most contexts is more natural in spoken English than though (which is a different word from thought, by the way) or however.
There used to be some teachers who had a mad idea that there was something wrong with this, but they've mostly gone now.
Yes, it is common and acceptable to start sentences with conjunctions such as "but". In fact, there is an argument that it is a better writing style than other constructions.
I would argue that "but", when it applies to a clause, cannot be anywhere else than at the start of a clause. It serves as a "clause-initial particle" or something like that.
The real rule isn't that "but" cannot be at the start of a sentence. Rather, a sentence starting with "but" should be avoided if you're starting a brand new discussion context, in which there is nothing prior that "but" can refer to. So the following makes no sense as the first sentence of a paper:
In writing, you probably want to avoid start a new section, paragraph or chapter with "but", let alone a book or paper.
The "do not start with but" rule is also intended to prevent writing like this:
which should be written:
There is no spoken difference here. Children will write it the first way, if nobody tells them otherwise.