Is it correct to use "But" at the start of a sentence like this?
You can afford it. But think twice before buying it.
Won't using "But" instead of "However" mark me as a non-native speaker?
If anything it would mark you as a native speaker, because native and non-native speakers alike are taught not to do it, but native speakers grow up hearing everyone else doing so, and ignore those teachers.
It's quite likely that the condemnation of starting with a conjunction is because some children have a habit of overusing such beginnings:
And then I went to the park. And my friend Paul was there. But he had to go home for his tea. And then I was lonely so I went for a walk. And then I found a toy someone had left behind.
Such overuse is certainly not good, and banning children from ever starting a sentence with a conjunction is easier than telling them to be moderate with it, absolute prohibitions being easier to judge after all. (Though if as an adult writer you find that you are starting a very large number of sentences with conjunctions, then banning yourself from doing so might be a good exercise!)
Of course, as adult writers we should be better able to judge what is wrong with the example I gave than just proscribing all such use.
Interestingly, while you speak of however as another possibility, it is starting a sentence with however that is the more controversial. One common style-guide only changed to permit it in the last few years (alas I forget which) and many still recommend but over however in those cases where one could reasonably use either. (There was an editor of the New Yorker who was forever changing initial but into however, but he was an exception, and even he changed his ways).
Now, I'll disagree with those people and say that however is just as good as but at the start of a sentence too, but it's worth noting that the opinion does exist.
(It's also worth noting, that if you start with a conjunctive however you must followed by a comma while but would not. Other senses of however have different requirements regarding commas, see https://english.stackexchange.com/a/104877/15770 for more).
But at the beginning of a sentence is fine, and in most contexts is more natural in spoken English than though (which is a different word from thought, by the way) or however.
There used to be some teachers who had a mad idea that there was something wrong with this, but they've mostly gone now.
Yes, it is common and acceptable to start sentences with conjunctions such as "but". In fact, there is an argument that it is a better writing style than other constructions.
I would argue that "but", when it applies to a clause, cannot be anywhere else than at the start of a clause. It serves as a "clause-initial particle" or something like that.
The real rule isn't that "but" cannot be at the start of a sentence. Rather, a sentence starting with "but" should be avoided if you're starting a brand new discussion context, in which there is nothing prior that "but" can refer to. So the following makes no sense as the first sentence of a paper:
But this document summarizes the results of research into the mating habits of Madagascar rats. *
In writing, you probably want to avoid start a new section, paragraph or chapter with "but", let alone a book or paper.
The "do not start with but" rule is also intended to prevent writing like this:
I looked in the fridge. But the sandwich wasn't there.
which should be written:
I looked in the fridge, but the sandwich wasn't there.
There is no spoken difference here. Children will write it the first way, if nobody tells them otherwise.
I think, and I am probably wrong, that each sentence written should be able to stand alone, that's why it is a sentence. I can think of no situation where "but", "however" or "also" can stand alone. But I like the rain. However it's dark at night. Also I love music. To me these "sentences" are actually fragments of an idea. I am not an english major, but I would discourage the use of these words at the beginning of a sentence. If I were to read a resume, cover letter, or any other document that was submitted to me for review, I would absolutely toss it. I struggle with the fact that "captain underpants" is required reading in fourth grade, however, anything written by Hemmingway is not. It is truly sad to me as a parent that my child is learning that these things are now acceptable and proper.