Your sentences #1 and #2 are both correct, so long as you add a comma after "programming":
I know about programming, [but/although] I have a little bit of confusion in Java programming.
In almost all situations but and although are synonyms. Although is slightly more formal, and is preferred if you want to stress that both halves of your statement may be true. But is used when you wish to stress contradiction between the halves of the statement. For example:
She said the book was blue, but I think that it's green. [Contradiction: prefer but.]
Usually we speak in English, although we sometimes use Russian. [Both halves are true: prefer although.]
Note that this is not a firm rule, since you could use either but or although in both of the sentences above without it being wrong.
However is a different matter. However is an adverb, and it's used to introduce a new sentence in circumstances where you would otherwise use but to join the sentences. So your example #3 should be:
I know about programming. However, I have a little bit of confusion in Java programming.
(Final note: "a little bit confusion" is grammatically incorrect. You need to add the word "of": a little bit of confusion.)