If I say : "It is hard to choose which to eat between a candy and a chocolate." Or
"If not stealing my stuff, I don't care about him acting like a thief."
It sounds so unnatural. Is there a grammar problem? Or if there isn't any, why does it sound unnatural?
The first sounds wrong because of the attraction of the infinitive "to eat" to the prepositional phrase "between a candy and a chocolate." "To eat between" makes little sense. The phrase goes with the previous infinitive, with the preposition's objects as antecedents to "which," the object of the infinitive "to choose."
Between a candy and chocolate, it is hard to choose which to eat.
The second sounds wrong because you have elided the subject of the subordinate "if" clause, which your reader will take as a nominative absolute attached to the pronoun following, namely "I." But you're not stealing from yourself; he's doing the stealing. This may be remedied by mentioning the thief:
If he is not stealing my stuff, I don't care about him acting like a thief.
Better:
It is hard to choose between a candy and a chocolate.
OR
It is hard to choose between candy and chocolate.
(It is clear from the context that once you choose one, you're going to eat it.)
As long as he's not stealing my stuff, I don't care about him acting like a thief.
OR
If it weren't for stealing my stuff, I wouldn't care about him acting like a thief.
Now, the sentences you proposed in the comment:
They thought about which to eat between candy and chocolate.
I don't care about him acting like a thief, if not stealing my stuff.
Better:
They thought about whether to eat candy or chocolate.
I don't care about him acting like a thief, as long as he's not stealing my stuff.
