I always use in my letter "Thanking you in advance for your time and consideration." But one of my colleagues said thanking you was not correct usage of English, it should be thank you. So my question is: can I use thanking you or should I use thank you?
I think your colleague wasn't really correcting the grammatical accuracy of your Complimentary Close (I'm assuming it's the Complimentary Close of your e-mails because otherwise it would be a bigger problem.)
Using Participial Phrases (V+ing) in the Complimentary Close lessens the total impact of the business letter:
Thanking you in advance for your time and consideration.
Assuring you of our wish to be of service to you in the future.
It's considered better style to end with a clear-cut idea:
We thank you in advance for your time and consideration.
If we can be of use to you in the future, will you let us know?
The preference for Participial Phrases came about because a letter was once considered less stiff if it ended with less emphasis and diverted from the actual subject.
Short answer: Both are correct.
"Thanking" is the present participle of thank,
English present participles: English verbs that are conjugated to indicate ongoing events at unspecified moments
So by using "thanking you", the author indicates that they are unsure of the exact moments at which the gratitude is offered (because they don't know when the reader has given up their "time and consideration"), but the gratitude is continuous. I would consider this more formal
"Thank" is the present simple. This would indicate "I thank you right now for anything you might do for me."
The participle form is more often used as a closing remark, thus:
Thanking you in advance for your ..., [I remain,]
indicating that you are anticipating some service from the adressee, while the straightforward present form would be used at the beginning (or anywhere else in the body of the letter) in the case where you are expressing thanks for a service that has already been rendered.
The present participle is generally used to indicate an ongoing activity. The present tense generally indicates a more discrete action.
"[I am] thanking you in advance . . ." suggests that the appreciative activity is beginning now and will continue until the sought for benefit is received (and perhaps long thereafter). It seems quite appropriate in that context.
If the benefit has already been received, or is a short or very discrete action (such as reading a brief missive that conveys the thanks), the terser "thank you" seems apropos.
'Thanking you' as a complimentary closing in a letter or mail seems a little awkward, because it has no subject and no main verb; it is a phrase, not a sentence. A 'Thank you' is complete and is definitely better than a , 'Thanking you'.