Short answer: No, it is not.
Long answer: "Avail myself of" means to "take advantage of" or "use". Some example usages would be:
- Please avail yourself of the resources available from our help center.
- I availed myself of the opportunity to speak to her while she was in town.
I think it's almost always used with words like "opportunity", "resource", or "chance": something that is not yet taken, but can be taken, and therefore you should take advantage of it.
(This seems slightly different to me from the actual phrase "take advantage of", because you can take advantage of "the fact" or "the time" or "the situation" and the time/timeline of events is not really as important.)
You might still be able use it in your response somehow, depending on what you mean to say, but the construction given sounds very odd. I (again) think it's partly because "not being in office" doesn't really sound like something especially advantageous or disadvantageous but mostly because it's already a fact, not a future chance.
Do you mean "I do not want to reschedule"? When do you want to get this done--on January 4th (when you're out) or on another date? Are you saying you want to do it over TeamViewer while you're out instead of rescheduling? If the last one, then I don't see a way for you to use "avail myself of" in that form of the sentence. A way to rewrite it would be
Thanks for rescheduling, but to be honest I’d rather take advantage of not being in office on January 4th to get this done without disturbance.
But Jim's suggestion is much better than that one because it makes clear all the details about the date and your reasoning. Nonnal's allows you to keep "avail...", but requires a little shuffling about of the sentence.
Related question: To avail oneself of an opportunity