I am always confused when I get an email stating "out of office until Thursday". Is the sender back on Thursday or still out of office (o.o.o.) on Thursday and only back on Friday? Is there a good reference that defines the meaning? Please do not tell me that it would be better to state "back on Thursday", since this does not answer the question.
Until x means that the change occurs on/at x:
I am here until Friday. | From Friday I will not be here.
I am playing football until 7pm. | At 7pm I finish playing football.
The store is closed until March. | In March the store opens.
In your example:
I am out of the office until Thursday. | From Thursday I am back in the office.