Here is the original sentence: So, I decided to walk only at night until I was far from the town.
A teacher said this sentence can be changed into the following one : Only at night, did I dicide to walk until I was far from the town.
I know subject-auxiliary verb should be inverted when a sentence starts with only in the beginning. But, I think here, the phrase, only at night, is a part of to-infinitive chunk, to walk only at night. It itself is not independent adverbial phrase and thus I think it can not be fronted. If it has to be fronted, I think the meaning changes a lot. It is not to walk only at night what you decided, but it is only at night that you made a decision to walk.
It would be really great if you could give me your opinion :) Thank you :)