There aren't that many written instances in Google Books, so the relative ratios here might not be statistically significant, but...
Don't you walk off on me! - 11 hits
Don't you walk away on me! - 4 hits
...reflects my gut feel that the former is more common for contexts where you're admonishing someone for "leaving mid-conversation". Far more common overall is...
Don't you walk out on me! - 581 hits
...but many (almost certainly most) of those will be for the "broader" context of leaving a relationship (or at least, something less ephemeral than an ongoing conversational interaction).
There are various "slang" usages, such as cut [someone dead], blank, and idiomatic usages such as cold-shoulder, turn your back [on someone].
For a more standard/formal term you could go for ignore or synonyms, but I can't think of anything in that register that specifically implies physically absenting oneself as a way of avoiding having to listen to whatever the other person is saying.