There is a fascinating site for dialect maps made by one Joshua Katz of NC State University, based on a linguistic survey by the University of Cambridge. It shows variations in dialects across the USA in a variety of topics, one of them is this "do you want to come with" that seems to annoy you.
Looking at this map, you can see that "coming with" is distincly common in Minnesota and the surrounding states, but rather rare elsewhere. I suggest following the link above and choosing Question 51 in the drop-down menu to see a full-sized map, as well as geographical distribution of specific answers to this question.
Such distinct usage in a specific dialect strongly hints that while it's not a formal usage that would pass copy editing for a paper or national newspaper, it's certainly not wrong for the dialects it's used in, and it's not an individual idiosyncracy of your coworker trying to annoy you. :)