This question is either about etymology or language generally, as names have this feature in other languages too, but I'm just curious how the practice of naming towns in proximity to bodies of water with a built-in reference to those bodies of water came about. Obviously it's not strange to me that a town on the Hudson would be called "-on-Hudson" etc., but what "determined" or at least influenced whether a town's name ended up having a reference to a body of water as well? It's not Chicago-on-Michigan or Philadelphia-on-Delaware, so why so many "-on-Hudson" towns?
So far as I'm aware, the normal reason for adding -on-[river] is the obvious one - to avoid confusion with another place of the same name which isn't on that particular river.
But I suppose sometimes -by/-on-sea, -in-the-moor, -in-the-dale, etc., might be used to publicise the local geography of a resort to encourage tourists, etc. It could well be that in the past, when people's water came from the river rather than the tap, naming a nearby river might also be helpful in attracting visitors/settlers.
Being British, I don't know any US place-names xxxx-on-Hudson, but I could believe that mentioning that mighty river adds a touch of kudos to some otherwise unremarkable places.
It's worth pointing out that by and large the name of a place is what the locals (and more specifically, their administrative bodies) choose to call it. Each town may have its own reasons for choosing their particular form of name, and it doesn't have to conform to any particular rules. Even granting that there might be any such rules in the first place.
In Ontario it's common to see place names that end in Station, and for there to be something nearby that is the same name without Station. (An example near me is Myrtle and Myrtle Station, about a kilometre apart. Myrtle presumably came first and got its name however it did, and Myrtle Station would be the settlement at the train tracks, presumably after a stop was added.)
In the same way, there may have been a just plain Hastings - on a train line, a trail, a road or whatnot - and a companion town on the river where boats could stop. In that case, Hastings-on-Hudson seems like a fine way to distinguish the two. Whether it's true in that case or not I can't say, but it's an explanation.
One answer: It's easier to pronounce than "Weckquaesgeek"!
Better answer: There are many ways in which place names reflect the local geography, and that often includes references to bodies of water. Think of all the place names that include words like "brook," "river," "bay," "beach," "creek," "lake," "pool," "spring," and so on.
One obviously important factor in the name of a place is the origin of the people who settled that place. The earliest Europeans to explore and settle eastern New York were the Dutch, and many place names there end in "kill" (Catskill, Freshkills, Beaver Kill, Fishkill, West Kill...) which is apparently Dutch for stream or river. The ...-on-Hudson nomenclature sounds English, like Berwick-upon-Tweed or Stratford-upon-Avon, and indeed Hastings-on-Hudson was earlier called Hastings-upon-Hudson.
I don't think there's any particular characteristic that determines whether a village is named ...-on-Hudson, although it would seem necessary that the place actually be located along the Hudson river. It's just one of many ways that places use their name to distinguish themselves. Other places near Hastings-on-Hudson are Dobbs Ferry and Sleepy Hollow, so the river isn't the only feature that made its way into local names.