As discussed in other answers, "worm" probably means a line of prisoners. I've occasionally seen "snake" used to mean "queue" in older writing, and "worm" makes sense as a way to describe a slow-moving line of people.
However, I disagree with the suggestion in other answers that "yellow" refers to prison uniforms. As best I can tell, the prisoners in this story would most likely have been wearing black-and-white stripes.
The story is set in the Bowery district of Manhattan. Given that it was published in 1893, and the passage in question describes a childhood incident for characters who later appear as adults, I'd guess that it's set somewhere around 1870.
Here's a coloured woodcut of a prison gang at Blackwell's Island (now Roosevelt Island) from the 1870s (source):
I am not sure whether "the Island" in Crane's story was Blackwell's, but it's the closest island prison that I'm aware of. (NY's famous prison at Rikers Island wasn't built until well after this story was published.)
You can see how a line of prisoners packed so closely together might resemble a "worm" when seen from a distance, and you can also see that the prisoners' uniform was striped black and white(ish). As far as I can tell, this was the standard in New York at the time; per Slate, black-and-white stripes were used in NY until 1904 when they were replaced by plain grey.
(Also, "yellow" is applied directly to the convicts - if it were referring to their uniforms, something like "yellow-clad" would be more natural.)
So what does "yellow" mean here? The word had connotations of "cowardice" at the time, but in context that doesn't seem likely - the prisoners aren't close enough to get that personal.
For lack of alternatives, I'd assume it's referring to skin colour. The terms "yellow" and "high yellow" were in common use at the time as a way of describing light-skinned people of mixed black/white ancestry.
However... alcoholism is a major theme of the story. One of the common effects of heavy drinking is liver damage, which in turn leads to jaundice, i.e. yellowing of the skin. Jaundice can also be caused by yellow fever, and there were outbreaks of yellow fever in NY up to around 1856 at least. So jaundice would be my best guess here.