As @Mick points out, seldomly can be found in some dictionaries.
However, I have not found any in those advanced learner dictionaries of contemporary English that I have checked.
The Collins dictionary offers a contradictory picture, saying on the one hand that it is obsolete while on the other showing a graph indicating that it may be coming back into use (note that in recent years the line goes up and down).
A search for seldom of the massive EnTenTen 2013 web corpus in Sketch Engine yields 172,000 or so occurrences, compared to 733 cases of seldomly. In my judgement, while comparatively infrequent, seldomly cannot therefore be said to be "unacceptable." However, because it is relatively rare and recent it is likely to invite criticism or even total disapproval from many quarters.
The fact that the antonym often does not have an -ly form has no doubt contributed to the erstwhile and partial abandonment of seldomly.
The synonyms rarely and frequently have the corresponding adjectival forms rare and frequent, so the -ly suffix is necessary to distinguish them from each other.
In the case of seldom and often there are no corresponding adjectival forms, a fact which renders an -ly form redundant. I have been assuming that when a language form is redundant it is likely to fall into disuse, but this assumption seems to be challenged somewhat by the tenuous re-emergence of seldomly.