The provided information allows one to say of the objects which neighbors which.
The sentence above completely skirts the singular-plural issue. It handles both because the relationship is a reciprocal one.
The provided information allows one to say of the objects which is neighbors with which.
That sentence has a singular verb too but to get caught up in that would be silly in the given context: the cumulative effect is that all neighbor relationships can be established.
The discussion below is for people, not things, but will support the second example I've given above.
who is neighbor of who? is idiomatic colloquial American English.
Your version with the plural is marginal, to my ear: Who are the neighbors of who? Why? Not because the number agreement is ungrammatical, not at all, as it isn't, but because that's not how the question is asked in colloquial contexts. It "mixes registers".
On the singular/plural issue... Each "who" may have multiple neighbors, true, but the singular version I gave above doesn't preclude that possibility, and it is far more likely to come out of the mouths of native American English speakers speaking casually than your plural version.
At this level of colloquialism, even the verb is can refer notionally to more than one person especially if they can be subsumed under a role. (In Who is neighbor of who? the word neighbor without article is being used as a role.) We could be talking, say, about couples living in a neighborhood, and someone listening in and trying to keep straight in their mind which couples are neighbors of which, might ask "Who is neighbor of who?" or even "Who is neighbors of who?"
An example of what may seem to be the chimerical nature of number agreement in this particular colloquial context with neighbors is the following:
In this article, the numerical order is used to determine which numbers are neighbors. As an example, the number 3 is neighbors of 2 and 4.
But let's go one step further and make it a circle, 0-9 and back to 0 (01234567890123...). With that, the number 9 is neighbors of both 8 and 0. And 0 is neighbors of both 9 and 1.
[my emphasis]
But there, neighbors is not so much a plural noun as it is a predicative complement that means "being in the role of neighbor".
(Compare "is partners with" here.)
And so a perfectly idiomatic normal-register version of your question that accommodates a notionally singular or plural subject is:
Who is neighbors with who?
with this version being in formal register:
Who is neighbors with whom?
NOTE: I said with who(m) there since that preposition is in the linguistic thal-weg, so to speak, of this predicative use of neighbors.
Note: That predicative use is neighbors is not an Americanism.
There are people who will see of who as wrong, and think whom is the correct choice. I can't tell you what to do about that issue. whom is what educated people would probably write when trying to be formal and correct, especially if they don't want to risk being tagged as uneducated. Moreover, even if the author were to have written "with who", in many journals the editorial staff would red-pencil it and replace it with "with whom". Written attestations are to some extent artificial in that they reflect editorial preference not actual usage.
Here's an illuminating contrast:
Who's kidding who? (from Creative Pastors)
Who's kidding whom? (from National Library of Medicine)
The irony there is that the author (editor?) of the article in the National Library of Medicine is using the "correct" whom, which has been falling out of use for centuries and is largely moribund in colloquial speech, in a colloquial phrase, violating the "register" . It's like saying "Who's punking whom?" or "Who's ghosting whom?"