Are you puzzled by the construct "X or Xes" in general, or by its use in this particular case?
The plural is often taken to include the singular when the number is unspecified. Like if you say, "How many people are in the room?", it is not considered at all strange for someone to reply, "One."
But it's common to list both a singular and a plural when we want to be clear that there might be only one, if someone might otherwise be confused into thinking that there must be more than one. Like, "Find the screw or screws that secure the face plate to the device." This makes clear there might be only one screw; someone might otherwise think you are implying that there are two or more.
I have no idea whether a reliquary typically holds a single relic or many. Apparently the writer of the dictionary entry wants to make clear that it could be either case.