The difference is in the type of parenthesis employed. In both variants, the parenthesis is a pragmatic marker adding (syntactically) non-essential information, further to the matrix sentence.
The first variant is a complete sentence (whose punctuation, as we have seen in comments, must be modified):
Loops — a loop is an arc that pairs a vertex to itself — are not allowed here. or
Loops (a loop is an arc that pairs a vertex to itself) are not allowed here.
The second variant is an apposition, syntactically equal to the subject of the matrix sentence, and hence needing grammatical agreement:
Loops, arcs that pair a vertex to itself, are not allowed here.
Here, commas as well as parentheses or dashes may be used to offset the apposition; notice that either Loops or arcs that pair a vertex to itself may be dropped without impairing syntax; the remaining noun phrase can fulfil the role of subject quite adequately on its own. One could switch the order of 'loops' and 'arcs that pair a vertex to itself', but then the commas would not be a good option.