The OED defines parenthesis as:
1.a An explanatory or qualifying word, clause, or sentence inserted into a passage with which it has not necessarily any grammatical connexion, and from which it is usually marked off by round or square brackets, dashes, or commas.
1659 in Burton's Diary (1828) IV. 283 You see the inconveniency of a long parenthesis; we have forgot the sense that went before. 1762 Sterne Tr. Shandy V. xvi, The phenomenon had not been worth a parenthesis.
This is the rethorical meaning of the word
parenthesis. Because you often put a parenthesis between round brackets, the term is also used to indicate the bracket symbol
(). However, a parenthesis need not necessarily to be put in parentheses!
Interestingly, the OED also lists the (obsolete) meaning of digression
†1.b A passage introduced into a context with which it has no connexion; a digression. Obs.
1600 Heywood 1st Pt. Edw. IV Wks. 1874 I. 29 Away with this parenthesis of words. 1654 Gataker Disc. Apol. 4 But let this go for a Parenthesis; return we to our task. 1757 H. Walpole Lett. H. Mann 5 May (1846) III. 288, I thought you would prefer this parenthesis of politics.
So, it is clear that the expression "to open a parenthesis" is used to say: "I will now start to talk about something not completely related for a while, then come back to my previous subject".
As a side note: in Italian and French you can use aprire una parentesi or ouvrir une parenthèse (to open a parenthesis) as an idiomatic expression to indicate a digression.