Explanation

 Parentheses resemble two curved vertical lines: ( ).

A single one of these punctuation marks is called a parenthesis.