In the example you gave, "I was not joking" is a subordinate clause. One way to think about this is that there are two sentences

1. He will understand X.
2. X = I was not joking.

"that" is not exactly a conjunction as Vincent said. A conjunction joins two independent clauses together, but here 2. is sort of the object of 1. Any time you want to use a clause as the object of a sentence, you can optionally precede it with "that." However, if you want to use a clause as the subject of a sentence, it will always be preceded by "that"

That I was not joking will be clear to him.

 

*I was not joking will be clear to him.

Sometimes clauses modify nouns. In these cases, they're called relative clauses, and what pronoun appears before the clause varies. One of the most important factors of whether or not "that" or "which/who" precedes the clause is whether the clause is "restrictive". To modify somee examples from Wikipedia:

(Restrictive) The house that Jack built fell down.

 

(Restrictive) The house __ Jack built fell down.

 

(Non-restrictive) The house, which we all thought was in fine condition, fell down.

In the first two examples, the relative clause picked out a particular house from all possible houses. In these sentences, there is a tendency to use "that" or nothing. In the third example, the particular house was already determined, and the relative clause simply introduced extra information. These relative clauses tend to be preceded by a wh- word.

In the example you gave, "I was not joking" is a subordinate clause. One way to think about this is that there are two sentences

1. He will understand X.
2. X = I was not joking.

"that" is not exactly a conjunction as Vincent said. A conjunction joins two independent clauses together, but here 2. is sort of the object of 1. Any time you want to use a clause as the object of a sentence, you can optionally precede it with "that." However, if you want to use a clause as the subject of a sentence, it will always be preceded by "that"

That I was not joking will be clear to him.

*I was not joking will be clear to him.

Sometimes clauses modify nouns. In these cases, they're called relative clauses, and what pronoun appears before the clause varies. One of the most important factors of whether or not "that" or "which/who" precedes the clause is whether the clause is "restrictive". To modify somee examples from Wikipedia:

(Restrictive) The house that Jack built fell down.

(Restrictive) The house __ Jack built fell down.

(Non-restrictive) The house, which we all thought was in fine condition, fell down.

In the first two examples, the relative clause picked out a particular house from all possible houses. In these sentences, there is a tendency to use "that" or nothing. In the third example, the particular house was already determined, and the relative clause simply introduced extra information. These relative clauses tend to be preceded by a wh- word.