According to the Oxford English Dictionary, the exact origins of the phrase to a T are unknown:
The original sense of T here has not been ascertained. Suggestions that it was the tee at Curling, or at Golf, or a T square, appear on investigation to be untenable; it has also been suggested that it referred to the proper completion of a t by crossing it (see 1b); or that it was the initial of a word; in reference to this it is notable that to a tittle (i.e. to a prick, dot, jot) was in use nearly a century before ‘to a T’, and in exactly the same constructions: see tittle n.
So, it is possible that the use is not referring to the use of a tee when building something. The Phrase Finder agrees that there is no accepted derivation, but offers the following:
Given Wright's earliest 'to a T' usage and the lack of evidence to support the 'tee' version, it is safe to assume the proper spelling is 'to a T'. So, what T was meant? Again, there are alternatives; 'T-shirt', or 'T-square', or some abbreviation of a word starting with T.
'T-shirt' is clearly as least 300 years too late, has no connection with the meaning of the phrase and can't be taken as a serious contender.
'T-square' has more going for it, in that a T-square is a precise drawing instrument, but also lacks any other evidence to link it to the phrase.
The first letter of a word. If this is the derivation then the word in question is very likely to be 'tittle'. A tittle is a small stroke or point in writing or printing and is now best remembered via the term jot or tittle. The best reason for believing that this is the source of the 'T' is that the phrase 'to a tittle' existed in English more than a century before 'to a T', with the same meaning.
When there isn't a definitive origin and there are several proposed derivations, the wisest course is to list the possibilities and leave it at that. In this case, although there is no smoking gun, the 'to a tittle' derivation would probably stand up in court as 'beyond reasonable doubt'.
So the OED and Phrase Finder agree that the most likely etymology is that the phrase to a T comes from to a tittle. There is also the phrase:
to a tittle, with minute exactness, to the smallest particular, to a T.