It just dawned on me what issue might be confusing. As used in the question, "that" serves as a conjunction to introduce a clause (see, for example, M-W discussion here). In that usage, it is not unusual for "that" to be followed by "the" or "there", which would be part of the clause being introduced.
In my comment on the question, I wrote, "What gave you the impression that there might be an issue?" "...there might be an issue" is the clause introduced by "that".
Where it is incorrect is to use it in a case like, "Look at that there red house." "There" is used as a "directional" to clarify what house you're referring to, which is redundant when it immediately follows "that", which is also serving to identify the house. If you require "there" to specify the house, "that" doesn't really apply; it isn't adequate to do the job of identifying what you are referring to. You would say either, "Look at that red house" or "Look at the red house, there".
A unique situation would be if you had just talked about red houses and came upon a row of them. "That" could refer to the previous mention, and you could want further specificity with "there", pointing to a specific one of the examples. In that case, you would need to separate the two so they aren't consecutive, like "Look at that red house, there".