Dossier (“a file containing detailed records on a particular person or
subject”) comes to English directly from the French, in which language
it referred to a compendium of files, arranged in folders, each one
with its name written on the spine, or back.
Is this sentence correct?
Assuming you mean grammatically correct, yes, it is. Whether or not it's correct in content, I imagine it is, but I can't attest to that fact.
I do not understand the clause beginning after the first comma.
It means: In French, it referred to a compendium of files, which were arranged in folders with each folder having its name written on the spine, or back.
You said, "taking language as the subject thereof," but if that is what you took from it, then you're incorrect. The subject thereof is "it." "It" refers to the antecedent "dossier."
You said, "...this is not a complete clause," but it is a complete clause. "It" is the subject. "Referred" is the verb. "To a compendium of files" and "in which language" are modifying prepositional phrases.
You said, "I see that French refers to the people," but it doesn't. The phrase "comes from the" followed by a language name is an idiomatic of saying the word derives from a word in that language.
Consider the following examples from various published works:
In the above, it becomes clear that "comes from the French" means that the word derives from French, that the words "the French" in that idiomatic expression refer to the language, not the people. It becomes especially clear in the last example because "Old French" and "Old Norse" are languages, not people. There is no people called "the Old French," no people called "the Old Norse," only languages called those things.
Hopefully, with these clarifications, the quote and its clause after the first comma now make sense to you and you now do understand it as it is.