For example, in a sentence like "He spoke softly, his voice barely audible over the dull roar of the ship's engine" what do you call the thing that comes after the comma? Is this even grammatical?
He spoke softly, [his voice barely audible over the dull roar of the ship's engine].
It's OK. This is an 'absolute' construction where the bracketed element is a kind of supplement that normally has a non-finite verb phrase, cf. "his voice being barely audible ...". Your example does not contain a verb and hence is the verbless analogue of this.
Notably, absolute clauses contain a subject and are thus syntactically independent of the main clause. Like other supplements, they are not integrated into the syntactic structure, but sit apart as non-constituents, typically marked off in writing by punctuation and a slight pause in speech.
Supplements are not constituents and nor are they modifiers. Instead they have a semantic 'anchor': here the anchor is the referent of "he".
It's not a sentence fragment. It's a dependent phrase.
Consider the reverse of the sentence:
His voice barely audible over the dull roar of the ship's engine, he spoke softly.
The fact that the dependent phrase in your original version comes after the independent clause doesn't change its semantic role. It's acting in an adverbial way to address the manner in which he spoke softly.
Update: I had originally used the term dependent clause, until it was pointed out to me that it had no verb. I then changed it to dependent phrase.
But apparently, there is a more specific linguistic term for this: verbless clause. Going by that terminology, it should really be a dependent verbless clause.
This nomenclature is expressed in an article called "Dependent Verbless Clause: Its Structure, Function and Use" by Jarmila Petrlíková:
The term “clause” is not only applied to structures which comply with formal prerequisites, containing a subject and a predicate conveyed by a finite verb, but also to such structures which are analysable into clause elements. The verbless clause is a structure containing no verb element at all (either finite or non-finite), usually having a covert subject, but containing other expressions which can be identified as a part of predicate (subject complement or adverbial). The verbless subordinate clause is joined to its superordinate clause syndetically or asyndetically (a supplementive verbless clause), or by the prepositions with or without. As an optional clause element, it functions as an adverbial, expressing a range of semantic roles, usually suggested by the introductory conjunction, or as an “optional subject/object adjunct” (supplementive verbless clauses), conveying a twofold relationship: to the predication and, at the same time, to the subject or object of its superordinate clause. Considered one of the means of sentence condensation, it is mainly used in written language.
I am choosing to leave my answer with the simpler dependent phrase—but with this additional note.
While dependent verbless clause may be a very specifically correct term, I find that the "fragment" (in conjunction with the independent clause) still looks like a phrase—and it meets the definitions I've seen of that word. (My original error was in not noticing that it lacked a verb.)
It's possible that it's both a phrase and a verbless clause, where the latter is a syntactical subset of the former—and, where the terminology overlaps, those who are more detailed choose to use the latter term instead.
One of the criteria for the more specific terminology is the assumption of an underlying elided be verb. This is fair. I base some of my own semantic interpretation of sentences on elision. So, it would be hypocritical of me to do so in some situations but object to it in others. (Unless I could determine firm criteria for the differences.)
Nonetheless, in terms of understanding what's going on with the sentence in question, I don't know if it's necessary to be pedantic about this particular terminology. The key point is understanding how the pieces are actually functioning.
I think that the short answer to the original question is: yes, it's grammatical.