The use of hyphens is to reduce ambiguity, and, in the case of multiple modifiers, to show which ones modify each other.
In the first case I would go with human-voice controlled system. In this case, however, it is pretty easy to guess that human must modify voice, even without hyphens.
In the second case, on the other hand, you need the hyphens because it's far from clear from the sentence which modifiers go together.
Software-and-finite-state machine-based approach might be correct ("software" and "finite-state" together modifying "machine-based").
Software and finite-state machine-based approach might be correct ("software" and "finite-state" both modifying "machine-based").
Software and finite-state-machine-based approach might be correct ("software" and "finite-state-machine-based" both modifying approach).
Software- and finite-state-machine-based approach might be correct ("software" and "finite-state-machine" both modifying "based") although in this case I would rewrite as software-based and finite-state-machine-based approach, because even with the hyphens, it's still ambiguous and confusing, and the dangling hyphen is ugly.
The point is, is it a finite-state machine (the machine is in a finite state)? Or a finite state-machine (the state-machine is finite)? You may know which single one of these different interpretations makes sense, but the reader may not, and the hyphens need to be placed to make it as easy as possible to figure out.