My question does not have to do with the correctness/incorrectness of 'neither do I'/'me neither', but with the presence of the 'yes' (or 'yeah', which is how it most often 'comes out' for me) at the beginning.
If someone were to say:
I love chocolate.
I'd naturally answer (orally) one of the two:
Yeah, so do I. / Yeah, me too.
But if someone were to say:
I don't like driving.
Would it be normal to answer:
Yeah, neither do I. / Yeah, me neither.
Or with it be best to say:
No, neither do I. / No, me neither.
I do not understand why my question was marked as a duplicate. My question is not about the personal pronoun (I or me) but about the adverbs (Yes/Yeah or No). I do state in the first paragraph that my question concerns "the presence of the 'yes' (...) at the beginning.", whereas the question I'm supposedly 'duplicating' is clearly focused on the personal pronoun.
I shall edit to make the words I'm concerned with in bold to see if that clarifies that I'm not interested in the pronouns.