I would like to know what the difference between "the key to the door" and "the the key of the door", or between "the servant to the master" and "the servant of the master" is.
English seems to have a preference for to or for when referring to an item for "operating" something. Hence:
On the other hand, 'of' is used more when you're naming a component part of something; sometimes either 'of' or 'to/for' is possible, depending on whether you're just naming a part of something or implying its intended use:
When identifying a part of something, English also permits other prepositions denoting location:
Various other languages tend to use the equivalent of 'of' in many of these cases: i.e. they have less of a distinction between naming the 'utility' vs 'existence' vs 'identification/location' of a component of something.
In English we speak of the key to the door, not the key of the door.
With respect to servants, you can perform as the servant to the master or be the servant of the master. You would use to when describing the act of performing as a servant, and of when describing the state of being a servant.