My question is that in the sentence:
The thief opened the door with the duplicate key.
Why can't I use by in place of with?
Doubtless as a result of historical changes in meaning and ellipsis, the meanings of prepositions in specific modern-day expressions are legion, often etically unpredictable, and not infrequently apparently illogical. There are broad rules, but they are very broad, and exceptions are many. Avoidance of ambiguity is desirable, but not always achieved: even 'The thief opened the door with the duplicate key.' is ambiguous (instrument (= using), or identification (= which had) of the door?)
David Thatcher, in Saving our Prepositions, writes [re-formatted]:
Greenbaum (103) cites “an empty aspirin bottle was found by the deceased.” This, he says, “sounds as though the dead person found the bottle rather than, as was presumably meant, that the bottle was found beside him.”
The art section of my local newspaper ran the headline, “the world of ballet has been blessed by many fine composers,” suggesting that composers, en masse, have been usurping a priestly prerogative. By, of course, should have been with. The broad distinction is that by denotes the agent, or essential agent, of an action, and with the instrument of an action. Compare “he was struck by the sun” with “the sun struck with its rays, “the tree was shaken by the wind” with “the wind shook the tree with its strong hands, “”the city was destroyed by fire” with “he destroyed the city with fire” (examples cited by Fernald 189).
In practice, by and with are used less strictly, but “where with or at can reasonably be used instead of by, they should be” (Greenbaum 103).
That the distinction is not universally made is shown by examples such as
We must do it by long division. [internet]
The secret doesn't lie on whether you made it by machine or by hand but on the embroidery supplies you use to craft it. [internet]
both showing instrumentality.
But in OP's example, 'with' is the accepted choice. This is probably strongly connected with the fact that 'key' is concrete whereas 'by hand', 'by long division' ... show methods (long division being abstract, and hand in this expression intermediate).
As a general rule of thumb that I use with my students, we use with to show the instrument itself. It must be being used as an instrument operated by an actor in the sentence. It is nearly always a noun:
We use by to show a method by which something was achieved. We can use a gerund participle clause or a noun:
And of course we use also by to indicate the agent of an action. Tha action may be implied:
Of course, this is just a rule of thumb. There are many exceptions as well as grey areas. For example, sometimes we may use a noun to indicate either a general method or an instrument:
Who said that you should use the preposition by?
According to Cambridge Dictionary Online, the meanings of the prepositions are the following:
with - "using something (method, instrument)", e. g. Fix the two pieces together with glue.
There is no any meaning of the preposition by to be possible in your case. The possible choice could be:
by - "used to show the person or thing that does something (agent)", e. g. The book was translated by a well-known author.
If by is used, there is the meaning that "duplicate key" is a person which opened the door but actually the thief opened it.