You don't use the present perfect in conjunction with the simple past like that; it sends conflicting signals about the time frame. To refer to an action that was already complete prior to the time you are referring to with the simple past, you need the past perfect:
I thought they had taken it off.
Given two times, "N" for now and "N-1" for some time in the past, that sentence says that you believed at time N-1 that the thing had been removed prior to time N-1.
To refer to an action that happened at (about) the same time as you are referring to with the simple past, the simple past is good:
I thought they took it off.
That says that you believed at time N-1 that the thing was being removed at approximately that same time.
With the combination in your first sentence of the simple past with the present perfect, you're sort of saying that you believed at time N-1 that the thing had been removed between time N-1 and N, which is a contradictory arrangement of times.