I think with careful speakers/writers, explicitly structurally adjectival magical is usually reserved for metaphoric usages, whereas magic tends to be more literally to do with the "supernatural". So if I were considering the utterances of a careful speaker, I would expect this distinction...
1: "You should read this magic book" (it's a book about magic)
2: "You should read this magical book" (reading it will be a metaphorically enchanting experience)
Note that where the referent is something that often occurs with the "noun-used-as-adjective" form (such as magic trick, lamp, beans, spell, wand, etc.) it usually carries a sense that the target noun has supernatural powers. My careful writer could imply that his book actually had supernatural powers in speech (by placing very heavy stress on the word magic), but in writing it would require rephrasing or help from context to disambiguate that third possible sense.
I suspect that (particularly in casual speech) there may be a tendency to use the shorter adjectival form more often, simply because the "more appropriate" full version sounds a bit highfalutin.
Having said that, there are plenty of established collocations where no literal reference to the supernatural is implied by the short form (magic square, lantern), and some that almost defy categorisation (magic marker, bullet). Those are the ones you just have to learn to get used to.