Side note: That sentence is a run-on. There should be a period or a semi-colon between "elegant" and "this", or a disjunction such as "but", or the sentence should be re-worded.
But to your question: Basically, I agree. In context, the writer appears to be using the adjectives as if he were describing clothing or home decor. In that context, "casual" means "informal". "Elegant" means "tasteful, refined, dignified" etc. "Contemporary" means "current, modern".
I tend to think of "elegant" as almost the opposite of "casual"; "elegant" usually implies formal, fancy, etc. but without being ostentatious. I suppose by the dictionary definition, this isn't necessarily so, but that's the usual connotation. It would be very odd to say, "Alice looked elegant in her blue jeans, army boots, and sweat shirt." If someone said that they would almost certainly mean that her general bearing or appearance were such that even in the grungiest clothes she somehow still managed to look elegant, and not that blue jeans etc. constituted elegant attire.
To a lesser extent, "elegant" is generally understood to imply a certain level of traditionalism, and thus would tend to be an opposite of "contemporary". I don't think that connotation is as strong though. If someone said that furniture or clothes were "contemporary and elegant", that would likely be seen as unusual, but not as a paradox. Still, I'd expect they'd really say "contemporary BUT elegant".