Because of racialization, or "colorism", this is complicated and it really does depend on context, and whether you wish to speak with cultural sensitivity or political correctness. Are you describing yourself in comparison to another? Are you asking about someone's complexion in relation to race, beauty, or blood status? There are many meanings that can be correctly applied and still have a resulting incorrectness.
If you're speaking about make-up or sunscreen, just saying "fair" or "sun-sensitive" would be a way to avoid the "loadedness or baggage" that pairing "fair" or "light" with skin or complexion might get you mixed up in.
However, in the appropriate cultural contexts, many terms to describe complexion are used freely, even with the inseparable "loadedness and baggage." Describing a person to help identify them to another: "Do you see that cute guy near the door? The tall, light-skinned one with the blue shirt..." In Padma Lakshmi's recently released memoir she talks about terms used in arranged marriage ads, including words like "light" "wheat" and "dusky"; all with ascribed value for a potential match.
When others have described me, I've been called everything: red-bone, light-skinned, high-yellow, colored, the n-word, etc. and I'm not even Black. My older sister however, burns easily and I always call her "fair", not "fair skinned". (And she is beautiful too.) My other sister, I call "dark" not "dark skinned". She is just a touch "darker" than me. But in winter, I consider myself "pale", as I'm "darker" (not tan) in summertime....
There is so much, and yet not enough literature on this topic...here are a few resources: