You said not to worry about why it's being used... but I can't help it!
Have you ever noticed that getting stuck on a question of weird grammar or social usage is an indicator that you're using the wrong word to begin with?
"Man-Hours" is not directly derived from the Noun man, rather it refers to the Verb man.
So the important word to consider here is how the word "manning" applies. The Verb man has 2 senses; first "to be at your station", second "to assign to stations". These 2 senses bring up the concept of perspective. If you are the person manning the station you are spending literal hours, if you are management making assignments you are spending man-hours.This harmonizes with standard business usage of man-hours which is either for estimation purposes or for cost accounting.
I looked for a good reference on this. While it's easy to find a ton of references showing that man-hour is primarily used for estimation and cost accounting, it's very hard to validate that it is only used that way.
So I'll propose some thought experiments to demonstrate that we have an intuition about correct usage;
"Wow I'm tired. I worked 12 (hours or man-hours) today."
"Staffing is going to take 78 (hours or man-hours) to keep this store open 24 (hours or man-hours) a day."
"That will take 3 hours for the two of us to finish. It's 5pm now, it will be on your desk (at 8pm or 11am tomorrow)."
"That will take 3 man-hours to finish. It's 4pm now, the 3 of us (can or can't) have it done by 5."
"It takes 2 hours from the time the metal is poured until it can be packed for shipping, at a cost of .12 (hours or man-hours) per part."
So assuming that your instincts have been guided by standard business usage (and not distorted by a pointy-haired-boss who missuses it and every other buzzword), you'll see that you would never refer to literal hours that you personally spent as "man-hours".
Using the correct term clarifies the gender aspect by clarifying that the subject is non-gendered in the first place.
If your industry doesn't use the term "manning" don't use "man-hour". (Manning remains a current phrase in manufacturing, since it clarifies the scope of discussion to the time of actual machine operators and excludes support staff and management.)
Using the phrase Labor Hours (as TD pointed out) clarifies the focus of the discussion is the work and NOT the identity of the workers.
In non-manufacturing (or military) contexts you probably are actually talking about:
- "Staff hours" - includes support staff
- "Business hours" - specific to the hours a business defines itself as "open"
- "Project hours" - specific to the fractured nature of time spent on a project by one or
more persons
- "Billable hours" - profession specific
{ References: You're not going to find precompiled discussions of the origin of "man-hour".
So if you need to validate the above statements you'll have to do your own Etymology research.
The following are some samples of highly edited and regulated usage of "man hour" as a measurement of "manning" and "manpower".
US Army - www.apd.army.mil/pdffiles/r570_4.pdf
US Navy - www.public.navy.mil/bupers-npc/organization/navmac/Documents/MANPOWERGLOSSARY.doc
Rand Organization - www.rand.org/content/dam/rand/pubs/monograph_reports/2007/MR1436.pdf
These organizations have been using the noun manpower, verb manning, and measurement man-hour since (at least) World War 2. The first attested use of Man Hour is attributed to 1911 in a business context, but the original source is not searchable online.