Someone asked me for something and I said, "Do you need it right now, or do I have a couple hours to monkey around with it?" They got really quiet. While certainly unprofessional, is this also somehow offensive?
The context you used your phrase is not offensive. However, the phrase itself, is.
As some comments had pointed out, there is some hypersensitivity surrounding that phrase. Most of the hypersensitivity is due to racial implications.
While I'm sure you meant no offense (your context even proves it), chances are that you probably were talking to someone either of ethnicity or someone who is just sensitive to ethnic slurs.
to monkey around OED
- intr. colloq. (orig. U.S.). To play mischievous or foolish tricks. Also: to fool or mess about or around; to tinker, tamper, or interfere (frequently with about with, around with, or with).
1989 Your Business In financial terms there is no lower end; we can always monkey around with the shares.
It is listed in dictionaries as either colloquial or idiomatic and not offensive. But in these times it is best to leave the monkey out of all conversation.