I searched COCA for modicum of *. Here's the list of nouns that had 3 or more results:
respect, privacy, control, dignity, stability, success, intelligence, justice, effort, security, sense, peace, order, decency, comfort, civility, relief, education, hope, knowledge, interest, discretion, protection, support, talent, trust, truth, understanding, research, decorum, cooperation, credibility, fame, grace, independence, normalcy, respectability, restraint, safety, sanity, wealth, sympathy, time, skill, wisdom, water, self-respect, science, shame, social, light, pride, prosperity, pressure, inquiry, integrity, happiness, courage, courtesy, deterrence, empathy, evidence, exercise, confidence
It's a list of almost entirely abstract items. And when I looked at the hits for "water", I found that all three referred to water as a utility (including one use that used it as an attribute noun: water flow).
The usage you want would not be common as I couldn't find a similar example even among the lower-frequency results.
(Modicum as a loose unit of measure did have some usage a long time ago, as COHA indicates. Most bizarrely, I was able to find an example of "a modicum of eggs". Alas this is wording from a bygone era.)
Instead, you can call it a "splash of coffee":
Where there are no judgements for adding a splash of milk- or just a splash of coffee! — Cohesive Coffee