The answer to your question is "no".
While its true that
visibility : seeing :: audibility : hearing in its normal context, the particular use of the word you're referring to would probably be classified as "jargon". It is derived from a military/transportation context, and is rarely (if ever) used outside of these. A pilot or a driver might say "I'm reducing speed due to poor visiblity", but if you were taking stroll with a friend in the fog and remarked that "visibility is poor", he or she would definitely give you a quizzical look.
Strictly speaking, visibility (and audibility) are nouns that refer to the ability of a thing to be seen (or heard). When one says "there is poor visibility" or "visibility is poor", there is an implicit abstract subject or indirect object: it means "the visibility [of things in general] is poor."
This type of thing is one of the stranger corners of the English language, one of the few ways that French influenced English grammar (as opposed to vocabulary, which was influenced much more) after the Norman conquest. In French, this type of grammar is common: "ça va, [on] ne pas", etc.
In English, there are a few other words that are used like this. For example, one might ask a shopkeeper "how's business?", and he might reply "business is poor". In this context, "business" does not refer to commerce in general, or even the shop itself (directly), but instead, it refers to the state of the shop's present profitability.
In any case, if you're looking for a more formal way of saying something is difficult or impossible to hear (although saying "hard to hear" is perfectly fine): you might use the adjective "inaudible", i.e., "you're inaudible" (I can't hear you), or "it is inaudible in the concert hall" (I can't hear in there).