There's nothing wrong with just eating away the building's fabric.
In fact, all of these are acceptable variations:
Decay and neglect are slowly eating away at the building's fabric.
Decay and neglect are slowly eating away the building's fabric.
Decay and neglect are slowly eating at the building's fabric.
Decay and neglect are slowly eating the building's fabric.
The use of the preposition, with our without away, is simply a method of adding a kind of metaphorical description to the action.
For instance, when I think of eating away at, I have a brief mental image of a mouse—or even of something like the titular character from the Pac-Man video game. It also, perhaps with the associated visual, makes me think of nibbling rather than eating in a generic sense.
I suspect I have this visual because the use of at implies something or someone being somewhere. So, not only is there something that is being eaten, but it more strongly enforces the (visual) idea that there is something else present doing the eating.
Similarly, the use of away implies a more long-term effect, one that's taking some time, rather than something that's just quickly swallowed. Of course, the use of slowly in the example sentence also conveys that meaning, but away reinforces it.
Grammatically, there's nothing wrong with any of these sentences. It's just a matter of style, personal preference, habitual language use, and, perhaps, implied imagery, that's determining the verbiage used.