The OED entry for audience is vast. But all senses relate in some way to hearing. OnlyThat which follows is sense 8, which might in some way justify using it forbe considered relevant, but see my further edit 2 below, which draws attention to sense 7b, which has greater relevance to books.:
I am now persuaded, following extensive discussion in the commentary below, that the word audience has relevance to literary works. However what we have not been able to agree upon is the nature of a book's audience. The OED seems to restrict it to "people who admire, support or take a consistent interest", but according to @Edwin Ashworth other dictionaries refer to the people reached, whatever that means.
Further Edit (2)
My attention has been drawn by @Roaring Fish, to the fact that the relevant OED sense of audience is not 8, as I wrongly stated, but 7b, which deals directly with books.
b. Those people who have read or regularly read a particular text, publication, or writer, considered collectively; a readership.
1760 B. Franklin Let. D. Hume 27 Sept. in Wks. (1887) III. 128 It often gives me pleasure to reflect how greatly the audience (if I may so term it) of a good English writer will, in another century or two, be increased.
a1854 H. Reed Lect. Eng. Lit. (1878) vii. 225 ‘Pilgrim's Progress’..has gained an audience as large as Christendom.
1867 Brit. Q. Rev. July 108 Many of Keble's poems impress us with the idea of..an audience of whom the writer was conscious.
1870 G. Meredith Let. 13 Oct. (1970) I. 428, I have an audience of about a dozen, but if they're satisfied I am too.
1883 G. Hamilton in E. C. Rollins New Eng. Bygones Pref. 1 This book is published with no thought of an audience.
1949 Los Angeles Times 13 Feb. (Comics Section) 1, There may be neurotics in our audience!
1991 Utne Reader July 109/3 (advt.) The smallest village 20 miles outside of New York City may be tiny, but it has a hot newspaper with a sophisticated audience.
1993 Locus Oct. 4/1 I'm a writer who has an ongoing dialog with an audience, and what that audience tells me feeds back into my work.
2008 Vanity Fair June 91/2 Like any blog site but grossly magnified due to the mass scale of its audience and influence, Daily Kos is a schizophrenic enterprise.