The term rag trade began in the eighteenth century to describe the sale of rags or second-hand clothes. Its ironic sense meaning the fashion industry began in the late nineteenth. Below are the relevant dates from the OED. Note that the 1983 quotation does refer to "The Manhattan rag trade", suggesting that it has been a term used in the US - at least in New York anyway.
a. Trade in cloth rags or second-hand clothing.
1745 D. Soyer & J. Lockman tr. ‘Monsieur de Blainville’ Trav. III.
xxiv. 189 A particular Tax is laid upon the Jews at Rome... But in
return, their Brokerage and Rag Trade is very beneficial to them.
1887 Jrnl. Royal Statist. Soc. 50 703 Resolutions in favour of
regulating the rag trade were passed; but the usual want of practical
information showed itself in sweeping proposals to prohibit all
imports of rags from infected countries.
1927 Bridgeport (Connecticut) Telegram 16 Sept. 4/5 There is a
slump in the business of the European Waste Material Merchants,
otherwise the rag trade.
2005 M. Vespeth Globaloney vii. 188 It does seem that the rag
trade offers people in third world countries a good opportunity to
earn an income if they sell used clothes.
b. colloquial (frequently ironic and humorous). The business of
designing, making, or selling clothes, esp. women's clothing. Cf. rag
1890 A. Barrère & C. G. Leland Dict. Slang II. 167/2 Rag
trade,..the tailoring business. Also the mantle-making trade.
1907 Daily Chron. 31 Dec. 8/4 They do an enormous business with
the ‘rag trade’—that is to say, the wholesale drapers, silk mercers,
hosiers, and so on.
1957 J. Coates Ship of Glass 241 I know that line. It's going to
be fashionable... Forgive the digression but I'm in the rag trade.
1983 T. Hoyle Last Gasp xvi. 207 He had the native New Yorker's
caustically laconic wit, honed to a fine art by a lifetime spent as a
cutter in the Manhattan rag trade.
2005 T. Hall Salaam Brick Lane i. 22 The old East End rag trade
still survived in the form of leather jacket shops and cheap clothing