As noted in the question you link to, "professor (of field or subject) at (school)" has become the prevailing usage. The change in preferred preposition happened over time. It isn't clear why the shift happened.
Professor in was once more common than professor at.
The Google NGram search suggested by one commenter does suggest that "professor in the university" was once a more common collocation than "professor at the university." However, the Book search results yield nothing in the time ranges where the usage was supposedly most common, so its results are hard to verify. (Two pages of responses before 1906 does not make for compelling support.)
To find a denser cluster of academic samples, I did a JSTOR search for both "professor in the university" and "professor at the university" for before 1950. The results are virtually equal: 2,528 for the former and 2,535 for the latter. Note that the total results are no contest: 3,833 for in, 27,680 for at. So something big changed after 1950, supporting the general recommendation today.
When the same time range is restricted to before 1900, "professor in the university" has 728 results and "professor at the university" has 239 results. Skimming the first few pages of results suggests that the search terms were solid. Names link to sources:
the late Dr. J. J. Hofmann, extra-ordinary professor at the university of Leyden
AUGUST MEITZEN, PH.D., PROFESSOR AT THE UNIVERSITY OF BERLIN.
M. de Wreden, Professor at the University of St. Petersburg
Charles Flint McClumnpha, acting associate professor in the University of the City of New York
JOSEF MAYER, M.D., Professor in the University of Cracow.
Dr. N. Archer Randolph, professor in the University of Pennsylvania
Results for "in" go back to 1742; results for "at" go back to 1820. This suggests that "Professor in" came first in academic publications, but "professor at" grew in usage in the 19th century and surpassed its partner in the 20th century.
They sometimes coexisted.
180 JSTOR results exist for the use of both expressions. 32 results are from before 1900. Here is the list of relevant people from one source from 1888, The American Journal of Archaeology and of the History of the Fine Arts. Most uses are "in," two are "at," and one uses a comma without preposition:
L'Abbe L. DUCHESNE, Professor of Christian Archaeology, Catholic Institute, Paris.
Dr. A. FURTWA(NGLER, Professor of Archaeology in the University of Berlin.
Dr. G. HIRSCHFELD, Professor of Archaeology in the University of Koenigsberg.
Dr. F. X. KRAUS, Professor at the University of Freiburg-im-Breisgau.
Prof. G. MASPERO, former Director of Antiq., Egypt; Prof. at College de France, Paris.
Prof. F. PIPER, Professor of Christian Archaeology in the University of Berlin.
Mr. W. M. RAMSAY, Professor in the University of Aberdeen.
Dr. FRANZ v. REBER, Professor in the University and Polytechnic of Munich, etc.
Dr. TH.SCHREIBER, Prof. of Archaeology in the Univ., and Director of Museum, Leipzig.
There's no strong difference between the prepositions here. So it was possible for publications to not care about the distinction between the two.
What changed?
One hypothesis is that universities themselves changed. (For a quick reference I suggest the book A Brief History of Universities by John Moore.) As universities grew through the 19th and 20th centuries, lots of conversations were held within and between universities about the best practices of everything from admissions policies to core curricula to credentialing and hiring faculty. This long process of standardization, sped up by the internet, is why many institutions ask for similar materials in both student and faculty applications. Over time, it is possible that the language used to describe a professor's institution has similarly standardized, either deliberately (professors liked the sound of being "at" a place rather than "in" it) or not (professors felt pressure to give the right answer for a publication or hiring document, which led to standardizing at arbitrarily).
