The answer is, unsatisfyingly, that it depends. Most native speakers aren't fluent in the borrowed language and so won't know the grammar principles there.
Sometimes things are borrowed exactly, like Latin sayings, and stand alone with no possibility of declining, like 'ceteris paribus'
Sometimes a simple thing, like a plural, if easy, is declined, like Greek the singular for criteria is usually said 'criterion'. But sometimes that strict adherence to a foreign grammar is lost and 'criteria' is used for singular, 'criterias' for plural.
Depending on how unnatural or infrequent the foreign word is, the foreign manner may be preserved or not: 'My fiancée' for a female to be married is correct English not because English has grammatical gender like French but English allows an alternate lexical item (ignoring for the moment that it is a difference in spelling only). But 'The banana was flambé' is grating in English, so 'flambéd' is preferred, despite it sounding very unFrench.
The principle is that someone who has heard the foreign word repeats it to an English speaker. However clear that foreign word is spoken (by native speaker or not), the English speaker will assimilate it as best they can, ignoring what just doesn't work in English, and preserving possibly what does work.
So, for an arbitrary foreign word slipped into English, what form should it take? You probably want to preserve what you can and drop what just sounds bad. And then English speakers will do with it what they may. Early on, 'data' was plural, but nowadays, since 'datum' is almost entirely unheard of, different groups continue to use 'data' as plural (rarer) and others use it as a mass noun (taking a singular verb).
So there's no hard and fast rule; try to conjugate as much as you can, but once it is firmly in English, then English rules will apply.