Consider these notes on dealing with reported speech containing the word ago:
Changing time and place references - EducationFirst:
- Phrase in direct speech: "I saw her a week ago," he said.
- Equivalent in reported speech: He said he had seen her a week before.
Time and Place in Reported Speech - English Club:
- Direct speech: two minutes ago
- Reported speech: two minutes before
Noun Clauses/Reported Speech (Indirect Speech); Tense Harmony or Sequencing - Mary Nell Sorensen, University of Washington:
- Direct speech: five days ago
- Reported speech: five days before, five days earlier
- Direct speech: ‘I finished the job three weeks ago,’ the boy protested.
- Indirect speech: The boy protested that he had finished the job three weeks before.
The last reference also provides a rationale for the shift from ago to earlier or before:
We often change demonstratives (this, that) and adverbs of time and place (now, here, today, etc.) because indirect speech happens at a later time than the original speech, and perhaps in a different place.
- Cambridge Dictionary
This accords with the normal use of ago as a time reference based on the speaker's present. In reported speech, the speaker changes, so the word ago no longer conveys the correct time. The word ago is therefore changed to earlier or before.
On this basis, the word ago in your example references the present (time) of the speaker of the sentence. Since you indicate that the full sentence was uttered in 2016, the time two years ago refers to 2014. This is so even though the sentence reports a traveller who spoke in 2015.