First, 'being' doesn't sound terribly natural. As @JohnLawler pointed out, the verb should be one of motion such as 'walk', 'go', 'come', etc. So if there has to be any participle that's omitted, it's not 'being' but any one of 'walking', 'going', 'coming', 'traveling', etc. But only 'being' can be omitted from the participial clause; therefore, 'a few miles into the town' cannot have originated from a participial clause.
Second, even if you could add 'being', I don't understand how being able to add 'being' would be a test for a verbless clause. Being able to add 'being' is a test for a predicative complement, and a predicative complement is not automatically a verbless clause.
Third, I think the right test for a verbless clause in this example should be whether you can add something like 'when' that is known to introduce a clause.
Make adjustments when [necessary].
Here, you could say 'necessary' is a verbless clause because whatever follows 'when' is a clause.
So does this work?
When [a few miles into the town], I saw a beautiful building that was now abandoned.
Not to my ears.
When [in the town], I saw a beautiful building that was now abandoned.
Where I'd say 'in the town' is a verbless clause.
Fourth, 'a few miles into the town' clearly provides locative information of the verb 'saw'. That's why putting it as part of the main clause is not problematic:
I saw a beautiful building that was now abandoned, a few miles into the town.
Note the comma is necessary to mark the phrase as not belonging to the relative clause. So you wouldn't need a comma if you get rid of the relative clause.
I saw a beautiful building a few miles into the town.
This clearly shows that 'a few miles into the town' is part of the main clause and an adverbial phrase. It's not clear what the head is. If it's 'into', the adverbial phrase is a prepositional phrase. If it's 'a few miles', the phrase is a noun phrase. In either case, it is an adverbial phrase.