The building is accurate to its schematics
Is this invalid use of the word "accurate"?
You can definitely say "accurate to" in some cases: for instance, "The stopwatch is accurate to a thousandth of a second" would be entirely correct, and many dictionaries specifically list this use of the word accurate.
The question, then, is whether something can be accurate not only to an amount or measure (be those seconds, meters, or grams) or to n significant digits or n decimal places, but also to a thing.
Personally, I think your example sentence is correct. Other phrases I've found in the corpus include:
Other ways of saying your sentence would include:
...but in both cases, I'd add "precisely" to make it obvious just how accurate the link between the building and the schematics is.
In this context, it seems to give the meaning you desire.
So, yes, you can use it.