There is a term little used outside the legal domain:
In law, the curtilage of a house or dwelling is the land immediately
surrounding it, including any closely associated buildings and
structures, but excluding any associated "open fields beyond", and
also excluding any closely associated buildings, structures, or
divisions that contain the separate intimate activities of their own
respective occupants with those occupying residents being persons
other than those residents of the house or dwelling of which the
building is associated. It delineates the boundary
within which a home owner can have a reasonable expectation of privacy
and where "intimate home activities" take place. It is an important
legal concept in certain jurisdictions for the understanding of search
and seizure, conveyancing of real property, burglary, trespass, and
land use planning.
In urban properties, the location of the curtilage may be evident from
the position of fences, wall and similar; within larger properties it
may be a matter of some legal debate as to where the private area ends
and the "open fields" start.
But 'grounds' is probably the safer option. Both legally, and for user-friendliness.