I have certainly heard the expression that something or someone was 'noticeable by their absence'. It contains a note of irony and also can imply sarcasm e.g. 'The member who had made such a fool of himself at the last meeting was very noticeable by his absence'.
But on a more straightforward level this is a philosophical matter, comprehensively addressed by Plato in his story of the prisoners in the cave. Did anything which they could not see, exist? And did the things which they did see, exist in the way that they saw them, which involved considerable distortions from a wider reality?
Before radio was invented radio waves existed even though no one could see them, and no one had even heard of them. So it does rather confirm the point that the 'observing' of the absence of something is a less certain use of the term 'observe' than it is when you positively see something before your eyes.
Notwithstanding all that, what does it mean to 'observe'. Plato's prisoners saw what they saw. It wasn't reality as it would have appeared to someone with a better vantage point, but nonetheless they 'observed'. If I am colour blind and a red light appears green to me, it is perfectly correct for me to say 'I observe a green light'.
So the answer is 'yes'. I think you can 'observe' the absence of something. 'Observe' simply refers to vision, and what someone sees, or doesn't see, irrespective of any wider reality.