In the first sentence, silence is general (silence that is silence, wherever and whenever) and then restricted to be the specific one that's as deep as the bottom of the sea. The awkward point, as ruakh has mentioned before me, is that the pronoun "one" is referring to an uncountable noun (silence). In some cases this might be more tolerable with the help of the context:
-Do you want black coffee or white coffee? (uncountable coffee)
-A black one please. (countable; a cup of coffee)
But in your sentence it still is a little awkward. So I, too, prefer silence to be countable. Furthermore, I'd replace deep (having a large distance to the bottom from the surface) with profound (far below the surface):
- I spent the whole night listening to a silence, one as profound as the bottom of the sea.
In the second sentence, we have "the silence" (a particular one) followed by an identifying expression: "one as ...".
The definite article "the" prompts us to look for qualifying information. Since no prior mention has been made of silence, this information should be deduced either from the context (the silence that I was immersed in at that moment), or from the following expression. But the following does not have one of those common structures to qualify a previous word (e.g. the silence that ..., the The Silence of the Lambs, etc), and if the context is to be sufficient to qualify silence, there won't remain a connection between silence and the following expression. This is probably why you feel funny about the second sentence.
On the other hand, omitting the verb to be, while not as common in modern usage, is (was) acceptable when the meaning is (was) clear:
Who so firm that cannot be seduced?
So you can interpret the second one as
- I spent the whole night listening to the silence, one (that was) as profound as the bottom of the sea.
My hunch is that neither is wrong, but the first (as modified here or in ruakh's post) is more natural.