There hasn't been a moment where/when science has ceased to amaze me. What do I choose? I have also seen use of "that" in some cases: There hasn't been a time that I haven't loved you.
Normally, in informal contexts, where is used to introduce a defining relative clause referring back to a place, and "when" to a time.
So, "There hasn't been a moment when science has ceased to amaze me." is more idiomatic.
Read more about Relative Pronouns here: English Grammar Today (Cambridge Dictionary)