If the only direction for answering this question is to choose the best fit for the blank in the sentence, then I have to agree with you that B) is the answer.
Both C) and D) result in grammatical errors. C) results in a subject that's missing a verb, and D) results in a verb that's missing a subject. We can easily ignore those possibilities.
Both A) and B) result in grammatically sound sentences. "Floated" could be the simple past tense form, or it could be the so-called past participle form. "Floating" could be the so-called present participle form, or it could be the gerund form. The blank in the model sentence can be filled with a participle. If we have to choose only one of these answers, then we have have to look at more than grammatical correctness.
You found a good reference sentence. Tectonic plates float on the molten mantle. This sentence uses the common intransitive sense of the verb "to float", and it employs the active voice. The same structure can be used to describe some of the information in the model sentence: Rubbish floats in the sea.
The verb "to float" also has a transitive sense. In fact, it has more than one. The molten mantle floats the tectonic plates. Children float their toy boats in this pond. We can form passive voice sentences from the information in these active voice statements: Tectonic plates are floated by the mantle. Toy boats are floated by children.
We cannot choose between A) and B) on the basis of which voice makes sense. Both voices make sense. The rubbish is floating. The rubbish was floated by someone or something.
I would choose B) because the resulting sentence is less confusing. It is easy to mistake the form "floated" for the simple past tense, since the forms are identical. The passive form implies an unmentioned agent, but no such agent is needed for the model sentence to make sense. The intransitive sense of "to float" is more common. English is my native (and only) language, so I can choose the option which sounds the most natural.
If I had to choose only one answer, I'd choose B) as the best answer. If I had to choose all applicable answers, I'd choose both A) and B) as correct answers. I don't see any way to eliminate B) as a valid choice.