Is the word "of" optional in this instance? Is either of these considered preferable to the other?
- Taste all our delicious treats.
- Taste all of our delicious treats.
Both sentences are grammatical.
See the following NGRAM (all our, all of our):
It would seem that the use of "all of our" is growing nowadays.
Also, "all of our" ("all of our delicious") gets 18,200,000 (745) hits on Google Book, while "all our" ("all our delicious") gets 26,500,000 (4,240) hits.
If you go back a century or so, "all of" would be very rare indeed. The fact that it occurs a bit more often nowadays is no real reason for using it - it's still far more common not to use "of" after "all".
Having said that, in OP's context either form is acceptable. For reasons that aren't clear to me, it seems that "of" is actually required when followed by a pronoun...
I can't think of any similar context where including the preposition is "unacceptable", though it would certainly be stylistically clumsy to overdo it...
"You can fool all [of] the people some of the time, and some of the people all [of] the time, but you cannot fool all [of] the people all [of] the time." (attrib. Abraham Lincoln, among others).
Idiomatically, all is slippery. You can take all of me/it/us/them, and you can take it/us/them all, but you can't take me/him/her all. That's not because those pronouns are singular (take it all is unremarkable) - it's just a matter of idiomatic usage.
Even though both versions scan as natural English, a copyeditor will always reduce “some/most/many/all/several/few/one/… of the” to remove the of the part. It’s just extraneous baggage. You don’t need it to create a partitive sense in English.
Even here in your example where the determininer is a possessive adjective instead of definite article, one can often reduce it — and should do so. You certainly don’t need the of portion. Your phrase is better without it.
One general guideline is that whenever you can remove a word without changing the sentence’s meaning, that you should do so to tighten the phrasing.
Allow me to further explain the grammar in these sentences.
In 1, "all our delicious treats", "all" is a predeterminer, because "our" is a determiner. A determiner is used before a noun, while a predeterminer is used before a determiner to further specify the noun phrase "delicious treats" in this case.
In 2, "all of our delicious treats", "all" is a pronoun.
The use of "all" as a determiner, a predeterminer, or a pronoun is explained here with more examples: https://jakubmarian.com/is-it-all-the-or-all-of-the-in-english/