Comma placement is a matter of style, and there numerous opinions about which style is best. These opinions are to be found in style guides, one of which you may want to consult. Bear in mind that while they will all agree on some rules, they all differ with each other in some respects. And they will leave some decisions to the author.
Your sentence juggles three commonly-found rules about commas:
- Separating items in a series
- Setting off asides
- Ending an introductory adverbial prepositional phrase
1) I embarked upon a journey filled with adventure, excitement and, at
Some people like a comma separating all items ("adventure, excitement, and despair"), and some believe that the conjunction serves to delimit ("adventure, excitement and despair"). This sentence 1 above chooses the latter for the list and treats the prepositional phrase "at times" as an aside, a comment that interrupts the normal flow, and so is set off by commas.
2) I embarked upon a journey filled with adventure, excitement, and at
In sentence 2, you take the suspenders-and-a-belt approach, as would Oxford University Press, which has lent its name to the comma before "and". In this sentence, "at times", as an adverbial prepositional phrase introducing "despair", is considered worthy of a trailing comma. That's a judgment call, since the phrase is short, only two words. A comma would clearly be in order for a longer phrase:
2a) I embarked upon a journey filled with adventure, excitement,and
at times and in the general course of things, despair.
3) I embarked upon a journey filled with adventure, excitement and at times, despair.
In permutation 3, you have abandoned the Oxford comma but kept the comma for the introductory phrase.
4) I embarked upon a journey filled with adventure, excitement, and, at times, despair.
In the final version above, you want both to keep the Oxford comma and to set off the aside. It's unlikely that an observant copyeditor would allow this because it has the unfortunate effect of making the list look five items long instead of three. To effect your purpose, you may uses dashes or parentheses instead to set off the aside:
4a) I embarked upon a journey filled with adventure, excitement, and
(at times) despair.