I started my odyssey on the splendid world of mathematics when...
Is this a right way to use the word "odyssey"? Is "odyssey on sth." correct? Can I use "splendid" to describe the world of mathematics here?
You are perfectly entitled to describe the world of mathematics as 'splendid' if you want. It is not the word I might choose but then I'm not a mathematician.
I think it would be better to say that you conducted your odyssey 'into'..... An odyssey is a 'journey', a very special journey, but journeys are taken 'into' (or possibly 'onto'- such as a journey onto the planet Mars), or if to a specific destination, 'to'. But by definition an odyssey is a journey which is not to a specific place, so I think your best bet is 'into'.