There is no real need to use 'literally' if what you are saying is obviously true in a literal sense. Considering your example, since the Olympic games are actually 'literally' watched by millions if not billions of people, you are right -- you don't need to use 'literally'! And as Dan Bron pointed out in comments, billions is probably too big a number, even for the Olympics, so I would advise you to change it to 'millions', or even 'hundreds of millions' as suggested by Mari-Lou A in comments.
The more orthodox usage of 'literally' is to give emphasis that something (that might be literally or figuratively expressed) is actually being used in a literal sense here. Example:
The concert literally brought the roof down (not figuratively but literally: the loud noise literally collapsed the roof!)
She literally kicked him out of the house.
Please note that the less orthodox but quite popular 'alternative use' of 'literally' is paradoxically meant to emphasise and intensify a figurative usage, as in
After the breakup, he was literally swimming in a sea of tears.
The headmaster literally burst with rage after seeing the boys' nasty trick.
IN SHORT, the 'literally' seems unnecessary in this context, though I expect the two sentences will not have the exact same meaning with and without 'literally.'