At is commonly used to refer to specific moments in time. (In particular, while we'd use on for dates, we'd use at for times on a given date.)
On is commonly used to refer to events that computers react to (especially in software, but it would also cover hardware reacting to interrupts).
An application starting up is both a specific moment in time, and an event that software can react to. As such, both can be used.
I'd lean toward using at for the existing conditions at the time of the application starting (e.g. what environment variables are set, what other applications are running) and on for events which happen because of the applicaiton starting (whether planned events, or unplanned problems). However, I wouldn't consider either entirely wrong in either case.