If I want to say that during the accident there were no passengers, how do I phrase it?
There were no passengers at the time.
There were no passengers at that time.
They are both correct, but "at that time" has that stilted, official-ese sound that some government employees like to affect when they are talking to the press. "At the time" is what normal people say.
Exception: if you are distinguishing between two times, especially between some former time and now, use "that". E.g. (emphasis added) "The National Association for the Advancement of Colored People was established in 1909; at that time, 'colored people' was considered the polite word for 'black'."