The expression refers to a problem in physics. If you were to watch a movie of a small number of particles interacting (e.g. molecules of gas bouncing off each other) then you could not tell whether the movie was running forwards or backwards: the equations that govern the particles work equally well in both directions. In this sense the equations do not have an "arrow of time" telling you which way time goes; you can switch the direction of time and everything still works exactly the same.
However if you were to watch a movie of thousands of molecules of two gasses, starting with all of gas A on the left and all of gas B on the right, you would immediately see that the film was running forwards as the molecules mixed. If it was run backwards you would see the two gasses "magically" separating.
So there is a paradox: somehow the "arrow of time" emerges from the collective behavior of the gas molecules rather than their individual behavior. "Entropy" is the technical term given to the degree of randomness in the situation. Roughly speaking, the start state with the gasses partitioned has less entropy than the end state with them all mixed. The mathematical definition is about the possible number of places all the molecules could be: obviously if the gasses are separated then the molecules of gas A are constrained to be on the left. But once the gasses are mixed then any molecule of A could be anywhere. Likewise for gas B starting on the right. So the end state has more possibilities than the start state.
