This has nothing to do with "all" and everything to do with "the".
You should use "the" wherever a definite article is required. ("The" is a definite article as opposed to "a/an" which are indefinite articles)
EXAMPLE
For example, if I say, "Let's read the book," I mean a specific book. If I say, "Let's read a book," I mean any book rather than a specific book.
Here's another way to explain it: The is used to refer to a specific or particular member of a group. For example, "I just saw the most popular movie of the year." There are many movies, but only one particular movie is the most popular. Therefore, we use the.
PS: You can say, "Lets read all the books." This is because you know which lot of books you are referring to. On the other hand, "Let's read all books" - sounds odd!!