It is very simple, and you are basically right. Let’s start with going out to lunch.
Imagine your boss says “I’m going out to lunch.” You would understand that s/he would be going to eat sandwiches in the park, or pop into a favourite eatery, alone or with a colleague or friend. An ordinary, informal lunch. As her/his personal assistant, you would make a mental note to get something to eat and be back by the time s/he returns.
If s/she told you s/he was going out to a business lunch, you would be taken aback. A business lunch is something formal and prearranged on the company time, with other parties involved. You are rather concerned that your boss has not involved you, her/his PA in the arrangements or had it in your diary. It will certainly take up a good two hours. “What business lunch?”, you might ask, with a hint of acerbity.
If, on the other hand, s/he says s/he is off to the business lunch, that use of the definite article can only indicate a business lunch you both already know about. Perhaps you have been reminding her/him for the past hour and a half and at last s/he has got on the move.
The use of the noun business as an adjective qualifying the noun lunch is common usage now. Many disapprove of this practice on stylistic grounds. as do I. But the practice is gaining ground irresistibly, and so must be accepted. Your example is at the most acceptable end: there is no sensible alternative. Language changes, and grammarians must recognise and adapt accordingly.