The sentence is
I bought a bottle of milk on Friday night, and I found that it had gone sour the following morning.
I think the past perfect tense is weird with "the following morning." Am I right?
This is ordinary and correct.
The clauses concerning purchase and finding are consecutive narrative pasts.
The clause with past perfect had gone sour is subordinate to the second of those clauses only, and is unaffected by the first:
On the following morning I found that it had gone sour [at some earlier time during the night].