The other answer is correct, but I wanted to supply some references.
knock against from McGraw hill:
to bump against someone or something. Mickey knocked against Mary and said he was sorry. I didn't mean to knock against your sore knee.
Both of the examples are quite literal there.
knock from Collins via TFD
verb
- (tr) informal to criticize adversely; belittle: to knock someone's work.
noun
- informal unfavourable criticism
So, from those definitions, we can replace the words "knock against" in your sentence with "unfavourable criticism":
"It's not an unfavourable criticism of you, I'm sure you did the best under these circumstances."
It sounds kind of clunky when put that way, but it demonstrates that you are using the phrase correctly.
