As is often the case, I get the feeling that "Is it wrong to do X"?" is perhaps not the most useful question to ask about a sentence such as yours:
Would it be odd to you if I do not say "bless you" when you sneezed loudly and keep silence instead?
Hearers and readers can work their way through a labyrinth of syntactical structures that might defeat a purely logical mechanical interpreter. Nevertheless, as I look at the example sentence, I get a feeling of vague disquiet because the wording forces me as a reader into making a more complicated series of interpretive adjustments than I would like in order to come to terms with so straightforward an underlying idea.
My advice would be to put "I do" and "[I] keep" into past tense for the greater interpretive comfort of your hearers or readers:
Would it be odd to you if I did not say "bless you" when you sneezed loudly, but kept silent instead?
or even more lucidly:
Would it seem odd to you if, instead of saying "bless you" when you sneezed loudly, I remained silent?