The distinction depends on what is on the mind of the person who is making the mistake: is that person mistaken about the relevant rules of English grammar, or merely about the spelling? When we see a mistake, we can usually think of a reasonable explanation of what led the person to make the mistake, and then classify the mistake accordingly. For example, it is reasonable to think that a person who wrote 'This iz the end' is relatively clear about how to structure this sentence and is only mistaken about the spelling of is. We would thus say that this is a spelling mistake.
Sometimes, it is, however, difficult to be sure what was on the person's mind. Did the person who wrote 'Your the best' want to write 'You're the best' and was mistaken in thinking that you're can be spelled as your? If so, this would be a spelling mistake. But maybe the person really wanted to write your and mistakenly thought that English syntax permits combining your and the best in this way. In that case, the person would be making a mistake about the grammar. Or, perhaps, the person knows that this combination does not fit the standard rules of the syntax, but mistakenly thinks that 'Your the best' is some sort of an idiom that is an exception to the standard rules. That would be a mistake of yet another kind. If we are not sure what was on the person's mind, we cannot be sure how to classify the mistake.
There is thus no simple rule for classification of such mistakes that can be applied solely on the basis of what the mistake looks like; the classification is always based on our (more or less reliable) reconstruction of how the person was led to make the mistake.