Only several is not OK, because only implies less than expected, in a way, while several implies more than expected. Using that combination would be somewhat ironical, a bit like this:
Of course I have time; I have only a
ton of work to do this week, as you well know. Sod off.
It is only a few blocks away is your best option, because a few also implies less than expected. If you want to say that the number of blocks is low, but less emphatically so, you could say it is a few blocks from there/here (for some reason, away sounds slightly awkward without only). If you know the number of blocks, simply say it is about eight blocks away.