Could is a little more polite than can, since it is a conditional verb. There is an implication of it relying on unspoken conditions being met. Since those conditions are in the hands of the person being asked, it gives more authority and power to them, so the asker comes across as being humble and polite.
One problem, however, is that both can and could are verbs that express ability as well as permission ("Am I able to borrow your pen?" / "Would I be able to borrow your pen?"), which makes them a bit ambiguous, and can sound strange. Therefore it is more correct (and just as polite, if not more) to use the verb may, which is a pure request for permission with no ambiguity. "May I borrow your pen?" is perhaps the best way to ask your question.
Incidentally, an alternative to may is the conditional verb might: "Might I borrow your pen?" Some would say this is even more polite, others may/might/could think it to be going a bit too far.