I have heard my friend say yesterday evening or yesterday night. I tell her it's last evening or last night. While she may be correct in that it is the night of yesterday, why is it then called last night (or last evening)?
If you're speaking on Tuesday about something that happened on Monday, it's yesterday.
If you want to be more specific, you can say yesterday morning, yesterday afternoon, yesterday evening. The exception here is that native speakers almost never say ?yesterday night - it's invariably last night (and conversely, we don't say ?last morning/afternoon/evening).
(Edit) Thanks to Talia for pointing out that US speakers actually prefer last evening over yesterday evening. As a Brit I find that odd, but at least we all agree last doesn't work for morning/evening.
If you're talking some time later (say, Friday) about what you did Tuesday, and you want to refer back to earlier events on Monday you can't use either yesterday or last, because they always mean the day before the day on which you are speaking (not ...the day of which you are speaking). You have to use alternatives such as...
"On Tuesday I slept through my alarm because I'd stayed up late the previous/preceding evening".
(or "...the evening/night before").