I know this question is both old and answered, but I would like to add something important.
Consider the following sentences:
John and Mary's car broke down.
John's and Mary's cars broke down.
Both sentences are correct, but they mean different things. In the first sentence, "John and Mary" are a unit that together own the car that broke down. In the second, "John" and "Mary" are two separate people with two separate cars.
In English, the genitive apostrophe-s ending is a lot more flexible than other endings (technically, it's what is known as a "clitic") and always appears at the end of the noun phrase. Hence,
the man in the black hat's bobcat
*"the man's in the hat bobcat.
This does not change when discussing pronouns. The first question you should be asking yourself is
Are I and my friend sharing this day that has been affected?
If your answer is "no", then Justin is correct - he has affected your and your friend's days.
If your answer is "yes", then Justin's argument does not apply, and the proper question is
Is it "me and my friend" or "I and my friend"?
Which is completely up to you, as English has no rules about whether to use "I" or "me" in this context.