There was this scene in the motion picture "Captain America: The First Avenger" where Steve Rogers and Agent Peggy Carter are driving through Brooklyn. The following dialogue ensued:
Steve Rogers: I know this neighborhood. I got beat up in that alley. And that parking lot. And behind that diner.
Peggy Carter: Did you have something against running away?
Steve Rogers: You start running they'll never let you stop. You stand up, push back. Can't say no forever, right?
Who is Steve Rogers referring to when he says "Can't say no forever"? Is he referring to himself, meaning he can't say no to a fight forever or is he referring to the men who used to beat him up?