- "We went swimming later in the afternoon, Jack and I."
That's a right dislocation construction. It is common in informal style (and usually the dislocated pronoun phrase will use accusative case).
The reason why your example sentence seems a bit strange or awkward sounding is probably because the personal pronoun in the dislocated phrase is in nominative case, which is rather unusual (though, if this is fiction, then the author could have done that intentionally to stay consistent with the narrative voice).
Here's a related tidbit. CGEL page 462:
Accusatives are also the only option for the left- and right-dislocaton constructions (which are themselves characteristic of informal style):
[18]
- i. Me, I wouldn't trust him further than I could throw him.
- ii. I don't much care for it, me.
If you're interested in more info about these kinds of information packaging constructions, you might be able to find it under the topics of right-dislocation (and left-dislocation).
Note that CGEL is the 2002 reference grammar by Huddleston and Pullum et al., The Cambridge Grammar of the English Language (CGEL).