The OED says it comes either from port (harbour) or port (gate), but whichever it comes from, the explanation is the same.
Etymology: Either < port n.3 [A gate, a gateway; spec. (from the 14th cent.) that of a city or walled town. Now Sc. (hist. exc. in the names of the spot or street where a city gateway stands or stood formerly, e.g. West Port in Edinburgh).], or < port n.1 [A town or place possessing a harbour which boats use for loading or unloading, or which forms the starting point or destination of a voyage; spec. such a place where charges may be levied under statute or by prescription on boats making use of the facilities. Now also occas.: an inland port.].
Whether derivation from port n.3 or from port n.1 is accepted, the underlying explanation is probably broadly the same. When the steering apparatus was on the right side of the vessel (compare starboard n.), the vessel when in port (compare port n.1 3 [A place on a coast or shore which boats use to shelter from storms, or to load and unload; a harbour, a haven]) would normally be placed so as to lie with her left side alongside the quay. Any opening to allow entry or loading (compare port n.3 2 [Naut. An opening in the side of a ship for entrance and exit, or for the loading and unloading of cargo; (also) an aperture in the side of a ship for a cannon; a porthole.]) would also have to be on this side.
Derivation < port n.3 appears the more likely of the two possibilities. This side of the ship was probably already known as the ‘loading’ side of the ship (see larboard n.) and as such contrasted with the ‘steering’ side (starboard n.); it is likely that these associations were maintained. Furthermore, port n.1 is not normally used spec. for the quay or a similar structure.
A suggested derivation < French bâbord (see babord n.) is not likely on phonological grounds and is not supported by evidence.