This involves the catenation of verbs (once you've sorted out the preposition if necessary).
We go / are going / went hiking
is a standard, accepted construction; come / go + -ing form. Quite a few other verbs catenate with the -ing form also.
And placing this after "Let's" poses no problems:
Let's go shopping / dancing / fishing / hiking.
But the other construction is not so easy to evaluate, being of the form verb + bare infinitive.
Here is found:
Catenative Verbs Followed by Bare Infinitive
Christophorsen and Sandred (1970:149) hold that there are few CVs
followed by this type
of complement. Among these CVs are: (1) help, (2) some verbs that may
be combined with the BI of another verb in certain set phrases such as
hear say, make believe, make do, (3) a number of combinations with the
verb let such as let go [= release], let fly, let drop, etc., (4) some petrified
phrases followed by BI such as had rather, had sooner, had better,
etc. Consider the following examples:
2.77 I helped tidy up the room.
2.78 He let fly a string of abuse.
2.79 I heard say that he is back.
2.80 You had better go now. ...
In this pattern, only the verb help, and the petrified phrases can
combine freely with the other verbs. Combinations such as let fly,
make do, hear tell, hear say are virtually a closed set. They are
fixed phrases or idioms (Palmer, 1974:160); (Chalker, 1984:149).
However, the bare infinitive is used, [p 48] especially in the US, in catenations with the verb go (but only in the infinitive):
Go fly a kite!
Go take a hike! (BrE would usually have 'Go and take a hike!')
Let's go take a ride.
Let's go hike the Palos Trail System.
Let's go hike in the Appalachians.
*He went take a ride.