A common idiom in English is to go where the wind blows, meaning "to do something without preparation" (MacMillan)
I travelled wherever the wind took me.
Our conversation started after dinner and it took us wherever the wind blew. The sun was rising before we were done.
Being more literal, you might use wander or aimless:
In my aimless travels, I wandered wherever the wind blew me.
We had a conversation about life that wandered aimlessly until morning.
