Understanding here depends less on the meanings of *church* than on the meanings of *go*.

There are [numerous uses of *go*][1]. Most commonly it refers to moving or traveling somewhere. In this sense, and when by *[church][2]* we mean a building used for Christian worship, we use the article with *church* according to the usual rules:

> I'm sure I lost my camera in Montmartre; I went to a church there— but I don't recall which one— and left it in a pew.
>
> We stopped for lunch in a small town, and I walked around a bit after we ate. I went to the church, then the square, then got an ice cream soda at the drugstore.


*Go* can also mean to attend or visit a place or type of place for a particular purpose, however. To say you *go to church* means not only that you physically situate yourself at the building, but that you are engaged in regular worship services there. In this sense, you do not use an article.

> I went to church in the morning so I could watch the football game later.
>
> I went to church growing up, first Blessed Sacrament and then St. Ann's after we moved. But I lapsed when I moved to the city.


The same change of meaning applies for a number of other words which can denote both a location and a particular engagement: *court*, *school*, *market*, *town*, and so on. To *go to a jail* is to visit a penitentiary facility; to *go to jail* is to be incarcerated; don't mix up the two in conversation.

English being English, unfortunately, this is not a strict rule. Most geologic features, for example, require a definitive article when used in either a specific or generic sense: *the mountains*, *the shore*, *the woods*, etc. Certain proper nouns always take the definitive article as well. Thus, a simple statement can be ambiguous.

> I went to the Church of St. Luke when I lived in Lexington.

could mean that you once visited the building known as St. Luke's, but it could also mean that you were a regular parishioner who attended services every Sunday.

> I went to the beach last summer.

could mean you visited a particular beach once last summer, but it could also mean you went to one or a number of different beaches as a regular activity last summer.

As always, context is key.

  [1]: http://oald8.oxfordlearnersdictionaries.com/dictionary/go_1
  [2]: http://oald8.oxfordlearnersdictionaries.com/dictionary/church