People who share a common interest or passion tend to form or join clubs, rather than groups. Another name for this type of interest group is association, which has an official sound to it and sounds more authoritarian.
For example; a book club is a group of people who exchange with each other their passion and thoughts on literature (general speaking). A [sports/activity] club is a group of people who regularly meet and practice their chosen sport or activity together.
Would we then say the people who belong to a club make up a community? Not really, although members who start off as being strangers to one other can, over time, build close relationships and feel a common bonding between themselves. Community is an interesting word, it can be countable as in the many Polish communities in the UK or it can be uncountable as in "There is a great sense of community in this town." It comes from the Latin words communis and communitas which means with/together (com) and service or function, (munus).
How large is a group of people? I would say a minimum of three is required, a group is usually a number of people who have something in common. They don't necessarily have to have an interest. You can have a group of teachers working in the same school, and likewise you can have a group of students who are in the same classroom or attend the same courses. However, when talking about the total number of students attending a large state school we would hardly ever use the term, group. Therefore, group is identifiable, you belong to a group whether by choice or de facto.
Finally, you can also have a group of inanimate objects or animals (useful if you have forgotten the collective noun; i.e. a litter of kittens; a clowder/kindle of cats = a group of kittens or a group of cats; "group" is an acceptable substitute) but never a community.