You can put an apostrophe in it if you want, but I'd write a boys bicycle at least as often as any apostrophized phrase. Either placement of the apostrophe would be considered correct by some, and incorrect by others.
In speech, there are no apostrophes, because they're always silent. In writing, apostrophes are misused probably more often than they're used "properly", so most English speakers can't tell the difference.
If it's for anyone except your English teacher (whom you should ask in any event), don't worry about it.
If it's for a boss, find out what they think. They might not notice, or they might prefer just about anything, and they're paying, after all.
Oh, and before I forget, possessive is not a case in English; rather, it's a noun phrase clitic that gets attached to the last word of a noun phrase, as in
- the Archduke of Austria-Hungary's assassination
where it was the Archduke, not Austria or Hungary, that got assassinated.