Neither of your examples is most correct; programs are compiled by compilers. GCC is doing the work to compile C code to machine code or whatever.
As for the subtle differences between your examples:
Compiled with is more correct than compiled in but both are awkward compared to the above. Using "with" to say "compiled using" is more appropriate if you are talking about the future:
"We will compile the C-code with GCC"
However, if you are talking about previously compiled code then "compiled by" makes the most sense as the work has been done, and it was performed by the compiler.
Finally, since GCC is a thing (ie a piece of software) rather than a place, "compiled in" doesn't make much sense in this case. When talking about building or making something, consider goods like a car being made in a specific country. This is why we tend to say that products are "made in China" instead of "made by China."
edit: As pointed out in the comments, we do tend to say that programs are written IN languages . Programs aren't written BY languages though, because the language is not doing the work, you are! That should also answer the additional sentences you added about writing IN C++ and then compiling that code WITH GCC to the point where the output was compiled BY GCC.