Portuguese has the expression "modéstia à parte" (literally: "modesty aside"), which is used to (still rather bluntly) introduce statements where the speaker praises himself. Is there something similar in English?
Most translations here actually miss the mark: "without being immodest" and "modestly speaking" actually have the opposite meaning. "Not to say modestly" seems to be the better one, but I don't know how idiomatic that is.