That being said, is this phrase considered ACTUALLY an vulgar phrase or at least too harsh for young children, or is this just my viewpoint?
As already commented, standards of rudeness are both cultural and personal; but here's the BBFC (the UK film censors) taking on exactly this phrase:
U stands for universal.
It means films that anyone can watch, including children as young as
four.
...
In a U rated film, you might hear infrequent use of very mild bad
language.
Here’s some examples from well known films.
...
Onward
[rated U for] mild threat, very mild bad language
Onward is an animated fantasy adventure in which two elf brothers
attempt to bring magic back to their world from which it has
disappeared.
...
What's the bad language?
"You act like you know what you're doing, but you don't have a clue!
And that's because you are a screw up... And now you've screwed up my
chance to have the ONE thing I never had!”
What we say:
In Onward, the term screw up is used in place of the word “messed up”,
and the phrase is not used as a substitute for strong language, or
used in a sexual sense. In this context, this word is acceptable at U.
However, if the character directed the phrase at another character in
an aggressive or angry manner, then this could result in a higher
classification.
My emphasis in the last paragraph. From A Parents' Guide To Language on the BBFC website.