Humans are Giants that undergone neoteny

I was thinking about this idea ever since I heard of Axolotl's being neoteny forms of Salamanders.

Neoteny is where a adult creature maintains juvenile features into adulthood and I thought that it would be a awesome idea to do for Giants, because I've seen stories and ideas of Giants being parental towards humans or seeing them as immature. I've also seen stories of humans once being giants so it connects nicely

The only thing is.. I'm not sure about Giant-folk and uhm.. Half-Giants (Like goliaths from DnD). Would that be weird?? Or am I just making it weird.