I done effed up!

I made a pyro, witchcraft, geo mage with dual wielding wands and a archer. I'm at level 3 and 10 hrs into the game (I did a lot of story) and I'm made one big mistake. Instead of putting 2 points into dual wielding I put them into two handed and one point into wand! Ahh!

What should I do now? Should I restart or just keep going and take the loss of points? Any suggestions?