If you are speaking about hugging your wife, you should avoid it while fasting.

Aisha, the beloved wife of the Prophet, peace be upon him, was asked whether one is allowed to kiss his wife while fasting? She said, ‘no,’ then when someone pointed out to her, ‘Didn’t the Prophet, peace be upon him, kiss you while he was fasting?’, She replied, “Who among you can restrain himself as the Prophet, peace be upon him, was able to?” (Ibn Hibban)

In other words, we should avoid such actions to save ourselves from breaking our fast.

I do not see any difference between hugging and kissing; it is likely to arouse desire, and therefore should be avoided unless someone knows that he can control himself.

As for hugging the members of the opposite sex — who are not related to you in blood or marriage–, one ought to avoid it all circumstances.

Almighty Allah knows best.