To judge a group or a movement, we have to fully understand its origin and basic tents. Ahl al-Hadith  which stands for “The people of hadith”; also Așḥāb al-ḥadīṯ “The adherents of hadith”) first emerged in the 2nd/3rd Islamic centuries as a movement of hadith scholars who considered the Quran and authentic hadith to be the only authority in matters of law and creed. Its adherents are also known as traditionists, i.e. those who adhere to tradition or hadith.

In modern context, ahl alhadeeth is a movement which emerged in the Indian sub content which is primarily based on adherence to Quran and authentic hadiths of the Prophet and basically understanding them in pursuit of the statements and acts of early Muslim generation including the Companions of the Prophet and those who followed them during the first three Islamic centuries. They attempt to fight superstitions and dominance of culture over true and pure Islamic teachings.

According to your message, it seems that your family practices prejudice against ahl al hadeeth. This is the reason they started gossiping to disunite you from this man.

Islamically speaking, as long as you had the contract of marriage which fulfilled its basic requirements, you do not have the right to claim for divorce without a justifiable reason which should be raised to your local judge.

According to your message, he is your husband even if you have not moved to his home. He has all full rights of a husband once he provides you with house and invites you to join and share his life.

You should not listen to those gossips or to tribal slogans which divided Muslims. The Prophet has been reported as saying, “People should stop boasting about their fathers who have died, while they are but coals of Hell, or they will be more humiliated with Allah than the dung beetle who rolls dung with his nose. Indeed Allah removed Jahiliyyah from you, and its boasting about lineage. [Indeed a person is either] a pious believer, or a miserable sinner. And people are all the children of Adam, and Adam was [created] from dust.” (At-Tirmidhi)

Allah Almighty knows best.