Answer
Wa `alaykum as-Salamu wa Rahmatullahi wa Barakatuh.
In the Name of Allah, Most Gracious, Most Merciful.
All praise and thanks are due to Allah, and peace and blessings be upon His Messenger.
Dear sister, thank you for your question and for trusting us.
It goes without saying that marriage must be built on mutual agreement. In Islamic marriage there is a main condition of the contract which is that of offer and acceptance (ijab and qabul in Arabic). If this condition is not met, then the marriage contract is invalid. Also, there must be consent from the prospective spouses, namely the bride and the groom must be satisfied with the partner he/she will live with the rest of his/her life.
As for this bothering man you are talking about, you should be tough with him declaring clearly that you will not accept him as a husband, nor your family will accept him as an in-law.
If by “Baddua” you mean bad supplication (du`aa’ in Arabic), do not give much attention to such threat as long as you have not not caused any harm or haven’t been unjust to him.
Moreover, if he still bothers and disturbs you and your family either at home, by phone calls or the like, you should stop him by law seeking help from your local authority in charge of such matters.
Allah Almighty knows best.