Allah never says He is with those who fast or those who pray. He says He is with those who are patient. Allah will never take a blessing from you but we need to improve our relationship with Allah.