I am dads caregiver

I have health problems of my own and my dr suggested I get out and get exercise and change my diet. I joined a gym and changing my eating habits from bad to good. My dad fusses about being alone but he is okay. I have to take care of myself so I can take care of him. He makes me feel really quilty but I am going to do what I have to do anyway. I am trying to be patient and not get upset just because he gets upset. That would not be good for either of us. Am I doing the right thing?