Living with Down Syndrome

I am a mom of a 4 yr old boy with Down Syndrome and I have been in a lot of stress and feelings of guilt when I can't keep up with doctors appointments, therapies, teaching my son new skills, etc. I have been living with depression and panic attacks since he was born. I love my son but I have to admit I tell myself I should have stuck with one child (my daughter, now 8). Hearing a lot of stories from parents who adore their kid/s with DS, I look at myself with guilt and shame because I do not feel the same way. I am dealing with behavior issues that maybe I wouldn't have to deal with if he didn't have DS. The constant whining and yelling when he gets frustrated because he can't say what he wants, the running off and chasing game whenever he gets the chance, the so many things he can't do yet that I have to constantly do for him, the stressful potty training, the stressful and chaotic visits to the library, stores, parks and going out for walks, the stubbornness and defiant behavior... all get to me and causing me to not look forward to being with him especially in the summer when he's off from school. 
I love my son and I can't imagine life if I lose him. And yet, I know when he's not around it gives me so much freedom and peace of mind because I do not have to deal with stress and guilt. This boy needs me but I don't feel whole to care for him. I feel I start falling apart the longer I spend time with him. 
How can someone so special and so fragile cause me so much stress, grief and emotional pain? What can I do to keep myself whole and ready to face the world with him? How can I show him that the world is a beautiful place to live in when I feel exhaustion and my stomach starts to churn whenever I look at him? 
This is a lonesome journey and I am very weary.