This is a little hard to answer without having more details, but a few things come to mind. Have you tried an honest and direct talk with your mom's doctor,
expressing your observations about her prior experience in rehab? Are there other family members who can join you at this talk, offering their support? Your mom's doctor will need medically viable alternatives to a rehabilitation facility, such as adequate in-home care, or a different facility or nursing home. Maybe you can do some research for places where you think your mom might have a more positive experience, and offer these as options. You should include your mom in these discussions as much as possible. Legally, the decision is hers and the doctor must honor her wishes. If you, your mom (and any other involved family members) present her doctor with a reasonable option to the rehabilitation experience she had before, one that makes medical sense, the doctor should take it seriously. This should include staying at home, with necessary care.
One more thing: You don't say if medical insurance is playing a role in the doctor's decisions, but he may be limited to certain rehabilitation facilities by your mom's medical plan. It's also possible her health coverage won't fund in-home care. It will be helpful for you to know these things as you look for alternatives. If, for example, your mom's medical coverage won't pay for in-home care, can the family pay for this? (In some states, Medicare allows family members to be paid in-home care providers.) As for the choice of rehabilitation facilities, some insurance plans have wiggle room with this, but you may need to push for your preference.
By the way, give yourself a big pat on the back for advocating for your mom during this time of her life.