Based on the information you've presented, you have a most urgent situation on your hands. It's critical to get your mother evaluated by a Geriatric Neuro-Psychologist immediately. Delays might diminish
your mother's prognosis and potentially extend her suffering unnecessarily. While this is not my specialty, it's my understanding that there are some medications and treatments which could dramatically improve your mothers quality of life.
If your mother has remained in an "emotionally and financially abusive" relationship for 51 years, it's unlikely at her age and in her diminished capacity that she will make any dramatic changes in her life at this point, so your opportunities to help her will probably be limited. You also indicate that you're going up against three people who benefit from your mother's diminished capacity. Given the complexity of relationships, there is probably plenty of room for your siblings and step-father to rationalize "leaving well enough alone." If you have to rely on persuasion, or your mother's efforts to change the situation. it sounds like you can't expect much success.
If you believe that your mother is truly being abused, you should contact your county department of protective services immediately. Your Area Agency on Aging should also be able to tell you about elder abuse services in your region; check out our local eldercare directory for contact information. It might also be a good idea to consult a lawyer who specializes in elder law for guidance.
Before you take any of these steps, you need to prepare yourself for the reality that you will probably become a scapegoat: other family members are likely to reject your efforts, and even to limit your contact with your mother. If you go forward you will need a lot of support. You might want to speak to your clergy if you belong to a congregation, or seek out the help of family therapist. Best of luck!