This must be so so stressful for you. I feel distressed just reading about your dilemma but to live with your mother calling multiple times a days and her ongoing
manipulation seems overwhelming. It sounds as if you are also having to deal with other family dynamics without everyone on the same page.
My immediate reaction is the same as fellow-responder Elaine who suggests using caller ID and screening your calls. From years of experience, I can report this works exceedingly well. In many cases the behavior has actually been reversed after a number of calls have been placed with only an answering machine to respond.
Do not expect her 'controlling and manipulative' personality to be any different with Dementia than it was before the cognitive changes occurred and chances are the anxiety will also remain; you may actually see an exagerration of her former personality traits.
The diagnosis of a dementing neurological process leaves her incapable of controlling many of her actions. If she is not able to be in control, then you and your siblings need to be in charge and to act accordingly. This role-reversal is difficult at best but take heart because it may be short lived once you begin monitoring the phone calls. The fact that she is being moved to a more secure unit certainly underscores the staff agreeing with your assessment; the dementia is progressing and her skills are failing. There is a distinct possibility that phones will not be allowed in her new residence. Hopefully, she will also be kept busier with activities and enrichment programs and have less time to be making phone calls. It is often boredom and not need that leads to this kind of repetitive behavior. Keeping the person with dementia occupied frequently negates the aberrant behavior.
Meanwhile you need to save your own sanity by not allowing her to still have control over your life. Do take care of you!