Should I stop him from coming home?

My husband has been in 24 hour care for 6 months and is improving because of care. He now wants to come home and has been insisting on 24 hour visits home. He got two three-day stints over the holidays. He still cannot come home as he is just too frail and it will drive me to elder abuse. Should I stop the overnight visits? As giving him that kindness may cause more hurt. I can't have him come home and the thought of it makes it hard for me to breathe.