I gather that the trust belongs to you and three of your siblings, not to your father and three of his siblings.
The terms of the trust, found in the
trust document, control what you and your siblings can do, and cannot do regarding the trust and trust property (the house). Without reading the trust document, I cannot tell if your siblings have the right "sign off their ownerships."
If the trust document doesn't contain any provisions regarding a sibling "signing off," you'd need to find out if state law—presumably, the state where the house is located—has any provision regarding that.
If neither the trust document nor state law has any provision on this, it seems that your siblings should be able to "sign off"and transfer their interest in the trust to you.
I doubt if the house can stay in the trust indefinitely. Trust documents normally specify how and when the trust ends.