I would agree with your assessment - that an allergic reaction to one of the medicines a very plausible explanation. Strokes can cause weakness of the tongue, but not swelling.
The one exception to this is if the tongue is sticking farther out of the mouth than it had before, and therefore looks "swollen", even though it is not truly any bigger.
I think you are absolutely correct in wanting the medications changed first before going with a surgery. I would identify which medications have been started since the stroke, and try to change them one at a time to a different option. Virtually all medications given for stroke treatment have an alternative. I would go through them one at a time with your physician and have them changed (probably best to change only one at a time so you know which medication was the culprit). If your doctor does not wish to proceed with this plan, I would get a second opinion.