I am uncertain about the answer to this question without knowing where you seem to notice the size discrepancy between the sides of your body with the weight gain and loss.
Generally speaking, most healthy adults will have small differences in the two halves of their body. This is especially noticeable in women in the breasts, where it is completely normal to have one breast that is smaller or differently shaped than the other. In this context, therefore, weight gain or loss on tops of these underlying differences might be exaggerated.
There is also a rare condition called hemihypertrophy, in which one side of the body is bigger than the other. In children this can be associated with certain cancers, but in adults it usually is not. Typical signs of hemihypertrophy include having one side of the tongue or face bigger than the other, and this can proceed all down the body, with one arm or leg appearing bigger or more muscular than the other.
I suggest that you set up a visit with your primary care doctor
. A good physical examination is in order. If you are otherwise healthy, and if nothing serious turns up on the exam, then you probably do not need to worry