Most women have breasts that are not completely symmetric. This can be for a variety of reasons, but usually they are not different enough for anyone but you to recognize. Breasts can change in size for a few different reasons. The most common reason is a change in hormones in your body. An increase in estrogen for example will stimulate gland formation in your breasts which will make them bigger. This is what happens when women get pregnant and the increase estrogen from the pregnancy increases their breast size. Breasts do contain a lot of fatty tissue. Thus if you lose weight, some of that weight will come from your breasts. It is unlikely that only one of your breasts is losing tissue while the other stays the same size. Likely, their reduction in size has made any small asymmetry more obvious to you making it seem that one of them is getting smaller.
Anytime you feel that there is a change in your breasts, you should make sure everything is okay by visiting your doctor
. In this case, I think it is fine for you to visit your primary care physician
. He or she can perform a simple breast exam and make sure that there are no problems.