This day in age, any type of adverse reaction to a medicine gets labeled as an allergy. This is both to protect you and to inform doctors
that you may react to a related medicine and to use this medicine with caution. The reaction you had, which appears to be very vague, may or may not have been an allergic reaction. Until proven otherwise, there will not be a doctor that will be willing to take a chance.
The major exception to this rule is if you develop an illness
where penicillin is a much better treatment than any other drug. One example of such an illness is syphilis. When a patient presents with syphilis and has an allergy to penicillin, we weight the risks vs the benefits. In your case, we would give the antibiotic anyway and watch you for a few hours to make sure you did not react to it. In the case of patients with an actual well documented allergy, we would desensitize them to penicillin by exposing them to greater and greater doses until the body got used to it.
I suggest that if you would like to know for sure, that you schedule an appointment with an allergist. He or she can perform allergy testing and determine if you have this allergy. I would only do this if you really want this removed from your record. Good luck.