What you have could be two things, judging from your description of it. It could either be a lipoma or an epidermoid cyst. In which case the definitive treatment is surgery
If it really does bother you and is causing issues, the best course of action would be to see a general surgery or have your primary care physician
refer you to a general surgeon
for evaluation. Both lipomas and cysts can be excised easily in the operating room.
Lipomas are small fatty lumps that are located under your skin. When they are excised (or taken out) they usually come as very well differentiated small masses. To take them out, a surgeon would make a small incision over the lump, and then simply gently push it and cut it out.
For a cyst, it's a bubble with keratin located inside of it. This is what it sounds like you have. For these, they are also taken out the same way - by a general surgeon. However, the borders of the cyst wall have to be clearly defined, because if they pop you have a high chance of recurrence. It sounds like you popped it yourself, and hence had it come back.
Even if you do undergo the surgery, both types of masses have a tendency to recur. For lipomas they can pop up any where on your body. For the cyst, there is a high chance it will come back at the same place even if you have surgery. However, by having the surgery done you can be sure it's not anything serious, and the surgeon will do their best to excise the whole thing so that there will be a minimal chance of recurrence.