First of all, congratulations on taking some steps toward a healthier diet! Eating a health, balanced diet is an important part of maintaining your health in the long term.
In order to answer this question, it would be helpful to know if you have been diagnosed with diabetes by your doctor
or if you are simply trying to make some improvements to your lifestyle. If the former, then I highly recommend that you coordinate all of your plans with your primary care doctor
or the doctor who is managing your diabetes. Controlling blood sugar when you are diabetic is complicated, and can involve a mixture of diet and medications. Your doctor will determine if you need medications and you can see a nutritionist
to give you diabetes - specific nutrition and diet education.
If on the other hand you are simply just trying to eat more healthy food, then cutting out candy, soda, and other sweets is a big first step. You should also add healthy foods to replace them (fresh fruits and vegetables). You may also want to moderate your carbohydrate intake (white breads, pasta, potatoes, etc).
If you have more questions about this topic, make sure to mention them to your primary care doctor.