There are a number of ways to treat urgent issues that arise regarding your child's health that do not necessarily require going to the ER. For instance, many pediatricians have on call services so that you can call your
pediatrician or one of their partners or specialized nurses who can go over the concerns you have and discuss what further actions may be required and whether they think you need to go to an emergency room.
The other side of the coin though is that you know your child better than anyone else, and so if there is something that concerns you and you are not comfortable with handling it, then it is not at all unreasonable to go to the emergency department to be evaluated by a professional. Even if they are not as concerned as you are, at least this is a better situation than if you did not take your child to the emergency department and there was something going on that does need emergency treatment. A general rule is that if you are not comfortable or if there is something that has changed or worsened in a short period of time, then it is never a bad idea to go to the nearest emergency department.