Well, as you yourself noted, you need to improve your posture. A lot of this has to do with how you sit at the computer. You should look into rearranging your work space with a chair and desk that are at the proper position for your body, or consider an ergonomic chair and workstation. This should help you a great deal.
In addition to this, there are a whole number of back stretches and back strengthening exercises that you can perform. In order to work, these have to be performed consistently every day for a long period of time, as the changes are slow and gradual. Your
primary care doctor can walk you through a list of appropriate back exercises or, if you prefer, you can find them on several reputable health web sites.
Daily aerobic exercise, like you have just started doing, is also a good deal. This will improve your overall health and also your flexibility, which has potential benefits for your back strain, especially if there is a component of stress worsening your back pain, as daily aerobic exercise is great for stress.
If you have further concerns or questions about your back pain, then your primary care doctor is the person you should talk to!