Fortunately, sperm have a very short half-life and do not do well under dessicated conditions (i.e. dried). While there are plenty of urban legends talking about becoming pregnant from dried semen, there is no truth to these kind of stories. Sperm can live for 4-5 days in warm, moist conditions such as the vagina. However, once dried sperm will die within a matter of minutes. If you had wiped the semen from your hand onto your vagina immediately after its ejaculation you would indeed be at risk for an unintended pregnancy. However, there is basically no chance that you could get pregnant from semen on your hand as it would have become inert by the time it took you to get home from your boyfriend's house.
That said, it is important that you practice safe sex, preferably of two types if you are concerned for pregnancy in the future. Birth control
and condoms for example are very effective at preventing pregnancies but neither alone is 100% foolproof. The same can be said of any kind of contraceptive method. However, both combined provide near total protection. You should discuss these issues with your primary care physician
if you are concerned with the potential for unintended pregnancy in the future.