These sorts of actions are not at all odd or even uncommon. I hear about parents worrying about these sorts of issues virtually every day.
Firstly, I applaud you for not punishing him or making him feel bad about his habit, as disturbing or even socially inappropriate as it might be. Not only will it make him feel bad, but it might further encourage the activity. Some above have stated that this tack is not the best means of altering behavior, but the studies I have read have shown otherwise.
Your ped. is correct that it likely a phase -- but you probably also want to know WHY he does it. It most likely came about through self-exploration...something that we all did at that age. (Maybe not the rectum, but we all, as humans, have the natural instinct the explore...first our bodies, then external events.) He noticed that there was a strange smell on his fingers after the exploration, and now finds that smell different and fascinating. He will likely eventually grow bored of the smell and stop doing it.
If, however, you are concerned about the health and/or social inappropriateness of the act, a little nonconfrontational education might be in order.
Show him examples of various diseases that can come from the consumption of feces...pictures of children with hookworm, roundworm, amoebic dysentery, giardia, etc. It might also be good to show him actual pictures of what these parasites look like. Those, alone, might dissuade him from rubbing his rectum and putting his fingers near his mouth. If he starts again, show him the pictures again, and restate the sorts of symptoms and problems that can come from these diseases. Tell him that you aren't angry at him, you just don't want to have to take him to the hospital.
Good luck, and again don't panic. Nothing about his actions are at all strange!