He is probably worried that he he saw that child he would want to hold him longer and not let him go...as in his previous life he was not allowed to see his babies and was directly taken to the cabin ....so he is now afraid that he would have to go to that cabin