@SveiNgthk him being a part of the previous harem of the princess had already made his reputation plumet in regards to the other knights, if he were join the said harem again it will result is people gossiping and sieger's reputation being harmed further potentially facing rumors, disrespect and knowingly being left out of the other knights out of hate or disgust