@Similew pretty sure it's some severe cases of inferiority complex (like in that one chapter where Kae got drunk and asked Nanami about their promise). plus she got some bad experiences in dating, so she's probably thinking all these women that have held feelings for her are too out of her league and would probably some day get tired of her or something
