No you're not wrong I think Inseo might feel guilty because he might've accidentally outed Yi-Hyeok's mom to herself which likely inspired a crisis and the mom left the family to possibly avoid shaming her son or causing him trauma. Since she never said anything Inseo likely internalized the event which makes him feel guilty for loving Yi-Hyeok when he feels he might've caused his loved harm due to contributing to his mom leaving. Even if it was an accident that wasn't Inseo's fault he took the guilt which doesn't make him unkind or careless rather he loved Yi-Hyeok that much. Poor guy. I hope Yi-Hyeok's mom understood Inseo meant no harm and that now Yi-Hyeok is suffering due to the love of his life feeling he betrayed him. I'll take a guy with a heart and conscientiousness over the selfish boss by at least a few good miles.