He might have been a victim on that night, that's true, but he had no reason to further victimize himself. He clearly disliked everything that was happening but still went along with it. I'm pretty sure In*** was just a phone call away if he needed reassurance. Yet he keeps repeating the same problematic behaviours while still having feelings for another person. One thing is that mf black-haired dude continuously pursuing him, another thing, is SaeIn going to see him on his own two feet and keep at it. He was a victim but he didn't have to stay one. And he's clearly not without friends, seeing as he had one over to his house after he was married that seemed to know everything and was quite protective of him. Unless we are presented with another flashback that shows us he was left without an alternative, yes he is indeed a pushover.