I'm not sure either, maybe it's because the director is making decisions to "repay" him without considering his feelings, because seojin doesn't remember the past, and doesnt want to re-live it either, for him the boss is his family, he loves the director and wants to start a new page with him as seojin not lee junyoung.
@mcp321 He's mad because director is still treating him like the boy from the past, and trying to "fix" all things done to the boy in the past. He's not really 'seeing' the Seojun right in front of Him and that's breaking his & my heart....