One more thing, from ml pov... He only care because that yellow guy his neighbor and close to ml's dead mother.. and after childhood kidnapped incident family members say take care him.. so ml doing take care him that reason.. but yellow guy has his hidden agenda because he misunderstood ml taking care..
Ml still slow to understand his behavior difference boundaries between love and repay help...that why he neglected his behavior toward mc. Which casue that thing
.. in novel tell why ml has yellow guy photo inside album.. (in manga they skip plot) ,ml butler reveled that because when they ml and yellow guy kid.. yellow guy come his home to play and ml mom like him and try different clothes at yellow hair guy.. that picture take from that time. And that photo book belong to ml mother