@BLACK PRIMORDIAL NOIR dude you have to wait a little bit more... romance is just a side quest it can wait a little bit more and he is doing right now is the main quest to change the fate of those people.... who will die eventually if he didn't do it... the main quest ...
IF YOUR ABSENCE DOESN'T BOTHER THEM THEN YOUR PRESENCE NEVER MATTERED TO THEM IN THE FIRST PLACE