progression and growth is typical for powerscaling shonen, which this manhwa is not. The people around Ed are bosses for the real protagonist of their world (swordsman) to take down. If you follow that one and his perspective, you will have the progression that you are looking for. But from ed's perspective, he knows that the world is a shimmy away from doom. While that swordsman classmate is taking down one boss after the other, Ed is stressing out from changing too many things in the timeline. 

The story is not about getting stronger, it's about keeping the original story flowing and setting all conditions towards the only good ending that Ed knows about.

That's if you treat this manhwa as your typical powerscaling shonen. If you remember the setting, he is always going to be an extra, and instead of enjoying his harem life, he's tied down by responsibility. Every other character has being OP as a defining trait. His job is not to be the best, but to play out the scenario towards the good ending he knows from his past. He's not a carry but a support, his growth relies on the growth of the people around him. 

Either way, from past chapters (exam arc where he dueled that professor), he is strong enough to stand beside them.