This premise seems exactly like the manga "Kyuuseishu ≪MESHIA≫~Isekai wo sukutta moto yuusha ga mamono no afureru genjitsu sekai wo musou suru~" Just change the hero with the summoner, and the beasts with the lover of the Meshia manga.
Now, I won't say that there won't be any difference later on, or that the story is bad. I have only read this first chapter; I can't judge the whole book.