Is it possible to connect my home phone line to my cable modem through my router? Does that even make sense?

So I'm not sure how stupid this sounds, but bare with me. Just went through buying a new modem and router and setting them up with the cable company and my house, yadya yadya. But my dumb ass completely forgot that we need to connect the modem to an RJ11 port in our wall in order to use our landline. Which our new modem doesn't have. I'm trying to figure out possible alternatives with what I have before ordering a new modem.

So what I'm hoping would work would be to find some RJ45>RJ11 connector, plug that into my router, run a line from that to my phone jack in the wall, and get home phone service that way. Is that completely insane? Is what I want to do even possible? I'd like to not order another modem, but I'm getting pretty sure I'll have to. I'm only half as smart as I need to be for this, so this is technically modem #3 and this process has taken over a week.