I spent a good chunk of yesterday fighting this, but it's not just modelers. I boiled it down to two things:
Monitoring. It's what makes the sound that hits your ears and it can either mask or exaggerate some issues in the tone.
I was trying to practice with headphones and dug out some of my headphones in my "stuff I need to sell" pile. Almost all of them sounded really bad because they either were really muffled or made the high end sound nasty. I find headphones can easily over-exaggerate some midrange or high end frequencies and that leads to endless tweaking. So I grabbed a pair of cheap Audio Technica M40X headphones that don't sound all the great for music, but they just kind of let the guitar tone exist in a boring way, and that helped me to focus on just getting something in the ballpark and start playing.
But it's not just headphones and monitors, it's real speakers too. I've been using an Orange 1x12 cab with a solid state power amp quite a bit this year to avoid the IR rabbit hole. It sounds really good with my modelers most of the time, but I don't like my Rocker 15 tube amp with it and high gain can be a challenge. I finally swapped out the Vintage 30 for a V Type I had sitting around, and that basically erased most of the nasty papery high end I was hearing.
Too damn many options. This is 100% my fault and I need to executive a massive gear purge.
This weekend I've been trying to work through more Truefire lessons and learning some new songs, and the absolute worst thing is gear getting in the way. My QC patch wasn't sounding great so I plugged into my tube amp. That also didn't sound great, so I plugged in my Friedman IR-X. That worked better, but then I had to figure out why the other stuff didn't sound good. And that led to the above monitoring changes, which helped quite a bit.
But seriously, I have way too much stuff and way too many options. I really just need to get rid of all the excess, extra modelers, extra guitars, pedals I don't use, all these extra headphones, even extra studio monitors.