Yes and no. Because in case RTL is 5ms, using a loop (splitting up the digital path of course) would already result in 10ms. And at least for me (plus for quite some other folks I know), this is when things start to feel sort of distracted, at least when using headphones. Using 2 serial loops would be pretty much impossible already. With the GT-1000, even with both loops set up serially, RTL (at least without any blocks adding their own latency - no idea about which those would be in Boss land), we'd still be kicking at 3ms (or even lower, according to Leo Gibson - I rounded his 0.7ms up to 1ms), which is just fabulous.
Now, if you never use digital FX loops, if you never run your signal into another digital device, into an additional digital mixer and/or IEM system, any DSP controlled monitors, all that while using a guitar cable instead of a wireless system - sure, in that case, even 5ms is a decent value.
But personally, I'm constantly running into various scenarios where it's not possible to avoid some of these things, sometimes in combination.
So, even in case you're not Steve Vai (which I certainly am not), it's not exactly too uncommon running beyond the "uh, that's not too much fun anymore" point.
It's the reason why I always bring my own little mixer with me on IEM jobs, so I can at least monitor my own signal without any irregularities (and believe me, I've been there multiple times... FOH folks apparently aren't always the brightest candles in my neck of the woods). And it's also why I rather start with the lowest possible latencies. It's also one of the reasons I kept using Logic, because at least for a long time it's been the only DAW where buffer sizes were pretty much a "set and forget" thing (even if you dialed them in pretty low).