Kemper Profiler MK 2

[LP59]

The issue isn’t just using LUFS, it’s that null tests are useless besides from telling you if something is identical or not.

My example above shows files sounding identical, but which have underlying differences that result in a poor null. How much it nulls doesn’t really tell us anything useful at all, especially in this context. There is literally nothing of value, Leo continually using this methodology and (even worse) framing it as “scientific” needs to stop.

Of course there are factors that can subvert a null test. That's why it's important to do the test properly to eliminate those factors. That obvious statement is true of almost any kind of test in almost any discipline. Done properly, the test will have little influence from those factors. Further, in real world tests between amps and amp sims that sound similar, those factors tend to be minor anyway.

The bottom line is his Kemper test matches quite well with subjective observations. That doesn't mean it's perfect, but it does mean it's hard to dismiss that kind of independent confirmation of his methods.

 

[Orvillain]

Of course there are factors that can subvert a null test. That's why it's important to do the test properly to eliminate those factors. That obvious statement is true of almost any kind of test in almost any discipline. Done properly, the test will have little influence from those factors. Further, in real world tests between amps and amp sims that sound similar, those factors tend to be minor anyway.

The bottom line is his Kemper test matches quite well with subjective observations. That doesn't mean it's perfect, but it does mean it's hard to dismiss that kind of independent confirmation of his methods.

Right, so you're not even going to engage with the obvious demonstration of a principle that flies in the face of everything you've been saying. Cool.

 

[MirrorProfiles]

Of course there are factors that can subvert a null test. That's why it's important to do the test properly to eliminate those factors. That obvious statement is true of almost any kind of test in almost any discipline. Done properly, the test will have little influence from those factors. Further, in real world tests between amps and amp sims that sound similar, those factors tend to be minor anyway.

I just used an extreme example to prove a point, but imagine there isn’t just one phase anomaly, but many across the entire frequency domain. Phase is one of many, many characteristics of a sound.

The issue really is that null tests can’t convey anything about the differences we actually perceive. The phase rotation in my example above is sonically benign - no one is going to claim that it doesn’t sound accurate despite the terrible null cancellation. It’s quite a massive skew in the null from something sonically benign - what about things like harmonic information which is something our perceive in a way that we can’t really quantify in null tests.

There are controlled environments where they can provide useful information in certain circumstances. Youtubers posing things as scientific test methods ain’t it.

 

[Orvillain]

I put the test on YouTube too, but used a different source file and shuffled them up (so if anyone cheats by saving the previous files, they'd have to get this right too to prove me wrong):



What % difference is -infinity and -12.6 LUFS?

I made the mistake of looking at the comments. These people are idiots. They pose as if they understanding the principles at play, but they really do not.

Even something as simple as not understanding the difference between time alignment and phase alignment .....

 

[MirrorProfiles]

I made the mistake of looking at the comments. These people are idiots. They pose as if they understanding the principles at play, but they really do not.

Luckily I screenshotted Francois now removed guess, which he claimed sounded VERY different to the others

 

[Orvillain]

This discussion and the stuff posted here in the last few days has been a final-total-eye-opening-confirmation for me on this whole "null-testing" issue.

 

[LP59]

There are controlled environments where they can provide useful information in certain circumstances.

I agree completely. In my experience with real world cases of amps and amp sims that sound similar to begin with, you can get useful information when the test is done properly. It's not perfect, but it can provide useful information. Empirical evidence of this abounds.

Youtubers posing things as scientific test methods ain’t it.

There are undoubtedly cases where this is true. However, that's a pretty broad generalization. You'd have know methodology details of a specific case before you could either dismiss it or approve of it.

 

[MirrorProfiles]

I agree completely. In my experience with real world cases of amps and amp sims that sound similar to begin with, you can get useful information when the test is done properly. It's not perfect, but it can provide useful information. Empirical evidence of this abounds.


There are undoubtedly cases where this is true. However, that's a pretty broad generalization. You'd have know methodology details of a specific case before you could either dismiss it or approve of it.

These null tests comparing LUFS residuals tell you precisely nothing about what differences might be benign/inaudible, which ones are in a frequency range that IS audible but not critical, or areas where it’s very obvious. It also tells you nothing about harmonic information. There’s simply too many things skewing it which can either have a big result on the residual, or a nominal effect, and there is no way of discerning the audible perception of those differences.

It’s just not a sufficient way to test. Using my example, merely listening and comparing tells you more meaningful information than listening to the residual. And even if you flip the phase and list to what’s left - then what? Even the differences don’t really mean much, unless you know already that it’s because of phase rotation (and not something else).

 

[laxu]

RE: Accuracy.

To my ears, the very first thing I tend to spot is rolled off or attenuated high-frequencies. A lot of these machine learning models do that on purpose, because that frequency range is a lot more prone to noise errors.

The next thing I tend to spot is things being undergained. I'm not always able to spot this with big wide open chords. But when I jump between palm mutes and big wide open chords, it becomes quite apparent.

Same here. When I made Tonex captures of my Mark V, the thing that made them sound off was that they had too much gain until I adjusted the global trim down by about 4 dB. Then they behave quite close - if amplified through the same poweramp (Fryette PS-100).

I think it's easy to tell if any of the big ones are off - lowest lows, high end and gain. But once you get into the midrange it can be much harder to figure out what exactly is off, you just know it's not right.

I kept being annoyed by a cocked wah style midrange thing going on the Modern channel of the BluGuitar Amp 1 Iridium Edition. None of the other channels has it, it's just voiced that way. Turns out that simply by enabling the boost feature the issue goes away and I just hadn't tried it because it was already very high gain. But it bugged me enough that I just avoided using the channel.

I literally never evaluate any of this stuff using lead playing, because I find it harder to tell the difference between the real amp and the model under such conditions.

Same. My go-to is having a moderately driven sound with plenty of high end on a bridge pickup, playing chords. I think pristine cleans and lead playing can often hide some of the issues, or just ends up sounding pleasant to the ear. Like I'm not going to be too unhappy if a totally clean sound isn't biting my head off even if the real amp does.

 

[Orvillain]

These null tests comparing LUFS residuals tell you precisely nothing about what diffetences might be benign/inaudible, which ones are in a frequency range that IS audible but not critical, or areas where it’s very obvious. It also tells you nothing about harmonic information. There’s simply too many things skewing it which can either have a big result on the residual, or a nominal effect, and there is no way of discerning the audible perception of those differences.

It’s just not a sufficient way to test. Using my example, merely listening and comparing tells you more meaningful information than listening to the residual. And even if you flip the phase and list to what’s left - then what? Even the differences don’t really mean much, unless you know already that it’s because of phase rotation (and not something else).

And what pisses me off more than anything else is, we've provided so much evidence and logic to support our position. Last time we did this, all I got from Mr. 59 was "MUH GPT BRO!"

 

[Chocol8]

=> Literally 0.8 LUFS / %2.3 difference.

Complete bullshit.

 

[Chocol8]

The LUFS measurement needs to be integrated. If you do that, it measures the residual in a way that provides useful information about profiler accuracy.

For anybody in doubt on this matter, just look at Leo Gibson's latest null test results. They show that Kemper improved from V1 to V2 and is now in the ballpark of IK and NDSP. That's entirely consistent with what people are concluding from subjective tests of V2. That serves as independent confirmation of Leo's methodology.

Good god no. Like literally everything you said here is wrong.

 

[Chocol8]

More or less, I think We all perceive NAM as the best quality capturing tech, followed by ToneX and then QC (Well, I have to say that currently I like QC more than all of them, but couldn´t certify that, since it´s a personal perception).

I don't agree. I need to get my hands on a QC again, but I think Tonex V2 and NAM are very close where you might prefer one over the other in different cases, but you are splitting hairs either way. Both have only slightly audible differences from the control amp. QC v2 just might be a slight touch better actually, and I think there are plenty of other people who feel the same.

BUT, it is ALL personal perception. The "scientific" testing is not scientific, nor even done well. There is no simple way to compare and rank, because the factors that matter are going to be different for different players. Is one adding a slight bit of compression better or worse than another that cuts a tiny bit of highs? That's ALWAYS going to be subjective.

 

[Orvillain]

I've actually not tried the QC v2 captures. But I really like what I've heard.

 

[LP59]

These null tests comparing LUFS residuals tell you precisely nothing about what differences might be benign/inaudible, which ones are in a frequency range that IS audible but not critical, or areas where it’s very obvious. It also tells you nothing about harmonic information. There’s simply too many things skewing it which can either have a big result on the residual, or a nominal effect, and there is no way of discerning the audible perception of those differences.

That's the purpose LUFS serves. It highlights discrepancies that are perceptually significant. Inaudible differences don't matter when, strictly speaking, for purposes of comparison, all you care about is the perceptual magnitude of the discrepancy. We're talking about comparing amp sims to amps where they already sound very similar. Factors that might fool you into misleading results in that narrow, but real-world, case tend to be minor.

I appreciate your thoughtful reply, but the general tone here from others is getting too toxic for me, so I'm out.

 

[Orvillain]

I appreciate your thoughtful reply, but the general tone here from others is getting too toxic for me, so I'm out.

Because you don't have the correct information or knowledge to even be a part of this discussion. You're out of your element Donnie.

I've explained it every way I can think of. If you don't get it, then you're not smart enough to get it. It is that simple.

I've given you so many thoughtful replies. Remember last time when you just accused me of being a GPTbro ?? No. You don't get off the hook that easily. Your tone policing can fuck itself right in the bumhole.

 

[MirrorProfiles]

That's the purpose LUFS serves. It highlights discrepancies that are perceptually significant. Inaudible differences don't matter when, strictly speaking, for purposes of comparison, all you care about is the perceptual magnitude of the discrepancy. We're talking about comparing amp sims to amps where they already sound very similar. Factors that might fool you into misleading results in that narrow, but real-world, case tend to be minor.

I appreciate your thoughtful reply, but the general tone here from others is getting too toxic for me, so I'm out.

No, LUFS is a measure of perceived volume and dynamics. It’s for getting TV shows on netflix the same ballpark volume, or streaming songs on spotify.

Again - look at my example of something with a poor null residual but an exact frequency domain, dynamics and volume match. The LUFS don’t tell you anything about what the differences are and aren’t. And they weren’t designed for these sorts of comparisons, it’s for broadcast.

 

[Chocol8]

It highlights discrepancies that are perceptually significant.

Wrong. It doesn't matter how many times you say it, it will never be correct. You don't have the faintest clue what you are talking about and it's really painful to watch. Just stop.

 

[BrownNote71]

No, LUFS is a measure of perceived volume and dynamics. It’s for getting TV shows on netflix the same ballpark volume, or streaming songs on spotify.

It's super useful for balancing backing tracks. Until we started using LUFS we really struggled as a few backing tracks would end up being way too loud or quiet - sometimes it was masked by a specific venue's PA/room and then at the next show the loudness differences would be very evident.

 

[Orvillain]