These A/B listening tests are always difficult to do in a meaningful way. The informal result on the show, however, is consistent with something I've noticed before. And that is that people are much better at being able to tell when a switch is made than which mode has been switched to.

I remember reading about an audiophile listening test where the golden ears were told they were going to do a tube versus solid state amplifier comparison. What they weren't told was that both A and B were the same solid state amp, but one mode had a tiny amount of 60 cycle hum mixed in. The golden ears preferred the one with the 60 cycle hum and identified it as the tube amp.

(Obviously this is irrelevant with regard to guitar amps)

Galanter, that test demonstrates that if you lie to people about the choices they have (tube/transistor, security/terror), and then appeal to their vanity by giving a false sense of authority or significance to their decision-making (you can tell the difference, I will listen to your opinions), then you can pretty much control the outcome of their decisions.

You do this by playing to their prejudices and giving them utterly irrelevant though somehow "connected" information (hum/aluminum tubes).

When you lie to people, it is to create circumstances you desire, not to establish a reliable truth about something else. This is true for just about every endeavor from police interrogation to poker.

In case you are wondering, the non-bullshit way to run the test you describe is to actually do the test described (let people listen to different amps) and see if they prefer one or the other. That won't make a general point aobut tubes/transistors, but it will make one data point regarding those two amplifiers.

i just watched this thing and the only thing that sticks out to me is that i'm kind of offended that i was not informed that a hot asian invasion would be taking place at EA without me being present.

andyk

galanter wrote:These A/B listening tests are always difficult to do in a meaningful way.

Not really.

The only way to do this--the ONLY way--is an A/B/X blind test, preferably one in which the listener controls the switch.

You record an analog master and a digital master of the same thing. Say, a grand piano being played in a room, direct to two-track. Or whatever. It could be the Stooges playing in a room, direct to two-track. Something you don't have to mix. Something with sections that are repeated, so the listener can review similar source material w/o rewinding.

Ideally, you take these masters and sync them by striping the analog master with time code, but you can get away w/not doing this if the sections repeat often enough.

You match levels (L vs R and overall levels) until each master plays back at the same level.

You put a dude in a room.

Dude has a selector box (Box One), with a three-way switch on it. Left is A, right is B, center is X. Switch can be three buttons, whatever. People make this shit for speaker selection, and you just wire one up for this purpose.

Selector box is fed by three separate A/B selector boxes. Those boxes are fed by analog master on one side and digital master on the other.

If you want Left to be analog, set the selector box feeding Box One's A to the analog side. Set the selector box feeding Box One's B to the digital side. Then set the selector box feeding Box One's X to one or the other--either analog or digital.

It takes a few seconds to go click, click, click.

Now the dude has analog on the A side, digital on the B side, and one or the other in the middle. But he only knows that A and B are the two things, not which is which.

The test is:

1. Can the dude tell which of A or B is X?

2. Does the dude prefer one to the other?

You have the dude do several trials of this, ideally w/different source material and breaks to rest the ears whenever he wants them.

This is how makers of professional audio equipment test their gear, at least the ones that pay any attention to stuff like this. Unfortunately, hardly anyone else cares enough to do it right.

Any recording engineer I know could tell the difference between analog and digital in such a test. I could tell the difference. No problem.

Would I prefer one to the other? Dunno. Probably not if the digital master was 24bit. If it was 16bit, I'd bet everything I own that I would prefer the analog master.

I've played something like this game at Abbey Road, switching between a half-inch analog master of our record, a 24bit/96kHz transfer of it, and the 16bit/44.1kHz transfer of that. I asked the engineer to switch when I wanted him to do so, w/o telling me what was what, of course.

The 24bit transfer, pretty much impossible to tell the diff between it and the analog master. I could get it based on, like, a slight difference in the sound of the noise floor or something, but I regard that as a trivial distinction and more of a parlor trick than hearing a real difference.

The 16bit transfer sounded like fucking dogshit compared to either of the other two. It was agonizingly easy to hear the difference.

Quad did a great test one time--they were getting shitty reviews of a new solid-state amp, and undoubtedly they'd already tested it thoroughly enough to know it performed very similarly to their other things. So they set up a blind test of this new, maligned amp and various of their highly regarded tube amps. Invited all these audiophiles to pick out the shitty amp from among the good ones. I want to say it was, like, 22 people all told.

There was, like, one guy who picked up a faint something-or-other in the high end response of the tube amps, and he latched onto that as a tell, which helped him do OK in the test. No one else could tell what was what.

To me, it's about methodology

Couldn't agree more. The band I'm in recently recorded an album with a local engineer who works on digital. We went that way because he's a friend of the band, has a huge famous studio here in seattle, and gave us a deal. In no way do I regret it, because we're broke and he's a great guy and gave us a deal. If I hit the lottery I'll be able to make a record exactly how I want, which will involve a plane trip to Chicago.

Anyway, when we were doing mixdown he came up with this idea about how to shuffle around one of our songs. He was very respectful and it wasn't necessarily the worst idea in the world, so we listened to it. The thing was, we were exhausted after a couple days of recording and listening to the tracks over and over and really couldn't make up our minds. I ended up insisting we stick with the original because that's how we've been playing it.

This wasn't a big deal because it was all among friends, but I wonder how that situation would have played out if we were young and irrational and there was a money person in the room.

steve wrote:Galanter, that test demonstrates that if you lie to people about the choices they have (tube/transistor, security/terror), and then appeal to their vanity by giving a false sense of authority or significance to their decision-making (you can tell the difference, I will listen to your opinions), then you can pretty much control the outcome of their decisions.

You do this by playing to their prejudices and giving them utterly irrelevant though somehow "connected" information (hum/aluminum tubes).

When you lie to people, it is to create circumstances you desire, not to establish a reliable truth about something else. This is true for just about every endeavor from police interrogation to poker.

In case you are wondering, the non-bullshit way to run the test you describe is to actually do the test described (let people listen to different amps) and see if they prefer one or the other. That won't make a general point aobut tubes/transistors, but it will make one data point regarding those two amplifiers.

I would never have thought this topic could be politicized. How silly of me.

FWIW "Lying" to subjects about the nature of the experiment is not uncommon practice in experimental psychology, including experiments having to do with perception. Within certain constraints it is considered ethical and at times essential to getting a valid result.

And to clarify my previous comment about difficulty I was thinking of things like matching levels with precision, the effect of any switching noise as sources are alternated, making sure the test is truly blinded, listener fatigue, and quibbles about impedance matching, signal paths, converter quality, etc. ... most of which tmidgett mentioned.

Seeing that dude talk about the horrors of analog recording was eerily similar to a Fox News hit piece segment opposing a reasonable claim by smearing it with 50 false reasons to be scared of it. They would have had to do an equally lengthy segment to debunk all that bullshit.

Also its hilarious that the quintessential analog vs. digital test was orchestrated by Digidesign, and was as useless and poorly designed as most of its products.

tmidgett wrote:

galanter wrote:These A/B listening tests are always difficult to do in a meaningful way.

Not really.

Tim -

What you describe below makes sense. But it is difficult.

= Justin

Is this Ken Andrews' bit some sort of a deadpan joke? The entire time he was going on about tape losing a bit on each pass, I found myself thinking "Has anyone told this guy there are companies who's bread and butter is recovering data from failed hard drives?"

this reminds me of ADAT. trying to track a song and get the vocals overdubbed in few enough passes that the tape was still readable and the interpolation light (a little sun-looking thing, yes?) wasn't constantly on. man the evolution of digital recording has been fun, huh?