Re: Online listening tests and psychoacoutics experiments with large N

[Matthew James Wright <matt@xxxxxxxxxxxxxxxxxx>]

I've also received IRB approval (from Stanford) for my research protocol:

	http://ccrma.stanford.edu/matt/together/protocol.pdf

In fact, they let it be "expedited" and didn't require a formal consent
process, because:

1) There's a certain difficulty in downloading, installing, and running
any software, which weeds out a lot of people who shouldn't take the
experiment.

2) The recruitment email states that subjects must be at least 18 years old.

3) The software has a series of opening click-through screens starting
with a short agreement (which is technically not an "consent").

4) The opening volume adjustment screen starts from zero gain and exhorts
subjects to increase the volume slowly until reaching a comfortable
listening level so as to protect their hearing.

5) The software forces at least 1-minute break every 15 minutes.

6) Subjects can do the whole thing anonymously if they desire.

7) Subjects have an optional final step of actually sending back the
results, so they can bail out at any point in the experiment.

8) There's no deception; subjects (and all of you!) can inspect the sound
files and software.

In terms of the question about controlling for what subjects actually
hear, I'm just asking them to say what kind of sound system they're using,
and giving them full control over volume.

All the software is in Max/MSP running locally on the subject's computer,
so I have complete control of what the subject a lot of flexibility in the
UI and interaction design.  In my case I'm looking at perceptual rhythmic
alignment of various sounds against each other, so the user adjusts the
relative timing (to the individual audio sample if desired) of various
samples in downloaded wav files.  Max generates an HTML file containing an
email message to me that I then parse in Matlab.  I'd be very happy to
collaborate if anybody else wanted to build from any of my (GPL) software:

	http://ccrma.stanford.edu/matt/together

-Matt

p.s. Sorry about my sloppy mischaracterization of (Honing, 2006) as being
the first use of the online listening experiment paradigm!




Dan Ellis said:
> We've done a couple of web-based experiments
> through our IRB.  We were able to get them exempt
> on the basis that the listening tests involved no risk
> to the subjects, that we kept absolutely no personal
> data (indeed, it was completely anonymous), and
> that we had a click-through consent form in which
> they asserted they were at least 18 and we told
> them who to complain to.
>
> We've gotten pretty good response rates (several
> hundred people giving us their opinion on the similarities
> between named pop musicians at the defunct musicseer.org,
> or typing words to describe music clips a la ESP game
> at http://game.majorminer.com/ ), but as to the repeatability
> or control of conditions -- well, point taken.
>
>   DAn.
>
> On 7/2/07, Pierre Divenyi <pdivenyi@xxxxxxxxx> wrote:
>> I see a looming danger, like a tiger hiding in
>> the shade ready to jump on any US investigator
>> actually running online listening experiments.
>> The tiger is called the IRB. I mean, (pray tell)
>> how the world would the investigator ensure
>> protection of the unsuspecting web subject who
>> takes part even in 15 minutes of listening? Those
>> not living under the tutelage of Institutional
>> Review Boards may have no idea what it takes to
>> get approval even for our obviously unthreatening
>> listening experiments and how serious the
>> consequences of even the slightest infringements
>> of their often arbitrary rules could be. I am
>> sure many of our colleagues have a few personally
>> experienced horror stories to tell. The large-N
>> studies for those boards spell simply "NO".
>>
>> Pierre