Re: Experiments with large N

[Robert Zatorre <robert.zatorre@xxxxxxxxx>]

Massimo, thank you for this very interesting reference. I think the 
point here is not so much about large N per se, but about whether one is 
interested in characterizing the typical population (for which a more 
limited sample is sufficient), or whether one is interested in 
investigating the extremes of a population, which by definition will 
require one to go and look for the tails of the distribution. Semal and 
Demany did a smart thing: they pretested 68 people (not such a huge 
number to test, actually, especially in a group setting as was done 
here), and then did more intensive testing with a small group of people 
(N=9) who specifically did poorly on the initial screening. This a good 
way to proceed if you want to find the extremes of a population.

Psychoacousticians tend to use a small number of highly trained 
listeners, but many labs including my own, do not follow this convention 
unless we are specifically interested in testing the effects of 
training. (Indeed, this sometimes gets us into trouble with reviewers 
who insist that we should only use experienced listeners or else the 
data are too noisy and not representative--actually it can be the 
opposite, in some instances, since training and experience have their 
own effects.)

Regards

Robert


-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Robert J. Zatorre, Ph.D.
Montreal Neurological Institute
3801 University St.
Montreal, QC Canada H3A 2B4
phone: 1-514-398-8903
fax: 1-514-398-1338
e-mail: robert.zatorre@xxxxxxxxx
web site: www.zlab.mcgill.ca


Massimo Grassi wrote:
> Dear all,
>
> > Huge samples are very nice if you can get 'em, though such is not 
> > always the case, alas.
>
> snip!
>
> If I had a large (very large) N I would be more interested in have a 
> glance of the population distribution. And this is the reason why, I 
> think, large N studies are important.
>
> For example, last year Semal and Demany (JASA, 2006) showed that there 
> are listerers that are able to detect subtle pitch differences but, at 
> the same time, are not able to tell the direction of these pitch 
> differences (up or down). They showed also that there are listeners that 
> do not show such a dissociation. How is the distribution of this trait 
> among the population? Are the first listeners a small or a relatively 
> large amount of the population?
>
> Think about. Pitch perception is one of the most investigated topic in 
> auditory perception. Why do we discover this only in 2006? Is it because 
>  the average study is run with few listeners only and these listeners 
> are, in the majority of cases, the "usual suspects" (e.g., authors, 
> people with a background of a minimum of 1000 hours of psycho-beeps 
> experiment, etc.).
>
> In textbooks we often read that JND for pitch is about 1%. After reading 
> Semal and Demany results I asked myself: is this figure real? Does it 
> reflect all the population? Which part of the population? For which 
> particular task?
>
> It is trivial to stress that, with large N, statistical power goes soon 
> very high. I think the goal of large N study is different: have, at 
> least once, a real glance of the population.
>
> m