Re: [AUDITORY] stats use in psychology and hearing science (Iftikhar Nizami )


Subject: Re: [AUDITORY] stats use in psychology and hearing science
From:    Iftikhar Nizami  <nizamii2@xxxxxxxx>
Date:    Sun, 23 Jun 2013 11:21:36 -0700
List-Archive:<http://lists.mcgill.ca/scripts/wa.exe?LIST=AUDITORY>

--1545434552-73998786-1372011696=:43110 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable Dear List - My thanks to Holger Mitterer for pointing out the paper by Simm= ons et al in Psych Sci, which promises to be an interesting read. It is jus= t one of a long string of papers in recent years which point out just how l= ittle of value can arise through statistical testing of experimental result= s (see also the numerous papers of John Ioannidis at Stanford on this topic= in medicine).=0A=A0=0AUnfortunately, this problem of designing experiments= for the data analysis - and the wider problem of inappropriate experimenta= l design and inappropriate data analysis - is only going to get worse, espe= cially in departments of education, psychology, and hearing research. There= , the older generation of researchers, who might have had at least an under= grad freshman calculus course, has been replaced by a new generation of wor= kers who do not have math beyond the 10th grade of high school and who bare= ly passed their weak undergraduate mandatory course in practical stats. Too= many people now seem to think of stats testing (ANOVA in particular) as an= act of magic that tells them what's "significant". It is exceedingly rare,= for example, to find any mention of whether the assumptions underlying the= statistical tests are actually obeyed, as no-one seems to realize that sta= tistical tests are derived from mathematical models that involve assumption= s.=A0There is a solution to this problem: stricter math requirements at the undergraduate = and graduate levels, including introductory theoretical statistics, not jus= t basic stats testing. If we're going to use stats, let's do it properly. -= Lance Nizami PhD, Palo Alto, Cal.=0A=0A=0A>_______________________________= _=0A>From: Holger Mitterer <holgermitterer@xxxxxxxx>=0A>To: AUDITORY@xxxxxxxx= TS.MCGILL.CA =0A>Sent: Saturday, June 22, 2013 7:40 AM=0A>Subject: [AUDITOR= Y] Reminder: Speed Sound Finding Experiment=0A>=0A>=0A>=0A>probably many re= aders caught this, but just to make sure:=0A>The reminder for the speech so= und finding experiment contained a somewhat questionable phrase:=0A>=0A>=0A= >> We are missing a few participants to reach statistical significance so= =0A>> please consider giving it a try:=0A>=0A>=0A>Stopping data collection = when an effect becomes significant is a very problematic research strategy,= =0A>see the paper by Simmons et al. in PsychScience (http://pss.sagepub.com= /content/22/11/1359.abstract).=0A>=0A>=0A>=0A>=0A>=0A> --1545434552-73998786-1372011696=:43110 Content-Type: text/html; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable <html><body><div style=3D"color:#000; background-color:#fff; font-family:ti= mes new roman, new york, times, serif;font-size:12pt"><div style=3D"RIGHT: = auto"><SPAN style=3D"RIGHT: auto">Dear List - My thanks to Holger Mitterer = for pointing out the paper by Simmons et al in Psych Sci, which promises to= be an interesting read. It is just one of a long string of papers in recen= t years which point out just how little of value can arise through statisti= cal testing of experimental results (see also the numerous pap<VAR id=3Dyui= -ie-cursor></VAR>ers of John Ioannidis at Stanford on this topic in medicin= e).</SPAN></div> <div style=3D"RIGHT: auto"><SPAN style=3D"RIGHT: auto"></SPAN>&nbsp;</div> <div style=3D"RIGHT: auto"><SPAN style=3D"RIGHT: auto">Unfortunately, this = problem of designing experiments for the data analysis - and the wider prob= lem of inappropriate experimental design and inappropriate data analysis - = is only going to get worse, especially in departments of education, psychol= ogy, and hearing research. There, the older generation of researchers, who = might have had at least an undergrad freshman calculus course, has been rep= laced by a new generation of workers who do not have math beyond the 10th g= rade of high school and who barely passed their weak undergraduate mandator= y course in practical stats. Too many people now seem to think of stats tes= ting (ANOVA in particular) as an act of magic that tells them what's "signi= ficant". It is exceedingly rare, for example, to find any mention of whethe= r the assumptions underlying the statistical tests are actually obeyed, as = no-one seems to realize that statistical tests are derived from mathematical models that involve <EM>assumptions</EM>.&nbsp;There is a sol= ution to this problem: stricter math requirements at the undergraduate and = graduate levels, including introductory <EM style=3D"RIGHT: auto">theoretic= al </EM>statistics, not just basic stats testing. If we're going to use sta= ts, let's do it properly. - Lance Nizami PhD, Palo Alto, Cal.</SPAN></div> <div style=3D"RIGHT: auto"><BR style=3D"RIGHT: auto"> <BLOCKQUOTE style=3D"BORDER-LEFT: rgb(16,16,255) 2px solid; MARGIN-TOP: 5px= ; PADDING-LEFT: 5px; MARGIN-LEFT: 5px; RIGHT: auto"> <DIV style=3D"FONT-FAMILY: times new roman, new york, times, serif; FONT-SI= ZE: 12pt"> <DIV style=3D"FONT-FAMILY: times new roman, new york, times, serif; FONT-SI= ZE: 12pt"> <DIV style=3D"RIGHT: auto" dir=3Dltr> <DIV style=3D"BORDER-BOTTOM: #ccc 1px solid; BORDER-LEFT: #ccc 1px solid; P= ADDING-BOTTOM: 0px; LINE-HEIGHT: 0; MARGIN: 5px 0px; PADDING-LEFT: 0px; PAD= DING-RIGHT: 0px; HEIGHT: 0px; FONT-SIZE: 0px; BORDER-TOP: #ccc 1px solid; B= ORDER-RIGHT: #ccc 1px solid; PADDING-TOP: 0px" class=3Dhr contentEditable= =3Dfalse readonly=3D"true"></DIV><FONT size=3D2 face=3DArial><B><SPAN style= =3D"FONT-WEIGHT: bold">From:</SPAN></B> Holger Mitterer &lt;holgermitterer@xxxxxxxx= YAHOO.CO.UK&gt;<BR><B><SPAN style=3D"FONT-WEIGHT: bold">To:</SPAN></B> AUDI= TORY@xxxxxxxx <BR><B><SPAN style=3D"FONT-WEIGHT: bold">Sent:</SPAN><= /B> Saturday, June 22, 2013 7:40 AM<BR><B><SPAN style=3D"FONT-WEIGHT: bold"= >Subject:</SPAN></B> [AUDITORY] Reminder: Speed Sound Finding Experiment<BR= ></FONT></DIV> <DIV style=3D"RIGHT: auto" class=3Dy_msg_container><BR> <DIV style=3D"RIGHT: auto" id=3Dyiv1008380456> <DIV style=3D"RIGHT: auto"> <DIV style=3D"BACKGROUND-COLOR: #fff; FONT-FAMILY: tahoma, new york, times,= serif; COLOR: #000; FONT-SIZE: 12pt"> <DIV>probably many readers caught this, but just to make sure:</DIV> <DIV>The reminder for the speech sound finding experiment contained a somew= hat questionable phrase:</DIV> <DIV style=3D"BACKGROUND-COLOR: transparent; FONT-STYLE: normal; FONT-FAMIL= Y: tahoma, new york, times, serif; COLOR: rgb(0,0,0); FONT-SIZE: 16px"><BR>= </DIV> <DIV style=3D"BACKGROUND-COLOR: transparent; FONT-STYLE: normal; FONT-FAMIL= Y: tahoma, new york, times, serif; COLOR: rgb(0,0,0); FONT-SIZE: 16px">&gt;= We are missing a few participants to reach statistical significance so<BR>= &gt; please consider giving it a try:</DIV> <DIV style=3D"BACKGROUND-COLOR: transparent; FONT-STYLE: normal; FONT-FAMIL= Y: tahoma, new york, times, serif; COLOR: rgb(0,0,0); FONT-SIZE: 16px"><BR>= </DIV> <DIV style=3D"BACKGROUND-COLOR: transparent; FONT-STYLE: normal; FONT-FAMIL= Y: tahoma, new york, times, serif; COLOR: rgb(0,0,0); FONT-SIZE: 16px">Stop= ping data collection when an effect becomes significant is a very problemat= ic research strategy,</DIV> <DIV style=3D"BACKGROUND-COLOR: transparent; FONT-STYLE: normal; FONT-FAMIL= Y: tahoma, new york, times, serif; COLOR: rgb(0,0,0); FONT-SIZE: 16px">see = the paper by Simmons et al. in PsychScience (http://pss.sagepub.com/content= /22/11/1359.abstract).</DIV> <DIV style=3D"BACKGROUND-COLOR: transparent; FONT-STYLE: normal; FONT-FAMIL= Y: tahoma, new york, times, serif; COLOR: rgb(0,0,0); FONT-SIZE: 16px"><BR>= </DIV> <DIV style=3D"BACKGROUND-COLOR: transparent; FONT-STYLE: normal; FONT-FAMIL= Y: tahoma, new york, times, serif; COLOR: rgb(0,0,0); FONT-SIZE: 16px"><BR>= </DIV></DIV></DIV></DIV><BR><BR></DIV></DIV></DIV></BLOCKQUOTE></div></div>= </body></html> --1545434552-73998786-1372011696=:43110--


This message came from the mail archive
/var/www/postings/2013/
maintained by:
DAn Ellis <dpwe@ee.columbia.edu>
Electrical Engineering Dept., Columbia University