Subject: Re: [AUDITORY] apps for web-based listening tests From: Peter Harrison <p.m.c.harrison@xxxxxxxx> Date: Wed, 13 Jan 2021 11:37:16 +0000--0000000000005464e905b8c69048 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Dear Jan, Maybe the Web Audio Evaluation Tool <https://github.com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfaces> would be a good choice for you? Pasting from their documentation, I see that it supports the following kinds of designs (alongside others, I daresay): - AB Test / Pairwise comparison [4 <https://github.com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfaces#r= eferences> , 5 <https://github.com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfaces#r= eferences>]: Two stimuli presented simultaneously, participant selects a preferred stimulus. *(AB)* - ABC/HR (ITU-R BS. 1116) (Mean Opinion Score: MOS) [6 <https://github.com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfaces#r= eferences>]: each stimulus has a continuous scale (5-1), labeled as Imperceptible, Perceptible but not annoying, Slightly annoying, Annoying, Very annoying= . - -50 to 50 Bipolar with reference: each stimulus has a continuous scale -50 to 50 with default values as 0 in middle and a reference. - Absolute Category Rating (ACR) Scale [7 <https://github.com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfaces#r= eferences>]: Likert but labels are Bad, Poor, Fair, Good, Excellent - ABX Test [8 <https://github.com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfaces#r= eferences>]: Two stimuli are presented along with a reference and the participant has= to select a preferred stimulus, often the closest to the reference. *(ABX)* - APE [1 <https://github.com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfaces#r= eferences>]: Multiple stimuli on one or more axes for inter-sample rating. *(APE)* - Comparison Category Rating (CCR) Scale [7 <https://github.com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfaces#r= eferences>]: ACR & DCR but 7 point scale, with reference: Much better, Better, Slight= ly better, About the same, Slightly worse, Worse, Much worse. - Degredation Category Rating (DCR) Scale [7 <https://github.com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfaces#r= eferences>]: ABC & Likert but labels are (5) Inaudible, (4) Audible but not annoying, (3) Slightly annoying, (2) Annoying, (1) Very annoying. - ITU-R 5 Point Continuous Impairment Scale [9 <https://github.com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfaces#r= eferences>]: Same as ABC/HR but with a reference. - Likert scale [10 <https://github.com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfaces#r= eferences>]: each stimulus has a five point scale with values: Strongly agree, Agree, Neutral, Disagree and Strongly disagree. - MUSHRA (ITU-R BS. 1534) [3 <https://github.com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfaces#r= eferences>]: Multiple stimuli are presented and rated on a continuous scale, which includes a reference, hidden reference and hidden anchors. *(MUSHRA)* - Pairwise Comparison (Better/Worse) [5 <https://github.com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfaces#r= eferences>]: every stimulus is rated as being either better or worse than the referen= ce. - Rank Scale [12 <https://github.com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfaces#r= eferences>]: stimuli ranked on single horizontal scale, where they are ordered in preference order. - 9 Point Hedonic Category Rating Scale [13 <https://github.com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfaces#r= eferences>]: each stimulus has a seven point scale with values: Like extremely, Like very much, Like moderate, Like slightly, Neither like nor dislike, Disli= ke extremely, Dislike very much, Dislike moderate, Dislike slightly. There = is also a provided reference. I haven't used it myself, but I've heard good things about it. Best wishes, Peter On Wed, Jan 13, 2021 at 5:25 AM Jan Felcyn <janaku@xxxxxxxx> wrote: > Dear List! > > I am looking for some web-based apps which allow to create listening > tests. By creating listening tests I mean rather simple approach =E2=80= =93 like > e.g. tests from Music Lab. Just playin=E2=80=99 stereo audio files, gathe= ring > responses and aggregate them in one file. One important feature is that i= t > should be possible to create psychoacoustic tests like pair comparisons, > 2AFS and so on. > > Do you know any web-based platform for such thing? I=E2=80=99ve heard abo= ut BRAMS > but never used it. I know Python and I know that I can use Django, but > maybe there is something already prepared for such tests? > > Thanks in advance for all comments! > > Regards > > --- > Jan Felcyn PhD > Chair of Acoustics, > Adam Mickiewicz University, > Pozna=C5=84, Poland > --0000000000005464e905b8c69048 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dutf-8"><d= iv dir=3D"ltr">D<font face=3D"arial, sans-serif">ear Jan,</font><div><font = face=3D"arial, sans-serif"><br></font></div><div><font face=3D"arial, sans-= serif">Maybe the <a href=3D"https://github.com/BrechtDeMan/WebAudioEvaluati= onTool/wiki/Interfaces">Web Audio Evaluation Tool</a> would be a good choic= e for you? Pasting from their documentation, I see that it supports th= e following kinds of designs (alongside others, I daresay):</font></div><di= v><font face=3D"arial, sans-serif"><br></font></div><div><ul style=3D"box-s= izing:border-box;padding-left:2em;margin-top:0px;margin-bottom:16px;color:r= gb(36,41,46)"><li style=3D"box-sizing:border-box"><font face=3D"arial, sans= -serif"><span style=3D"box-sizing:border-box;font-weight:600">AB Test / Pai= rwise comparison</span> [<a href=3D"https://github.com/BrechtDeMan/Web= AudioEvaluationTool/wiki/Interfaces#references" style=3D"box-sizing:border-= box;background-color:initial;text-decoration-line:none">4</a>, <a href= =3D"https://github.com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfaces#r= eferences" style=3D"box-sizing:border-box;background-color:initial;text-dec= oration-line:none">5</a>]: Two stimuli presented simultaneously, participan= t selects a preferred stimulus. <em style=3D"box-sizing:border-box">(A= B)</em></font></li><li style=3D"box-sizing:border-box;margin-top:0.25em"><f= ont face=3D"arial, sans-serif"><span style=3D"box-sizing:border-box;font-we= ight:600">ABC/HR (ITU-R BS. 1116)</span> (Mean Opinion Score: MOS) [<a= href=3D"https://github.com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfa= ces#references" style=3D"box-sizing:border-box;background-color:initial;tex= t-decoration-line:none">6</a>]: each stimulus has a continuous scale (5-1),= labeled as Imperceptible, Perceptible but not annoying, Slightly annoying,= Annoying, Very annoying.</font></li><li style=3D"box-sizing:border-box;mar= gin-top:0.25em"><font face=3D"arial, sans-serif"><span style=3D"box-sizing:= border-box;font-weight:600">-50 to 50 Bipolar with reference</span>: each s= timulus has a continuous scale -50 to 50 with default values as 0 in middle= and a reference.</font></li><li style=3D"box-sizing:border-box;margin-top:= 0.25em"><font face=3D"arial, sans-serif"><span style=3D"box-sizing:border-b= ox;font-weight:600">Absolute Category Rating (ACR) Scale</span> [<a hr= ef=3D"https://github.com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfaces= #references" style=3D"box-sizing:border-box;background-color:initial;text-d= ecoration-line:none">7</a>]: Likert but labels are Bad, Poor, Fair, Good, E= xcellent</font></li><li style=3D"box-sizing:border-box;margin-top:0.25em"><= font face=3D"arial, sans-serif"><span style=3D"box-sizing:border-box;font-w= eight:600">ABX Test</span> [<a href=3D"https://github.com/BrechtDeMan/= WebAudioEvaluationTool/wiki/Interfaces#references" style=3D"box-sizing:bord= er-box;background-color:initial;text-decoration-line:none">8</a>]: Two stim= uli are presented along with a reference and the participant has to select = a preferred stimulus, often the closest to the reference. <em style=3D= "box-sizing:border-box">(ABX)</em></font></li><li style=3D"box-sizing:borde= r-box;margin-top:0.25em"><font face=3D"arial, sans-serif"><span style=3D"bo= x-sizing:border-box;font-weight:600">APE</span> [<a href=3D"https://gi= thub.com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfaces#references" sty= le=3D"box-sizing:border-box;background-color:initial;text-decoration-line:n= one">1</a>]: Multiple stimuli on one or more axes for inter-sample rating.&= nbsp;<em style=3D"box-sizing:border-box">(APE)</em></font></li><li style=3D= "box-sizing:border-box;margin-top:0.25em"><font face=3D"arial, sans-serif">= <span style=3D"box-sizing:border-box;font-weight:600">Comparison Category R= ating (CCR) Scale</span> [<a href=3D"https://github.com/BrechtDeMan/We= bAudioEvaluationTool/wiki/Interfaces#references" style=3D"box-sizing:border= -box;background-color:initial;text-decoration-line:none">7</a>]: ACR & = DCR but 7 point scale, with reference: Much better, Better, Slightly better= , About the same, Slightly worse, Worse, Much worse.</font></li><li style= =3D"box-sizing:border-box;margin-top:0.25em"><font face=3D"arial, sans-seri= f"><span style=3D"box-sizing:border-box;font-weight:600">Degredation Catego= ry Rating (DCR) Scale</span> [<a href=3D"https://github.com/BrechtDeMa= n/WebAudioEvaluationTool/wiki/Interfaces#references" style=3D"box-sizing:bo= rder-box;background-color:initial;text-decoration-line:none">7</a>]: ABC &a= mp; Likert but labels are (5) Inaudible, (4) Audible but not annoying, (3) = Slightly annoying, (2) Annoying, (1) Very annoying.</font></li><li style=3D= "box-sizing:border-box;margin-top:0.25em"><font face=3D"arial, sans-serif">= <span style=3D"box-sizing:border-box;font-weight:600">ITU-R 5 Point Continu= ous Impairment Scale</span> [<a href=3D"https://github.com/BrechtDeMan= /WebAudioEvaluationTool/wiki/Interfaces#references" style=3D"box-sizing:bor= der-box;background-color:initial;text-decoration-line:none">9</a>]: Same as= ABC/HR but with a reference.</font></li><li style=3D"box-sizing:border-box= ;margin-top:0.25em"><font face=3D"arial, sans-serif"><span style=3D"box-siz= ing:border-box;font-weight:600">Likert scale</span> [<a href=3D"https:= //github.com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfaces#references"= style=3D"box-sizing:border-box;background-color:initial;text-decoration-li= ne:none">10</a>]: each stimulus has a five point scale with values: Strongl= y agree, Agree, Neutral, Disagree and Strongly disagree.</font></li><li sty= le=3D"box-sizing:border-box;margin-top:0.25em"><font face=3D"arial, sans-se= rif"><span style=3D"box-sizing:border-box;font-weight:600">MUSHRA (ITU-R BS= . 1534)</span> [<a href=3D"https://github.com/BrechtDeMan/WebAudioEval= uationTool/wiki/Interfaces#references" style=3D"box-sizing:border-box;backg= round-color:initial;text-decoration-line:none">3</a>]: Multiple stimuli are= presented and rated on a continuous scale, which includes a reference, hid= den reference and hidden anchors. <em style=3D"box-sizing:border-box">= (MUSHRA)</em></font></li><li style=3D"box-sizing:border-box;margin-top:0.25= em"><font face=3D"arial, sans-serif"><span style=3D"box-sizing:border-box;f= ont-weight:600">Pairwise Comparison (Better/Worse)</span> [<a href=3D"= https://github.com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfaces#refer= ences" style=3D"box-sizing:border-box;background-color:initial;text-decorat= ion-line:none">5</a>]: every stimulus is rated as being either better or wo= rse than the reference.</font></li><li style=3D"box-sizing:border-box;margi= n-top:0.25em"><font face=3D"arial, sans-serif"><span style=3D"box-sizing:bo= rder-box;font-weight:600">Rank Scale</span> [<a href=3D"https://github= .com/BrechtDeMan/WebAudioEvaluationTool/wiki/Interfaces#references" style= =3D"box-sizing:border-box;background-color:initial;text-decoration-line:non= e">12</a>]: stimuli ranked on single horizontal scale, where they are order= ed in preference order.</font></li><li style=3D"box-sizing:border-box;margi= n-top:0.25em"><font face=3D"arial, sans-serif" style=3D""><span style=3D"bo= x-sizing:border-box;font-weight:600">9 Point Hedonic Category Rating Scale<= /span> [<a href=3D"https://github.com/BrechtDeMan/WebAudioEvaluationTo= ol/wiki/Interfaces#references" style=3D"box-sizing:border-box;background-co= lor:initial;text-decoration-line:none">13</a>]: each stimulus has a seven p= oint scale with values: Like extremely, Like very much, Like moderate, Like= slightly, Neither like nor dislike, Dislike extremely, Dislike very much, = Dislike moderate, Dislike slightly. There is also a provided reference.</fo= nt></li></ul><div><font color=3D"#24292e" face=3D"arial, sans-serif">I have= n't used it myself, but I've heard good things about it.</font></div><div><= font color=3D"#24292e" face=3D"arial, sans-serif"><br></font></div><div><fo= nt color=3D"#24292e" face=3D"arial, sans-serif">Best wishes,</font></div></= div><div><font color=3D"#24292e" face=3D"arial, sans-serif">Peter</font></d= iv></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_att= r">On Wed, Jan 13, 2021 at 5:25 AM Jan Felcyn <<a href=3D"mailto:janaku@xxxxxxxx= amu.edu.pl">janaku@xxxxxxxx</a>> wrote:<br></div><blockquote class=3D"= gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(20= 4,204,204);padding-left:1ex"> <div dir=3D"ltr"> <div dir=3D"ltr"> <div style=3D"font-size:12pt;font-family:Calibri;color:rgb(0,0,0)"> <div>Dear List!</div> <div> </div> <div>I am looking for some web-based apps which allow to create listening t= ests.=20 By creating listening tests I mean rather simple approach =E2=80=93 like e.= g. tests from=20 Music Lab. Just playin=E2=80=99 stereo audio files, gathering responses and= aggregate=20 them in one file. One important feature is that it should be possible to cr= eate=20 psychoacoustic tests like pair comparisons, 2AFS and so on.</div> <div> </div> <div>Do you know any web-based platform for such thing? I=E2=80=99ve heard = about BRAMS=20 but never used it. I know Python and I know that I can use Django, but mayb= e=20 there is something already prepared for such tests?</div> <div> </div> <div>Thanks in advance for all comments!</div> <div> </div> <div>Regards</div> <div> </div> <div>---</div> <div>Jan Felcyn PhD</div> <div>Chair of Acoustics,</div> <div>Adam Mickiewicz University,</div> <div>Pozna=C5=84, Poland</div></div></div></div> </blockquote></div> --0000000000005464e905b8c69048--