Subject: Re: [AUDITORY] apps for web-based listening tests From: Hyunkook Lee <000000932b130266-dmarc-request@xxxxxxxx> Date: Fri, 15 Jan 2021 01:42:20 +0000--_000_VI1PR01MB38708F7B061BAEAB6C7C6692B5A70VI1PR01MB3870eurp_ Content-Type: text/plain; charset="windows-1250" Content-Transfer-Encoding: quoted-printable Dear Jan, and everyone, I'd like to introduce another listening test tool HULTI-GEN, which we devel= oped at the University of Huddersfield. This one is Max-based. I know you are looking for web-based one, but the la= test version (v2) includes psychometric tests (nAFC, Yes-No, Staircase, ABX= , etc.) as well as quality evaluation tests (ITU-R and ITU-T methods). And = it provides headphone equalisation using an extensive set of headphone filt= ers included or any custom-loaded headphone filter you might use. So it can= be useful in remote testing where people use different headphones. It als= o offers very convenient test block design with repeats and randomisation, = JSON config file saving, recall, result file saved in csv, etc. It also su= pports multichannel audio up to 64 channels, and can binauralise the audio = for arbitrary speaker positions using a SOFA file. Hulti-gen is the only li= stening tool that provides such multichannel/binaural functionality as far = as I know. You can download it from here: https://doi.org/10.5281/zenodo.4108976 Although it's not online, the participants can easily install the software,= load the config file you send them, and the test is ready to go. I've rece= ntly conducted a Yes/No test remotely using the tool, and it was very smoot= h. If any questions, please let me know. Best wishes, Hyunkook =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Dr Hyunkook Lee, BMus(Tonmeister), PhD, FAES, FHEA Reader (Associate Professor) in Music Technology Director of the Centre for Audio and Psychoacoustic Engineering (CAPE) Founder/Leader of the Applied Psychoacoustics Laboratory (APL) http://www.hyunkooklee.com <http://www.hyunkooklee.com> http://www.hud.ac.uk/apl Phone: +44 (0)1484 471893 Email: h.lee@xxxxxxxx Office: HA2/06 School of Computing and Engineering University of Huddersfield Huddersfield HD1 3DH United Kingdom ________________________________ From: AUDITORY - Research in Auditory Perception <AUDITORY@xxxxxxxx>= on behalf of Jan Felcyn <janaku@xxxxxxxx> Sent: 14 January 2021 08:42 To: AUDITORY@xxxxxxxx <AUDITORY@xxxxxxxx> Subject: Re: [AUDITORY] apps for web-based listening tests Dear List! Thank you all for your comments and interesting findings! Definitely we can= choose something from your mails. Hopefully, we will run some experiments = using these tools in the near future. Regards Jan From: Chris Stecker Sent: Wednesday, January 13, 2021 7:57 PM To: AUDITORY@xxxxxxxx Subject: Re: apps for web-based listening tests Hi Jan and List, I=92d like to point you to a resource that you might find helpful in develo= ped you web-based or other remote testing approach: The =93Remote Testing W= iki=94 at https://spatialhearing.org/remotetesting<https://eur02.safelinks.= protection.outlook.com/?url=3Dhttps%3A%2F%2Fspatialhearing.org%2Fremotetest= ing&data=3D04%7C01%7Ch.lee%40HUD.AC.UK%7Cf2d3394b57d6491e808408d8b86d7275%7= Cb52e9fda06914585bdfc5ccae1ce1890%7C0%7C0%7C637462127504751601%7CUnknown%7C= TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0= %3D%7C1000&sdata=3DkpQaTTsziYBkDRhsh8AooinVoKXCYceRZzbHReldkS0%3D&reserved= =3D0>. The Wiki is the result of activities by the Remote Testing Task Force, whic= h was launched last year by the Acoustical Society of America Technical Com= mittee on Psychological and Physiological Acoustics. It includes informatio= n about issues and practices for remote testing, a database of platforms th= at can be used to develop and deploy remote testing research, and a variety= of other resources including brief reports of project experiences, bibliog= raphy of related works, and links to other online resources. The Platform Survey Database includes information that Jan could use to fin= d and select platforms, including several Python-based approaches, that Tas= k Force members and others have evaluated. Finally, the Wiki has a lot of information in it currently but is still gro= wing. We still need help identifying and evaluating platforms the we might = not know about yet. There are links for submitting new examples and platfor= msSo if you have experience with a platform (for example, Borys seems famil= iar with SenseLabOnline), it would be GREAT if you could visit the wiki and= submit information about it. That goes for new platforms and for updating = information about features of existing platforms. There is a simple survey = tool that just takes a few minutes. Feel free to reach out to me (or better yet, via the contact links on the W= iki) if you have additional question. Chris =97 G. Christopher Stecker, PhD, FASA Coordinating Editor, Psychological and Physiological Acoustics Journal of the Acoustical Society of America Chair, Task Force on Remote Testing ASA Technical Committee on Psychological and Physiological Acoustics cstecker@xxxxxxxx On Jan 12, 2021, at 11:46 PM, Borys Kowalewski <borys.k@xxxxxxxx> wrote: Hello Jan, Good to hear from you! For paired comparisons, ABX etc. you might find SenseLabOnline interesting.= It allows you to create a multidimensional design and prepare a simple gra= phical summary. You can also download the raw data in a JSON file for furth= er analysis. However, it is very much a quality- and preference-evaluation = tool, and except for the ABX it does not support the typical =93performance= -based=94 n-AFC tests with wrong and correct answers. Adaptive procedures a= re not an option either, since all the tests require pre-loading of wave fi= les. Hopefully other list members can suggest viable alternatives :) All best, Borys On Wed, Jan 13, 2021 at 06:23 Jan Felcyn <janaku@xxxxxxxx> wrote: Dear List! I am looking for some web-based apps which allow to create listening tests.= By creating listening tests I mean rather simple approach =96 like e.g. te= sts from Music Lab. Just playin=92 stereo audio files, gathering responses = and aggregate them in one file. One important feature is that it should be = possible to create psychoacoustic tests like pair comparisons, 2AFS and so = on. Do you know any web-based platform for such thing? I=92ve heard about BRAMS= but never used it. I know Python and I know that I can use Django, but may= be there is something already prepared for such tests? Thanks in advance for all comments! Regards --- Jan Felcyn PhD Chair of Acoustics, Adam Mickiewicz University, Pozna=F1, Poland University of Huddersfield inspiring global professionals. [http://marketing.hud.ac.uk/_HOSTED/EmailSig2014/EmailSigFooterMarch2019.jp= g] This transmission is confidential and may be legally privileged. If you rec= eive it in error, please notify us immediately by e-mail and remove it from= your system. If the content of this e-mail does not relate to the business= of the University of Huddersfield, then we do not endorse it and will acce= pt no liability. --_000_VI1PR01MB38708F7B061BAEAB6C7C6692B5A70VI1PR01MB3870eurp_ Content-Type: text/html; charset="windows-1250" Content-Transfer-Encoding: quoted-printable <html> <head> <meta http-equiv=3D"Content-Type" content=3D"text/html; charset=3Dwindows-1= 250"> <style type=3D"text/css" style=3D"display:none;"> P {margin-top:0;margin-bo= ttom:0;} </style> </head> <body dir=3D"ltr"> <div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif; font-size= : 12pt; color: rgb(0, 0, 0);"> Dear Jan, and everyone,</div> <div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif; font-size= : 12pt; color: rgb(0, 0, 0);"> <br> </div> <div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif; font-size= : 12pt; color: rgb(0, 0, 0);"> I'd like to introduce another listening test tool HULTI-GEN, which we devel= oped at the University of Huddersfield. </div> <div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif; font-size= : 12pt; color: rgb(0, 0, 0);"> <span style=3D"color: rgb(0, 0, 0); font-family: Calibri, Arial, Helvetica,= sans-serif; font-size: 12pt;">This one is Max-based. I know you are lookin= g for web-based one, but the latest version (v2) includes psychometric test= s (nAFC, Yes-No, Staircase, ABX, etc.) as well as quality evaluation tests (ITU-R and ITU-T methods). And it prov= ides headphone equalisation using an extensive set of headphone filters inc= luded or any custom-loaded headphone filter you might use. So it can be use= ful in remote testing where people use different headphones. It also offers very convenient test block = design with repeats and randomisation, JSON config file saving, recall, res= ult file saved in csv, etc. It also supports multichannel audio up to= 64 channels, and</span><span style=3D"color: rgb(0, 0, 0); font-family: Ca= libri, Arial, Helvetica, sans-serif; font-size: 12pt;"> can binauralise the audio for arbitrary speaker positions using a SOFA file. H= ulti-gen is the only listening tool that provides such multichannel/binaura= l functionality as far as I know. </span></div> <div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif; font-size= : 12pt; color: rgb(0, 0, 0);"> <span style=3D"color: rgb(0, 0, 0); font-family: Calibri, Arial, Helvetica,= sans-serif; font-size: 12pt;"><br> </span></div> <div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif; font-size= : 12pt; color: rgb(0, 0, 0);"> <span style=3D"color: rgb(0, 0, 0); font-family: Calibri, Arial, Helvetica,= sans-serif; font-size: 12pt;">You can download it from here: <a href= =3D"https://doi.org/10.5281/zenodo.4108976" style=3D"margin:0px;background-= color:rgb(255, 255, 255)">https://doi.org/10.5281/zenodo.4108976</a></span>= </div> <div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif; font-size= : 12pt; color: rgb(0, 0, 0);"> <br> </div> <div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif; font-size= : 12pt; color: rgb(0, 0, 0);"> Although it's not online, the participants can easily install the software,= load the config file you send them, and the test is ready to go. I've rece= ntly conducted a Yes/No test remotely using the tool, and it was very smoot= h. </div> <div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif; font-size= : 12pt; color: rgb(0, 0, 0);"> <br> </div> <div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif; font-size= : 12pt; color: rgb(0, 0, 0);"> If any questions, please let me know.</div> <div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif; font-size= : 12pt; color: rgb(0, 0, 0);"> <br> </div> <div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif; font-size= : 12pt; color: rgb(0, 0, 0);"> Best wishes,</div> <div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif; font-size= : 12pt; color: rgb(0, 0, 0);"> Hyunkook</div> <div style=3D"font-family: Calibri, Arial, Helvetica, sans-serif; font-size= : 12pt; color: rgb(0, 0, 0);"> <span style=3D"font-family: Tahoma; font-size: 13px;"> = </span><br> </div> <div> <div id=3D"Signature"> <div> <div style=3D"font-family:Tahoma; font-size:13px"> <div style=3D"font-family:Tahoma; font-size:13px"> <div style=3D"font-family:Tahoma; font-size:13px"> <div style=3D"font-family:Tahoma; font-size:13px"> <div style=3D"font-family:Tahoma; font-size:13px"> <div style=3D""> <div style=3D""> <div style=3D""> <div style=3D""> <div style=3D""> <div style=3D""> <div style=3D""> <div style=3D""> <div style=3D""> <div style=3D""> <div class=3D"BodyFragment" style=3D""><font style=3D""> <div class=3D"PlainText" style=3D"font-family:Tahoma; font-size:13px">=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D<br> Dr Hyunkook Lee, BMus(Tonmeister), PhD, FAES, FHEA<br> </div> <div class=3D"PlainText" style=3D"font-family:Tahoma; font-size:13px"><span= style=3D"font-family:Tahoma; background-color:rgb(255,255,255); display:in= line!important">Reader (Associate Professor) in Music Technology</span><br> </div> <div class=3D"PlainText" style=3D"font-family:Tahoma; font-size:13px"></div= > <span style=3D"font-family:Tahoma; text-align:start; background-color:rgb(2= 55,255,255); display:inline!important">Director of the Centre for Audio and= Psychoacoustic Engineering (CAPE)</span></font></div> <div class=3D"BodyFragment" style=3D""><font style=3D""><span style=3D"font= -family:Tahoma; text-align:start; background-color:rgb(255,255,255); displa= y:inline!important">Founder/Leader of the Applied Psychoacoustics Laborator= y (APL)</span></font></div> <div class=3D"BodyFragment" style=3D""><font style=3D""> <div class=3D"PlainText" style=3D""><a href=3D"http://www.hyunkooklee.com">= http://www.hyunkooklee.com </a></div> <div class=3D"PlainText" style=3D"font-family:Tahoma; font-size:13px"><u st= yle=3D"color:rgb(0,0,238)">http://www.hud.ac.uk/apl</u></div> <div class=3D"PlainText" style=3D"font-family:Tahoma; font-size:13px">Phone= : +44 (0)1484 471893<br> Email: h.lee@xxxxxxxx</div> <div class=3D"PlainText" style=3D"font-family:Tahoma; font-size:13px">Offic= e: HA2/06 </div> <div class=3D"PlainText" style=3D"font-family:Tahoma; font-size:13px">Schoo= l of Computing and Engineering<br> University of Huddersfield<br> Huddersfield<br> HD1 3DH<br> United Kingdom<br> </div> </font></div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> <div id=3D"appendonsend"></div> <hr style=3D"display:inline-block;width:98%" tabindex=3D"-1"> <div id=3D"divRplyFwdMsg" dir=3D"ltr"><font face=3D"Calibri, sans-serif" st= yle=3D"font-size:11pt" color=3D"#000000"><b>From:</b> AUDITORY - Research i= n Auditory Perception <AUDITORY@xxxxxxxx> on behalf of Jan Fel= cyn <janaku@xxxxxxxx><br> <b>Sent:</b> 14 January 2021 08:42<br> <b>To:</b> AUDITORY@xxxxxxxx <AUDITORY@xxxxxxxx><br> <b>Subject:</b> Re: [AUDITORY] apps for web-based listening tests</font> <div> </div> </div> <div dir=3D"ltr" style=3D"word-wrap:break-word"> <div dir=3D"ltr"> <div style=3D"font-size:12pt; font-family:'Calibri'; color:#000000"> <div>Dear List!</div> <div>Thank you all for your comments and interesting findings! Definitely w= e can choose something from your mails. Hopefully, we will run some experim= ents using these tools in the near future.</div> <div>Regards</div> <div>Jan</div> <div style=3D"font-size:small; text-decoration:none; font-family:"Cali= bri"; font-weight:normal; color:#000000; font-style:normal; display:in= line"> <div style=3D"font:10pt tahoma"> <div> </div> <div style=3D"background:#f5f5f5"> <div style=3D""><b>From:</b> <a title=3D"cstecker@xxxxxxxx">Chris= Stecker</a> </div> <div><b>Sent:</b> Wednesday, January 13, 2021 7:57 PM</div> <div><b>To:</b> <a title=3D"AUDITORY@xxxxxxxx">AUDITORY@xxxxxxxx= .CA</a> </div> <div><b>Subject:</b> Re: apps for web-based listening tests</div> </div> </div> <div> </div> </div> <div style=3D"font-size:small; text-decoration:none; font-family:"Cali= bri"; font-weight:normal; color:#000000; font-style:normal; display:in= line"> Hi Jan and List, <div> </div> <div>I=92d like to point you to a resource that you might find helpful in d= eveloped you web-based or other remote testing approach: The =93Remote Test= ing Wiki=94 at <a href=3D"https://eur02.safelinks.protection.outlook.com/?url=3Dhttps%3A%2= F%2Fspatialhearing.org%2Fremotetesting&data=3D04%7C01%7Ch.lee%40HUD.AC.= UK%7Cf2d3394b57d6491e808408d8b86d7275%7Cb52e9fda06914585bdfc5ccae1ce1890%7C= 0%7C0%7C637462127504751601%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQ= IjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3DkpQaTTsziYBkD= Rhsh8AooinVoKXCYceRZzbHReldkS0%3D&reserved=3D0" originalsrc=3D"https://= spatialhearing.org/remotetesting" shash=3D"AkDYbSThvHY51rX1kxA/K/bTUb1Uv2O+= Xu/pLuT1MEyRIClDMSC9anKnCctwOG1++BlqDYshUutZPRjArDAnwFVo581T5W2AOfnyu1BvuHc= 7LlMx9zmvRv+ikJd87eHSKjL7aCjU3IOlPmLTDw98w+ed88Kx2qLmb5BH5iAKZ5g=3D"> https://spatialhearing.org/remotetesting</a>. </div> <div> </div> <div>The Wiki is the result of activities by the Remote Testing Task Force,= which was launched last year by the Acoustical Society of America Technica= l Committee on Psychological and Physiological Acoustics. It includes infor= mation about issues and practices for remote testing, a database of platforms that can be used to develop an= d deploy remote testing research, and a variety of other resources includin= g brief reports of project experiences, bibliography of related works, and = links to other online resources. </div> <div> </div> <div>The Platform Survey Database includes information that Jan could use t= o find and select platforms, including several Python-based approaches, tha= t Task Force members and others have evaluated. </div> <div> </div> <div>Finally, the Wiki has a lot of information in it currently but is stil= l growing. We still need help identifying and evaluating platforms the we m= ight not know about yet. There are links for submitting new examples and pl= atformsSo if you have experience with a platform (for example, Borys seems familiar with SenseLabOnline), i= t would be GREAT if you could visit the wiki and submit information about i= t. That goes for new platforms and for updating information about features = of existing platforms. There is a simple survey tool that just takes a few minutes. </div> <div> </div> <div>Feel free to reach out to me (or better yet, via the contact links on = the Wiki) if you have additional question. </div> <div> </div> <div>Chris </div> <div> </div> <div> <div> </div> <div> <div>=97<br> <br> G. Christopher Stecker, PhD, FASA<br> <br> Coordinating Editor, Psychological and Physiological Acoustics<br> Journal of the Acoustical Society of America<br> <br> Chair, Task Force on Remote Testing<br> ASA Technical Committee on Psychological and Physiological Acoustics<br> <br> <a>cstecker@xxxxxxxx</a><br> <br> <br> </div> </div> <div><br> <blockquote type=3D"cite"> <div>On Jan 12, 2021, at 11:46 PM, Borys Kowalewski <<a>borys.k@xxxxxxxx= M</a>> wrote:</div> <div> </div> <div> <div>Hello Jan,</div> <div> </div> <div>Good to hear from you!</div> <div> </div> <div>For paired comparisons, ABX etc. you might find SenseLabOnline interes= ting. It allows you to create a multidimensional design and prepare a simpl= e graphical summary. You can also download the raw data in a JSON file for = further analysis. However, it is very much a quality- and preference-evaluation tool, and except for the AB= X it does not support the typical =93performance-based=94 n-AFC tests with = wrong and correct answers. Adaptive procedures are not an option either, si= nce all the tests require pre-loading of wave files.</div> <div> </div> <div>Hopefully other list members can suggest viable alternatives :)</div> <div> </div> <div>All best,</div> <div>Borys</div> <div> <div> </div> <div class=3D"x_gmail_quote"> <div class=3D"x_gmail_attr" dir=3D"ltr">On Wed, Jan 13, 2021 at 06:23 Jan F= elcyn <<a>janaku@xxxxxxxx</a>> wrote:<br> </div> <blockquote class=3D"x_gmail_quote" style=3D"padding-left:1ex; border-left:= rgb(204,204,204) 1px solid; margin:0px 0px 0px 0.8ex"> <div dir=3D"ltr"> <div dir=3D"ltr"> <div style=3D"font-size:12pt; font-family:calibri"> <div style=3D"font-family:calibri">Dear List!</div> <div style=3D"font-family:calibri"> </div> <div style=3D"font-family:calibri">I am looking for some web-based apps whi= ch allow to create listening tests. By creating listening tests I mean rath= er simple approach =96 like e.g. tests from Music Lab. Just playin=92 stere= o audio files, gathering responses and aggregate them in one file. One important feature is that it should be pos= sible to create psychoacoustic tests like pair comparisons, 2AFS and so on.= </div> <div style=3D"font-family:calibri"> </div> <div style=3D"font-family:calibri">Do you know any web-based platform for s= uch thing? I=92ve heard about BRAMS but never used it. I know Python and I = know that I can use Django, but maybe there is something already prepared f= or such tests?</div> <div style=3D"font-family:calibri"> </div> <div style=3D"font-family:calibri">Thanks in advance for all comments!</div= > <div style=3D"font-family:calibri"> </div> <div style=3D"font-family:calibri">Regards</div> <div style=3D"font-family:calibri"> </div> <div style=3D"font-family:calibri">---</div> <div style=3D"font-family:calibri">Jan Felcyn PhD</div> <div style=3D"font-family:calibri">Chair of Acoustics,</div> <div style=3D"font-family:calibri">Adam Mickiewicz University,</div> <div style=3D"font-family:calibri">Pozna=F1, Poland</div> </div> </div> </div> </blockquote> </div> </div> </div> </blockquote> </div> <div> </div> </div> </div> </div> </div> </div> University of Huddersfield inspiring global professionals.<br> <img src=3D"http://marketing.hud.ac.uk/_HOSTED/EmailSig2014/EmailSigFooterM= arch2019.jpg"><br> <br> This transmission is confidential and may be legally privileged. If you rec= eive it in error, please notify us immediately by e-mail and remove it from= your system. If the content of this e-mail does not relate to the business= of the University of Huddersfield, then we do not endorse it and will accept no liability. </body> </html> --_000_VI1PR01MB38708F7B061BAEAB6C7C6692B5A70VI1PR01MB3870eurp_--