Subject: Re: [AUDITORY] apps for web-based listening tests From: Sam Mathias <samuel.mathias@xxxxxxxx> Date: Wed, 13 Jan 2021 07:37:30 -0500--000000000000bafbcc05b8c76781 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Gin Best et al. created this extremely helpful wiki on the topic of remote psychoacoustical testing: https://www.spatialhearing.org/remotetesting/ I used this resource and others to make a decision for our lab. We went with Gorilla (https://gorilla.sc/), which we chose after surveying the available options because: 1. It was not self-hosted, so we didn't have to set up our own server. 2. Many simple tests can be created without code. We had ~20 tests to deploy quickly, and didn't have the time/manpower to learn a new web framework. 3. Easily presents pre-generated audio, video, and images. 4. Reasonably cheap, $1-2 per participant per experiment, and experiments can be many sessions. 5. We know some folks who are also using it and they seem happy enough. 6. They have a helpful, responsive support team. 7. They have published some of their data on reaction times. Potential disadvantages: 1. Suitable for simple designs (e.g., 2AFC w/ method of constant stimuli), perhaps not for others (e.g., staircases). 2. Data munging and analysis must be done offline. 3. While you can add your own JavaScript for special functionality, their API seems a bit clunky and limited. 4. We've not actually rolled out our studies yet. Perhaps my opinion might change once we've collected real data. Hope this is helpful. On Wed, 13 Jan 2021 at 03:46, Borys Kowalewski <borys.k@xxxxxxxx> wrote: > Hello Jan, > > Good to hear from you! > > For paired comparisons, ABX etc. you might find SenseLabOnline > interesting. It allows you to create a multidimensional design and prepar= e > a simple graphical summary. You can also download the raw data in a JSON > file for further analysis. However, it is very much a quality- and > preference-evaluation tool, and except for the ABX it does not support th= e > typical =E2=80=9Cperformance-based=E2=80=9D n-AFC tests with wrong and co= rrect answers. > Adaptive procedures are not an option either, since all the tests require > pre-loading of wave files. > > Hopefully other list members can suggest viable alternatives :) > > All best, > Borys > > On Wed, Jan 13, 2021 at 06:23 Jan Felcyn <janaku@xxxxxxxx> wrote: > >> Dear List! >> >> I am looking for some web-based apps which allow to create listening >> tests. By creating listening tests I mean rather simple approach =E2=80= =93 like >> e.g. tests from Music Lab. Just playin=E2=80=99 stereo audio files, gath= ering >> responses and aggregate them in one file. One important feature is that = it >> should be possible to create psychoacoustic tests like pair comparisons, >> 2AFS and so on. >> >> Do you know any web-based platform for such thing? I=E2=80=99ve heard ab= out BRAMS >> but never used it. I know Python and I know that I can use Django, but >> maybe there is something already prepared for such tests? >> >> Thanks in advance for all comments! >> >> Regards >> >> --- >> Jan Felcyn PhD >> Chair of Acoustics, >> Adam Mickiewicz University, >> Pozna=C5=84, Poland >> > --000000000000bafbcc05b8c76781 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">Gin Best et al. created this extremely helpful wiki on the= topic of remote psychoacoustical testing: <a href=3D"https://www.spatialhe= aring.org/remotetesting/">https://www.spatialhearing.org/remotetesting/</a>= <div><br></div><div>I used this resource and others to make a decision for = our lab. We went with Gorilla (<a href=3D"https://gorilla.sc/">https://gori= lla.sc/</a>), which we chose after surveying the available options because:= </div><div><ol><li>It was not=C2=A0self-hosted, so we didn't have to se= t up our own server.</li><li>Many simple tests can be created without code.= We had ~20 tests to deploy quickly, and didn't have the time/manpower = to learn a new web framework.</li><li>Easily presents pre-generated audio, = video, and images.</li><li>Reasonably cheap, $1-2 per participant per exper= iment, and experiments can be many sessions.</li><li>We know some folks who= are also using it and they seem happy enough.</li><li>They have a helpful,= responsive support team.</li><li>They have published some of their data on= reaction times.</li></ol><div>Potential disadvantages:</div></div><div><ol= ><li>Suitable for simple designs (e.g., 2AFC w/ method of constant stimuli)= , perhaps not for others (e.g., staircases).</li><li>Data munging and analy= sis must be done offline.</li><li>While you can add your own JavaScript for= special functionality, their API seems a bit clunky and limited.</li><li>W= e've not actually rolled out our studies yet. Perhaps my opinion might= change once we've collected real data.</li></ol><div>Hope this is help= ful.</div></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class= =3D"gmail_attr">On Wed, 13 Jan 2021 at 03:46, Borys Kowalewski <<a href= =3D"mailto:borys.k@xxxxxxxx">borys.k@xxxxxxxx</a>> wrote:<br></div><bl= ockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-lef= t:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"auto">Hello Jan,= </div><div dir=3D"auto"><br></div><div dir=3D"auto">Good to hear from you!<= /div><div dir=3D"auto"><br></div><div dir=3D"auto">For paired comparisons, = ABX etc. you might find SenseLabOnline interesting. It allows you to create= a multidimensional design and prepare a simple graphical summary. You can = also download the raw data in a JSON file for further analysis. However, it= is very much a quality- and preference-evaluation tool, and except for the= ABX it does not support the typical =E2=80=9Cperformance-based=E2=80=9D n-= AFC tests with wrong and correct answers. Adaptive procedures are not an op= tion either, since all the tests require pre-loading of wave files.</div><d= iv dir=3D"auto"><br></div><div dir=3D"auto">Hopefully other list members ca= n suggest viable alternatives :)</div><div dir=3D"auto"><br></div><div dir= =3D"auto">All best,</div><div dir=3D"auto">Borys</div><div><br><div class= =3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, Jan 13, 2021= at 06:23 Jan Felcyn <<a href=3D"mailto:janaku@xxxxxxxx" target=3D"_bl= ank">janaku@xxxxxxxx</a>> wrote:<br></div><blockquote class=3D"gmail_q= uote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,2= 04);padding-left:1ex"> <div dir=3D"ltr"> <div dir=3D"ltr"> <div style=3D"font-size:12pt;font-family:Calibri;color:rgb(0,0,0)"> <div style=3D"font-family:Calibri">Dear List!</div> <div style=3D"font-family:Calibri">=C2=A0</div> <div style=3D"font-family:Calibri">I am looking for some web-based apps whi= ch allow to create listening tests.=20 By creating listening tests I mean rather simple approach =E2=80=93 like e.= g. tests from=20 Music Lab. Just playin=E2=80=99 stereo audio files, gathering responses and= aggregate=20 them in one file. One important feature is that it should be possible to cr= eate=20 psychoacoustic tests like pair comparisons, 2AFS and so on.</div> <div style=3D"font-family:Calibri">=C2=A0</div> <div style=3D"font-family:Calibri">Do you know any web-based platform for s= uch thing? I=E2=80=99ve heard about BRAMS=20 but never used it. I know Python and I know that I can use Django, but mayb= e=20 there is something already prepared for such tests?</div> <div style=3D"font-family:Calibri">=C2=A0</div> <div style=3D"font-family:Calibri">Thanks in advance for all comments!</div= > <div style=3D"font-family:Calibri">=C2=A0</div> <div style=3D"font-family:Calibri">Regards</div> <div style=3D"font-family:Calibri">=C2=A0</div> <div style=3D"font-family:Calibri">---</div> <div style=3D"font-family:Calibri">Jan Felcyn PhD</div> <div style=3D"font-family:Calibri">Chair of Acoustics,</div> <div style=3D"font-family:Calibri">Adam Mickiewicz University,</div> <div style=3D"font-family:Calibri">Pozna=C5=84, Poland</div></div></div></d= iv> </blockquote></div></div> </blockquote></div> --000000000000bafbcc05b8c76781--