Subject: [AUDITORY] CHiME 2024 Challenge - call for proposals / deadline 25th Sept From: Jon Barker <j.p.barker@xxxxxxxx> Date: Tue, 1 Aug 2023 10:53:26 +0100--000000000000625eea0601d985f7 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable *https://www.chimechallenge.org <https://www.chimechallenge.org/>* *Submission deadline - 25 Sept 2023* --- CHiME is a challenge series whose purpose is to produce scientific advances in the field of distant microphone speech processing. The challenge=E2=80= =99s six previous iterations from 2011 until 2020 have resulted in major improvements in word error rate and diarisation error rate for single-speaker voice commands in various environments and for multi-speaker conversations in homes. These six previous iterations have been centrally organised. Starting from last year, we have now introduced a more open model, where the annual CHiME Challenge features independent tasks organised by teams who work within a schedule defined by the CHiME Steering Group. Teams will be responsible for managing the tasks, i.e., providing data and instructions, scoring submissions and results, etc. The CHiME Steering Group will then work to support teams in the development of their task. The Steering Group will also organise a CHiME workshop where the tasks will be presented. The CHiME Steering group is now inviting task proposals for the CHiME-2024 Challenge. Proposed tasks will be evaluated according to the following criteria: - relevance to the CHiME goals - methodology - i.e., likelihood to lead to robust and novel scientific findings - accessibility - i.e., how easy is it for teams to engage with - feasibility - likelihood to complete on schedule, risk management - reproducibility - e.g., openness of data, baseline - diversity - amongst organisers and expected participants, e.g., academia vs industry, geography, gender etc. We wish to foster a collaborative proposal process that minimises overlap between proposals. To this end, if you are considering proposing a task, please sign up to the CHiME Google group and feel free to discuss your plans. Or if you wish information to be shared more privately, i.e., amongst other bidders only, then you may share details or initial plans with the CHiME Steering Group who will then share with others whose plans may overlap. *Proposal* Task bidders should submit a task proposal for review by the Steering Group. The proposal should be a maximum of two pages (pdf format preferred) including the following information: - Task name - Coordinators and their roles - Keywords (e.g. enhancement, diarization, speaker recognition, etc) - Definition (one sentence) - Short description (incl. research question the task is tackling) - Dataset description: development, evaluation (short description, how much data is already available and prepared, how long would it take to prepare the rest, mention if you allow external data/transfer learning o= r not) - Evaluation method / metric - Baseline system (brief description, planned method if you do not have one from a previous challenge, current readiness including an expected readiness date) - List of invited participants who are likely to participate - Contact person (for main communication, website) *Pitching Session* We will be running an *optional* =E2=80=9Cproposal pitching=E2=80=9D sessi= on at the CHiME-2023 workshop in Dublin, 25th August. This will be an opportunity for teams who are developing a submission to get some feedback from members of the CHiME community. If you are interested in presenting in this session please contact us to discuss. *Submission* The proposal should be sent to the CHiME Steering group by 26th September. The Steering Group will review the proposals and provide feedback by 10th October. In case task amendments are requested, the task acceptance decision will be finalized by 4th November. If you want any advice on preparing your proposal, please do not hesitate to contact us. *Important dates* The tentative schedule for CHiME-2024 challenge round is as follows. - 25 Aug 2023 - Pitching session at CHiME-2023 workshop - 25 Sept 2023 - Submission of task proposals - 9 Oct 2023 - Feedback from Steering Group - 20 Oct 2023 - Resubmission of conditionally accepted proposals - 3 Nov 2023 - Notification of acceptance - 1 Feb 2024 - Challenge opening - 28 June 2024 - Challenge Submission deadline - 6 Sep 2024 - Interspeech satellite workshop (date and details TBC) For further information visit https://www.chimechallenge.org CHiME Steering Group - Jon Barker Univ. of Sheffield, UK - Emmanuel Vincent Inria, France - Shinji Watanabe Carnegie Mellon Univ., USA - Michael Mandel CUNY/META, USA - Marc Delcroix NTT, Japan - Leibny Paola Garcia Perera Johns Hopkins Univ., USA --000000000000625eea0601d985f7 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div style=3D"color:rgba(0,0,0,0.87);font-family:Roboto,Ro= botoDraft,Helvetica,Arial,sans-serif;font-size:14px"><div><div><b><a href= =3D"https://www.chimechallenge.org/" rel=3D"nofollow" target=3D"_blank" sty= le=3D"text-decoration-line:none;color:rgb(26,115,232)">https://www.chimecha= llenge.org</a></b></div></div></div><div style=3D"color:rgba(0,0,0,0.87);fo= nt-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:14px"><b>= Submission deadline - 25 Sept 2023</b></div><div style=3D"color:rgba(0,0,0,= 0.87);font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:1= 4px">---</div><div style=3D"color:rgba(0,0,0,0.87);font-family:Roboto,Robot= oDraft,Helvetica,Arial,sans-serif;font-size:14px"><br></div><div style=3D"c= olor:rgba(0,0,0,0.87);font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-s= erif;font-size:14px">CHiME is a challenge series whose purpose is to produc= e scientific advances in the field of distant microphone speech processing.= The challenge=E2=80=99s six previous iterations from 2011 until 2020 have = resulted in major improvements in word error rate and diarisation error rat= e for single-speaker voice commands in various environments and for multi-s= peaker conversations in homes. These six previous iterations have been cent= rally organised.<br><br>Starting from last year, we have now introduced a m= ore open model, where the annual CHiME Challenge features independent tasks= organised by teams who work within a schedule defined by the CHiME Steerin= g Group. Teams will be responsible for managing the tasks, i.e., providing = data and instructions, scoring submissions and results, etc. The CHiME Stee= ring Group will then work to support teams in the development of their task= . The Steering Group will also organise a CHiME workshop where the tasks wi= ll be presented.<br><br>The CHiME Steering group is now inviting task propo= sals for the CHiME-2024 Challenge. Proposed tasks will be evaluated accordi= ng to the following criteria:<br><ul><li>relevance to the CHiME goals</li><= li>methodology - i.e., likelihood to lead to robust and novel scientific fi= ndings</li><li>accessibility - i.e., how easy is it for teams to engage wit= h</li><li>feasibility - likelihood to complete on schedule, risk management= </li><li>reproducibility - e.g., openness of data, baseline</li><li>diversi= ty - amongst organisers and expected participants, e.g., academia vs indust= ry, geography, gender etc.</li></ul>We wish to foster a collaborative propo= sal process that minimises overlap between proposals. To this end, if you a= re considering proposing a task, please sign up to the CHiME Google group a= nd feel free to discuss your plans. Or if you wish information to be shared= more privately, i.e., amongst other bidders only, then you may share detai= ls or initial plans with the CHiME Steering Group who will then share with = others whose plans may overlap.<br></div><br style=3D"color:rgba(0,0,0,0.87= );font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:14px"= ><div style=3D"color:rgba(0,0,0,0.87);font-family:Roboto,RobotoDraft,Helvet= ica,Arial,sans-serif;font-size:14px"><b>Proposal</b></div><br style=3D"colo= r:rgba(0,0,0,0.87);font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-seri= f;font-size:14px"><div style=3D"color:rgba(0,0,0,0.87);font-family:Roboto,R= obotoDraft,Helvetica,Arial,sans-serif;font-size:14px">Task bidders should s= ubmit a task proposal for review by the Steering Group. The proposal should= be a maximum of two pages (pdf format preferred) including the following i= nformation:<br><ul><li>Task name</li><li>Coordinators and their roles</li><= li>Keywords (e.g. enhancement, diarization, speaker recognition, etc)</li><= li>Definition (one sentence)</li><li>Short description (incl. research ques= tion the task is tackling)</li><li>Dataset description: development, evalua= tion (short description, how much data is already available and prepared, h= ow long would it take to prepare the rest, mention if you allow external da= ta/transfer learning or not)</li><li>Evaluation method / metric</li><li>Bas= eline system (brief description, planned method if you do not have one from= a previous challenge, current readiness including an expected readiness da= te)</li><li>List of invited participants who are likely to participate</li>= <li>Contact person (for main communication, website)</li></ul></div><div st= yle=3D"color:rgba(0,0,0,0.87);font-family:Roboto,RobotoDraft,Helvetica,Aria= l,sans-serif;font-size:14px"><b>Pitching Session</b></div><div style=3D"col= or:rgba(0,0,0,0.87);font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-ser= if;font-size:14px"><b><br></b></div><div style=3D"color:rgba(0,0,0,0.87);fo= nt-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:14px">We = will be running an=C2=A0<b>optional</b>=C2=A0 =E2=80=9Cproposal pitching=E2= =80=9D session at the CHiME-2023 workshop in Dublin, 25th August. This will= be an opportunity for teams who are developing a submission to get some fe= edback from members of the CHiME community. If you are interested in presen= ting in this session please contact us to discuss.<br><br></div><div style= =3D"color:rgba(0,0,0,0.87);font-family:Roboto,RobotoDraft,Helvetica,Arial,s= ans-serif;font-size:14px"><b>Submission</b><br></div><div style=3D"color:rg= ba(0,0,0,0.87);font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;fo= nt-size:14px">The proposal should be sent to the CHiME Steering group by 26= th September. The Steering Group will review the proposals and provide feed= back by 10th October. In case task amendments are requested, the task accep= tance decision will be finalized by 4th November.<br><br>If you want any ad= vice on preparing your proposal, please do not hesitate to contact us.<br><= br></div><div style=3D"color:rgba(0,0,0,0.87);font-family:Roboto,RobotoDraf= t,Helvetica,Arial,sans-serif;font-size:14px"><b>Important dates</b></div><d= iv style=3D"color:rgba(0,0,0,0.87);font-family:Roboto,RobotoDraft,Helvetica= ,Arial,sans-serif;font-size:14px"><br></div><span style=3D"color:rgba(0,0,0= ,0.87);font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:= 14px">The tentative schedule for CHiME-2024 challenge round is as follows.<= /span><div style=3D"color:rgba(0,0,0,0.87);font-family:Roboto,RobotoDraft,H= elvetica,Arial,sans-serif;font-size:14px"><br><div>- 25 Aug 2023 - Pitching= session at CHiME-2023 workshop</div><div>- 25 Sept 2023 - Submission of ta= sk proposals</div><div>- 9 Oct 2023 - Feedback from Steering Group</div><di= v>- 20 Oct 2023 - Resubmission of conditionally accepted proposals</div><di= v>- 3 Nov 2023 - Notification of acceptance</div><div>- 1 Feb 2024 - Challe= nge opening</div><div>- 28 June 2024 - Challenge Submission deadline</div><= div>- 6 Sep 2024 - Interspeech satellite workshop (date and details TBC)</d= iv><br><div>For further information visit=C2=A0<a href=3D"https://www.chime= challenge.org/" rel=3D"nofollow" target=3D"_blank" style=3D"text-decoration= -line:none;color:rgb(26,115,232)">https://www.chimechallenge.org</a></div><= div><br></div><div>CHiME Steering Group=C2=A0</div><div>- Jon Barker Univ. = of Sheffield, UK<br>- Emmanuel Vincent Inria, France<br>- Shinji Watanabe C= arnegie Mellon Univ., USA<br>- Michael Mandel CUNY/META, USA<br>- Marc Delc= roix NTT, Japan<br>- Leibny Paola Garcia Perera Johns Hopkins Univ., USA</d= iv></div><div><br></div></div> --000000000000625eea0601d985f7--