Subject: Re: [AUDITORY] arXiv versus the peer machines we trust From: Matt Flax <flatmax@xxxxxxxx> Date: Thu, 1 Jun 2023 22:44:58 +1000--0000000000008a240d05fd10cee6 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable The history of the peer review practice is fascinating. I definitely agree with Matt that the Ivory tower is still unobtainable for large segments of society and it is definitely growing taller and harder to enter in a lot of societies, I am grateful for the opportunity to have access in my own way. Often we can tease out people's theoretical opinions (and sometimes come to surprising results) by taking a complex system and performing peripheral mind experiments at the edge of what is currently known and common opinion. If Kurzweil is right, then it is likely that by 2029 computers will be able to reason on par with humans. The question of what the implications are around machines of utility being available on the large scale for creating more accessible information, support, companionship and probably learning for people is still open. One thing is for certain, if a company or institution can speed its system with machines, it probably will. If future peer reviews in journals are conducted by machines, in an effort to speed up the system, improve bandwidth, profit and compete with preprint servers, is that still legitimate peer reviewed publishing in your opinion? Does this machine led scenario change anything for anyone who uses preprint servers ? Does this machine led scenario change anything for people who use peer reviewed journals ? Are we going to have/need a third class of publishing which is biologically based ? We have a spread of classical to modern preference for information dissemination. In a little more detail: Dick pointed out earlier (in the quality vs speed thread) that he was happy to publish his Gamma Distribution research on a preprint server and wait for peer reviewed publication (then was faced with maintaining the arXiv version). In that way some people are happy to use both modes of release. Then some people are pretty keen to maintain classical peer review mechanisms, whilst others prefer a more modern approach for various reasons. Do people with classical preference get split into classical and neoclassical ? How does the parameter space change from our current one dimensional (possibly periodic) spread ? Matt On Thu, Jun 1, 2023 at 2:12=E2=80=AFPM Matthew Winn < 0000011b522b2e6a-dmarc-request@xxxxxxxx> wrote: > There are statements in this thread that cannot go unchallenged, because > they condone and perpetuate harmful ideas that need to end. Specifically: > 1) =E2=80=9CIf one is not a sufficiently confident and independent thinke= r such > that one can express ideas, arguments, disagreements, etc. with anyone in > the field, regardless of stature, then that is a weakness=E2=80=9D > This statement ignores the multiple power structures that affect the live= s > and employment of those below the =E2=80=98upper echelon=E2=80=99 in the = field. Expressing > an idea involves risk when your position is precarious. Adapting to and > weighing that risk is a key survival strategy, not a weakness. I have a > blind spot for this risk =E2=80=93 not because I=E2=80=99m so great at sc= ience, but because > my culture gives me unearned respect because of my demographics. For peop= le > like me (and, I will note, virtually everyone on this thread), we live in= a > culture that insulates us from any sense that our voice doesn=E2=80=99t b= elong. > > 2) =E2=80=9Cthink about how such researchers earned such status. It was = not > because they had friends, it was not because people liked them. It was > because they established a track-record of contributions that the field, = in > general, held in very high regard.=E2=80=9D > This is a self-serving narrative that reflects survivorship bias and whic= h > ignores everything we know about how people act in real life. Science is > done by humans, who have personal interests, biases, and who live within = a > culture where status is built on many layers of privilege. Every decision > we make is filtered by these factors, which allow some people (like me) t= o > accumulate a variety of advantages at every career stage, simply because = of > how they look, who their friends are, and where they grew up. They are mo= re > likely to have papers accepted, to be selected for podium presentations, = to > have a job application reviewed, to be interviewed, to be hired, to be > selected as editors and reviewers, to be elected to positions of > leadership, and to be given favorable treatment in the workplace. To be > taken seriously. If we pretend that these advantages are ALL due to the > scientific merit of one=E2=80=99s work, we are characterizing scientists = as some > species entirely separate from the rest of humanity. > > 3) =E2=80=9CStature does not count. Everyone should be held to the very s= ame > standard=E2=80=9D > We all agree that work should not be judged on the basis of who wrote it. > But importantly, the influence of stature doesn=E2=80=99t need to be expl= icitly > suggested in order to actually take place. Similar to the last point, the > idea of equal standards and equal treatment is a convenient fiction that > allows people like me to feel superior because I can attribute my success > to my own hard work and merit, even though many factors that led to that > success were unearned. > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D > What does this have to do with preprints? The point is to consider that > others have a different set of constraints, and that our definitions of > merit are tailored to suit those who are already enjoying a wide variety = of > privileges. Consider the forces that lead authors to think that preprints > are useful, and also whether you are facing the same expectations and > constraints that they are. Numerous people have pointed at the apparent > generational divide on this issue - let's figure out why. Graduate > admissions and fellowship review increasingly expect a publication record > that far exceeds anything that would have been expected of the reviewers > when they were at that same career stage. For various reasons, the timeli= ne > of publication is increasingly long. Exacerbating this, it is no longer > enough to simply conduct a good study; one must also curate a data > management and sharing plan that includes open-access data and documented > code. One must learn and conduct the latest statistical techniques that > their advisors never needed to learn, and sift through a much broader set > of literature that includes a lot of garbage. To compete for stable > employment, younger scholars need an internet presence and must learn to > incorporate inclusive language in their writing, even if that were not pa= rt > of their training. They need to express how their work contributes to the > reduction of harm in society, despite being advised by some of the people > who are doing the harm. > > Preprints are not a magical solution that can eliminate the multiple > barriers that I described above. But they have tangible value, and reflec= t > adaptation to a changing academic landscape, rather than reflecting some > loss of =E2=80=9Cstandards=E2=80=9D that are designed to protect those al= ready at the top, > and which were established under an entirely different system of > constraints. > > Preprints help address the needs for 1) visibility and 2) quicker feedbac= k > on your work from a wider variety of scholars who might not have been > invited to review, simply because they were not in the network of the > associate editor. These factors are often yoked together; the channels th= at > spread awareness of a preprint (like Twitter) might also be the same > channels that generate discussion that becomes useful feedback. The > tendency (or need) to use these dissemination channels probably reinforce= s > the generational divide on this thread. I assure you that the comments I'= ve > received from people enthusiastic enough to read a preprint have had > meaningful influence and value. And those comments can come from a wider > variety of people whose opinions have been historically discounted. > Experienced reviewers will always have a place in our scientific discours= e, > but to discount the benefit and potential of preprints is to be willfully > detached from our current reality. > > Matt > --0000000000008a240d05fd10cee6 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr">The history of the peer review practice i= s fascinating. I definitely=C2=A0agree with Matt that the Ivory tower is st= ill unobtainable for large segments of society and it is definitely growing= taller and=C2=A0 harder to enter in a lot of societies, I am grateful for = the opportunity to have access in my own way.<div><br></div><div>Often we c= an tease out people's theoretical opinions (and sometimes come to surpr= ising results) by taking a complex system and performing peripheral mind ex= periments at the edge of what is currently known and common opinion.</div><= div><br></div><div>If Kurzweil is right, then it is likely that by 2029 com= puters will be able to reason on par with humans. The question=C2=A0of what= the implications are around machines of utility being available on the=C2= =A0large scale for creating more accessible information, support, companion= ship and probably learning for people is still open. One thing is for certa= in, if a company or institution can speed its system with machines, it prob= ably will.</div><div><br></div><div>If future peer reviews in journals are = conducted by machines, in an effort to speed up the system, improve bandwid= th, profit and compete with preprint servers, is that still legitimate peer= reviewed publishing in your opinion? Does this machine led scenario change= anything for anyone who uses preprint servers ? Does this machine led scen= ario change anything for people who use peer reviewed journals ? Are we goi= ng to have/need a third class of publishing which is biologically based ?</= div><div><br></div><div><div>We have a spread of classical to modern prefer= ence for information dissemination. In a little more detail: Dick pointed o= ut earlier (in the quality vs speed thread) that he was happy to=C2=A0 publ= ish his Gamma Distribution research on a preprint server and wait for peer = reviewed publication (then was faced with maintaining the arXiv version). I= n that way some people are happy to use both modes of release. Then some pe= ople are pretty keen to maintain classical peer review mechanisms, whilst o= thers prefer a more modern approach for various reasons.</div></div><div><b= r></div><div>Do people with classical preference get split into classical a= nd neoclassical ?</div><div>How does the parameter space change from our cu= rrent one dimensional (possibly periodic) spread ?</div><div><br></div><div= >Matt</div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"g= mail_attr">On Thu, Jun 1, 2023 at 2:12=E2=80=AFPM Matthew Winn <<a href= =3D"mailto:0000011b522b2e6a-dmarc-request@xxxxxxxx">0000011b522b2e6a= -dmarc-request@xxxxxxxx</a>> wrote:<br></div><blockquote class=3D= "gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(2= 04,204,204);padding-left:1ex"><div dir=3D"ltr"><div dir=3D"ltr">There are s= tatements in this thread that cannot go unchallenged, because they condone = and perpetuate harmful ideas that need to end. Specifically:<br>1) =E2=80= =9CIf one is not a sufficiently confident and independent thinker such that= one can express ideas, arguments, disagreements, etc. with anyone in the f= ield, regardless of stature, then that is a weakness=E2=80=9D<br>This state= ment ignores the multiple power structures that affect the lives and employ= ment of those below the =E2=80=98upper echelon=E2=80=99 in the field. Expre= ssing an idea involves risk when your position is precarious. Adapting to a= nd weighing that risk is a key survival strategy, not a weakness. I have a = blind spot for this risk =E2=80=93 not because I=E2=80=99m so great at scie= nce, but because my culture gives me unearned respect because of my demogra= phics. For people like me (and, I will note, virtually everyone on this thr= ead), we live in a culture that insulates us from any sense that our voice = doesn=E2=80=99t belong. <br><br>2) =E2=80=9Cthink about how such researcher= s earned such status.=C2=A0 It was not because they had friends, it was not= because people liked them.=C2=A0 It was because they established a track-r= ecord of contributions that the field, in general, held in very high regard= .=E2=80=9D<br>This is a self-serving narrative that reflects survivorship b= ias and which ignores everything we know about how people act in real life.= Science is done by humans, who have personal interests, biases, and who li= ve within a culture where status is built on many layers of privilege. Ever= y decision we make is filtered by these factors, which allow some people (l= ike me) to accumulate a variety of advantages at every career stage, simply= because of how they look, who their friends are, and where they grew up. T= hey are more likely to have papers accepted, to be selected for podium pres= entations, to have a job application reviewed, to be interviewed, to be hir= ed, to be selected as editors and reviewers, to be elected to positions of = leadership, and to be given favorable treatment in the workplace. To be tak= en seriously. If we pretend that these advantages are ALL due to the scient= ific merit of one=E2=80=99s work, we are characterizing scientists as some = species entirely separate from the rest of humanity. <br><br>3) =E2=80=9CSt= ature does not count. Everyone should be held to the very same standard=E2= =80=9D<br>We all agree that work should not be judged on the basis of who w= rote it. But importantly, the influence of stature doesn=E2=80=99t need to = be explicitly suggested in order to actually take place. Similar to the las= t point, the idea of equal standards and equal treatment is a convenient fi= ction that allows people like me to feel superior because I can attribute m= y success to my own hard work and merit, even though many factors that led = to that success were unearned.<br><br>=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D <br>What does this have to do with = preprints? The point is to consider that others have a different set of con= straints, and that our definitions of merit are tailored to suit those who = are already enjoying a wide variety of privileges. Consider the forces that= lead authors to think that preprints are useful, and also whether you are = facing the same expectations and constraints that they are. Numerous people= have pointed at the apparent generational divide on this issue - let's= figure out why. Graduate admissions and fellowship review increasingly exp= ect a publication record that far exceeds anything that would have been exp= ected of the reviewers when they were at that same career stage. For variou= s reasons, the timeline of publication is increasingly long. Exacerbating t= his, it is no longer enough to simply conduct a good study; one must also c= urate a data management and sharing plan that includes open-access data and= documented code. One must learn and conduct the latest statistical techniq= ues that their advisors never needed to learn, and sift through a much broa= der set of literature that includes a lot of garbage. To compete for stable= employment, younger scholars need an internet presence and must learn to i= ncorporate inclusive language in their writing, even if that were not part = of their training. They need to express how their work contributes to the r= eduction of harm in society, despite being advised by some of the people wh= o are doing the harm.</div><div dir=3D"ltr"><br>Preprints are not a magical= solution that can eliminate the multiple barriers that I described above. = But they have tangible value, and reflect adaptation to a changing academic= landscape, rather than reflecting some loss of =E2=80=9Cstandards=E2=80=9D= that are designed to protect those already at the top, and which were esta= blished under an entirely different system of constraints.</div><div dir=3D= "ltr">=C2=A0 <br>Preprints help address the needs for 1) visibility and 2) = quicker feedback on your work from a wider variety of scholars who might no= t have been invited to review, simply because they were not in the network = of the associate editor. These factors are often yoked together; the channe= ls that spread awareness of a preprint (like Twitter) might also be the sam= e channels that generate discussion that becomes useful feedback. The tende= ncy (or need) to use these dissemination channels probably reinforces the g= enerational divide on this thread. I assure you that the comments I've = received from people enthusiastic enough to read a preprint have had meanin= gful influence and value. And those comments can come from a wider variety = of people whose opinions have been historically discounted. Experienced rev= iewers will always have a place in our scientific discourse, but to discoun= t the benefit and potential of preprints is to be willfully detached from o= ur current reality.<br><br>Matt<br></div></div> </blockquote></div></div> --0000000000008a240d05fd10cee6--