From prechelt@inf.fu-berlin.de Thu Apr 02 09:58:22 2015 Received: from outpost1.zedat.fu-berlin.de ([130.133.4.66]) by list1.zedat.fu-berlin.de (Exim 4.85) for rqc-interest@lists.fu-berlin.de with esmtp (envelope-from ) id <1Yda16-000aMG-DM>; Thu, 02 Apr 2015 09:58:20 +0200 Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost.zedat.fu-berlin.de (Exim 4.85) for rqc-interest@lists.fu-berlin.de with esmtp (envelope-from ) id <1Yda16-00419v-CL>; Thu, 02 Apr 2015 09:58:20 +0200 Received: from kaduna.imp.fu-berlin.de ([160.45.118.73] helo=KADUNA) by inpost2.zedat.fu-berlin.de (Exim 4.85) for rqc-interest@lists.fu-berlin.de with esmtpsa (envelope-from ) id <1Yda16-000PfX-BA>; Thu, 02 Apr 2015 09:58:20 +0200 From: "Lutz Prechelt" To: Date: Thu, 2 Apr 2015 09:58:20 +0200 Message-ID: <0a2901d06d1a$c90a5820$5b1f0860$@inf.fu-berlin.de> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 15.0 Thread-Index: AdBtGEajV/w4B9s0RQGLbwDKmNbrHQ== Content-Language: de X-Originating-IP: 160.45.118.73 X-purgate: clean X-purgate-type: clean X-purgate-ID: 151147::1427961500-00000DE8-60B66AC9/0/0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.061284, version=1.2.4 X-Spam-Flag: NO X-Spam-Status: No, score=-50.0 required=5.0 tests=ALL_TRUSTED X-Spam-Checker-Version: SpamAssassin 3.4.0 on Vanuatu.ZEDAT.FU-Berlin.DE X-Spam-Level: Subject: [RQC-interest] Review Quality Collector (RQC) status update April 2015 X-BeenThere: rqc-interest@lists.fu-berlin.de X-Mailman-Version: 2.1.16 Precedence: list List-Id: sporadic information on Review Quality Collector List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Apr 2015 07:58:22 -0000 In the previous update [1], I told you that - we were going to run a survey regarding the participating editor's perceptions of the first Elsevier review quality grading pilot experiment (and of their review quality situation in general) and - we were looking for editors interested in trialing to develop a review quality _definition_, RQdef [2]. For the latter, one such editor has spoken up and I am just starting the discussion with him. For the former, the insights from the survey can be summarized as follows (the survey had 9 questions, Q1 to Q9, a mix of categorical, numerical and free text; the statistical base is small (53 respondents), but wide): - Q5+Q9: at least about one third of all reviews could be improved - Q6: editors know who the good reviewers will be, but getting them to agree is hard. We should market RQC by "reviewing is a very important and valuable part of your work -- and now visible too". - Q6: time delays created by non-refusing-but-should reviewers are a big problem; the RQ timeliness dimension should cover that. - Q3: most editors do not think about RQ criteria explicitly. - Q4: But they could. They only need some coaching. My personal conclusion is: RQC will not only provide benefit to reviewers and to research institutions, it will also be possible to convince many (if not all) editors that it is also beneficial for them. And finally, as usual: Please tell me if you know anybody else who may like to be on this mailing list [3]. Lutz [1] https://lists.fu-berlin.de/pipermail/rqc-interest/2014/msg00005.html [2] http://www.inf.fu-berlin.de/w/SE/ReviewQualityCollectorHome#Steps_44_Element s_44_and_Effects_of_the_RQC_Procedure [3] https://lists.fu-berlin.de/listinfo/rqc-interest (you can subscribe here) From prechelt@inf.fu-berlin.de Wed Jun 24 16:43:23 2015 Received: from outpost1.zedat.fu-berlin.de ([130.133.4.66]) by list1.zedat.fu-berlin.de (Exim 4.85) for rqc-interest@lists.fu-berlin.de with esmtp (envelope-from ) id <1Z7ltY-0001Sg-NR>; Wed, 24 Jun 2015 16:43:20 +0200 Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost.zedat.fu-berlin.de (Exim 4.85) for rqc-interest@lists.fu-berlin.de with esmtp (envelope-from ) id <1Z7ltY-0003zN-MO>; Wed, 24 Jun 2015 16:43:20 +0200 Received: from kaduna.imp.fu-berlin.de ([160.45.118.73] helo=KADUNA) by inpost2.zedat.fu-berlin.de (Exim 4.85) for rqc-interest@lists.fu-berlin.de with esmtpsa (envelope-from ) id <1Z7ltY-0001M9-Hn>; Wed, 24 Jun 2015 16:43:20 +0200 From: "Lutz Prechelt" To: Date: Wed, 24 Jun 2015 16:43:20 +0200 Message-ID: <019101d0ae8c$1d5dded0$58199c70$@inf.fu-berlin.de> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 15.0 Thread-Index: AdCuigp7Rhqkp3PDQCCh1nIy+zNkRg== Content-Language: de X-Originating-IP: 160.45.118.73 X-purgate: clean X-purgate-type: clean X-purgate-ID: 151147::1435157000-00000D33-DDB1E6A2/0/0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.390772, version=1.2.4 X-Spam-Flag: NO X-Spam-Status: No, score=-50.0 required=5.0 tests=ALL_TRUSTED X-Spam-Checker-Version: SpamAssassin 3.4.1 on Vanuatu.ZEDAT.FU-Berlin.DE X-Spam-Level: Subject: [RQC-interest] Review Quality Collector (RQC) status update June 2015 X-BeenThere: rqc-interest@lists.fu-berlin.de X-Mailman-Version: 2.1.16 Precedence: list List-Id: sporadic information on Review Quality Collector List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Jun 2015 14:43:23 -0000 In the previous update [1], I told you that - our survey of editors regarding reviewing/reviewers found that - at least about one third of all reviews could be improved, - but the biggest perceived problem is getting good reviewers to agree to review at all (so giving something in return may help) and that - it should likely be possible to convince editors of the usefulness of RQC. What happened since? I have now spoken with two editors and their corresponding publishers. A publisher is a person (with a suitable PhD) from the publishing company responsible to working with a handful of journals on their continued development. With these four people I will discuss how a review quality definition should look like to be suitable for the respective journal. We have started this a bit and have already found out it is far from straightforward; the tradeoffs involved are not simple. I have also visited an editorial board meeting for one of these two journals, explained the RQC idea, and found attitudes consistent with the survey results: positive reactions from some of the editors (about 10 were present) and skeptical ones from others. No strict opposition. And finally, as usual: Please tell me if you know anybody else who may like to be on this mailing list [2]. Lutz [1] https://lists.fu-berlin.de/pipermail/rqc-interest/2015/msg00000.html [2] https://lists.fu-berlin.de/listinfo/rqc-interest (you can subscribe here) From prechelt@inf.fu-berlin.de Thu Aug 20 13:40:14 2015 Received: from outpost1.zedat.fu-berlin.de ([130.133.4.66]) by list1.zedat.fu-berlin.de (Exim 4.85) for rqc-interest@lists.fu-berlin.de with esmtp (envelope-from ) id <1ZSOCa-002b92-Kr>; Thu, 20 Aug 2015 13:40:12 +0200 Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost.zedat.fu-berlin.de (Exim 4.85) for rqc-interest@lists.fu-berlin.de with esmtp (envelope-from ) id <1ZSOCa-001Hu4-Jo>; Thu, 20 Aug 2015 13:40:12 +0200 Received: from kaduna.imp.fu-berlin.de ([160.45.118.73] helo=KADUNA) by inpost2.zedat.fu-berlin.de (Exim 4.85) for rqc-interest@lists.fu-berlin.de with esmtpsa (envelope-from ) id <1ZSOCa-002M80-FR>; Thu, 20 Aug 2015 13:40:12 +0200 From: "Lutz Prechelt" To: Date: Thu, 20 Aug 2015 13:40:12 +0200 Message-ID: <016501d0db3c$f9815db0$ec841910$@inf.fu-berlin.de> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 15.0 Thread-Index: AdDYwOngW84RFI4uSGyuWcPuWVyUsA== Content-Language: de X-Originating-IP: 160.45.118.73 X-purgate: clean X-purgate-type: clean X-purgate-ID: 151147::1440070812-00000D05-83E2D702/0/0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.354798, version=1.2.4 X-Spam-Flag: NO X-Spam-Status: No, score=-50.0 required=5.0 tests=ALL_TRUSTED X-Spam-Checker-Version: SpamAssassin 3.4.1 on Tokelau.ZEDAT.FU-Berlin.DE X-Spam-Level: Subject: [RQC-interest] Review Quality Collector (RQC) status update August 2015 X-BeenThere: rqc-interest@lists.fu-berlin.de X-Mailman-Version: 2.1.16 Precedence: list List-Id: sporadic information on Review Quality Collector List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Aug 2015 11:40:14 -0000 In the previous update [1], I told you that I have started discussions with some journal editors in order to learn more about what an actual review quality definition might look like. What happened since? I have now seriously started to work towards a software implementation. The first step will be the (much simpler) version of RQC for conferences (RQCconf). Then I will develop that into the full version for journals (RQC proper). As a first functional part I have implemented reading a review quality definition from an RTF file, using a human-friendly format that will help discussing an emerging review quality definitions among the PC chairs (or journal editors) of one conference (journal). You can get a rough itemized look at the implementation status at [2]. And finally, as usual: Please tell me if you know anybody else who may like to be on this mailing list [3]. Lutz [1] https://lists.fu-berlin.de/pipermail/rqc-interest/2015/msg00001.html [2] http://www.mi.fu-berlin.de/w/SE/RQCconf#Software_implementation_status [3] https://lists.fu-berlin.de/listinfo/rqc-interest (you can subscribe here) From prechelt@inf.fu-berlin.de Tue Sep 22 18:33:17 2015 Received: from outpost1.zedat.fu-berlin.de ([130.133.4.66]) by list1.zedat.fu-berlin.de (Exim 4.85) for rqc-interest@lists.fu-berlin.de with esmtp (envelope-from ) id <1ZeQVH-0024Jl-U3>; Tue, 22 Sep 2015 18:33:16 +0200 Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost.zedat.fu-berlin.de (Exim 4.85) for rqc-interest@lists.fu-berlin.de with esmtp (envelope-from ) id <1ZeQVH-0012XE-T1>; Tue, 22 Sep 2015 18:33:15 +0200 Received: from kaduna.imp.fu-berlin.de ([160.45.118.73] helo=KADUNA) by inpost2.zedat.fu-berlin.de (Exim 4.85) for rqc-interest@lists.fu-berlin.de with esmtpsa (envelope-from ) id <1ZeQVH-0015vd-NW>; Tue, 22 Sep 2015 18:33:15 +0200 From: "Lutz Prechelt" To: Date: Tue, 22 Sep 2015 18:33:15 +0200 Message-ID: <036901d0f554$618a5ca0$249f15e0$@inf.fu-berlin.de> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 15.0 Thread-Index: AdD1U2L/l/Vc2FtRS/KibWZwO3RLNw== Content-Language: de X-Originating-IP: 160.45.118.73 X-purgate: clean X-purgate-type: clean X-purgate-ID: 151147::1442939595-00000D05-EED939AE/0/0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.422337, version=1.2.4 X-Spam-Flag: NO X-Spam-Status: No, score=-50.0 required=5.0 tests=ALL_TRUSTED X-Spam-Checker-Version: SpamAssassin 3.4.1 on Tuvalu.ZEDAT.FU-Berlin.DE X-Spam-Level: Subject: [RQC-interest] Review Quality Collector (RQC) status update September 2015 X-BeenThere: rqc-interest@lists.fu-berlin.de X-Mailman-Version: 2.1.16 Precedence: list List-Id: sporadic information on Review Quality Collector List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Sep 2015 16:33:18 -0000 In the previous update [1], I told you that I have now started to work on a software implementation of RQC, first for conferences, later for journals as well. Today I would like to share with you an anecdote: In the doctoral symposium at the top conference of my field, a super-respected colleague of mine gave a presentation on "Writing Paper Reviews That You Would Be Thankful to Receive". At the end, he had a slide on "Crazy ideas how to improve reviewing". RQC was not among them. A PhD candidate of mine asked him why he had not listed RQC and he said "That was a crazy idea a few years ago, but now it's becoming mainstream." An encouraging view! Please tell me if you know anybody else who may like to be on this mailing list [2]. Lutz [1] https://lists.fu-berlin.de/pipermail/rqc-interest/2015/msg00002.html [2] https://lists.fu-berlin.de/listinfo/rqc-interest (you can subscribe here) From prechelt@inf.fu-berlin.de Mon Nov 30 10:12:14 2015 Received: from outpost1.zedat.fu-berlin.de ([130.133.4.66]) by list1.zedat.fu-berlin.de (Exim 4.85) for rqc-interest@lists.fu-berlin.de with esmtp (envelope-from ) id <1a3KVJ-00394S-EM>; Mon, 30 Nov 2015 10:12:13 +0100 Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost.zedat.fu-berlin.de (Exim 4.85) for rqc-interest@lists.fu-berlin.de with esmtp (envelope-from ) id <1a3KVJ-001QeV-DJ>; Mon, 30 Nov 2015 10:12:13 +0100 Received: from kaduna.imp.fu-berlin.de ([160.45.118.73] helo=KADUNA) by inpost2.zedat.fu-berlin.de (Exim 4.85) for rqc-interest@lists.fu-berlin.de with esmtpsa (envelope-from ) id <1a3KVJ-002j1p-8J>; Mon, 30 Nov 2015 10:12:13 +0100 From: "Lutz Prechelt" To: Date: Mon, 30 Nov 2015 10:12:13 +0100 Message-ID: <03b901d12b4f$333b7380$99b25a80$@inf.fu-berlin.de> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 15.0 Thread-Index: AdErSxt+K4GzuuucQsqIpSAH3aKabw== Content-Language: de X-Originating-IP: 160.45.118.73 X-purgate: clean X-purgate-type: clean X-purgate-ID: 151147::1448874733-000A31C7-909AE937/0/0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.268950, version=1.2.4 X-Spam-Flag: NO X-Spam-Status: No, score=-50.0 required=5.0 tests=ALL_TRUSTED, T_RP_MATCHES_RCVD X-Spam-Checker-Version: SpamAssassin 3.4.1 on Tuvalu.ZEDAT.FU-Berlin.DE X-Spam-Level: Subject: [RQC-interest] Review Quality Collector (RQC) status update November 2015 X-BeenThere: rqc-interest@lists.fu-berlin.de X-Mailman-Version: 2.1.16 Precedence: list List-Id: sporadic information on Review Quality Collector List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Nov 2015 09:12:14 -0000 In the previous update [1], I shared an anecdote about RQC becoming mainstream. Today I would like to direct your attention to an upcoming article about RQC, RRP, and Publons: A writer for Science asked me for an interview, which we then did, and which he will use for a piece for Science Careers [2] about services that allow researchers to get credit for their peer reviews. (I am not sure when it will appear.) RRP, the Reviewer Recognition Platform [3], is the Elsevier project (that I also worked with) that counts your reviews for many journals and collects and presents status and badges on a private or public website. Publons [4] is a company from New Zealand that does roughly the same thing in a different manner and with optional publishing of reviews and post-publication reviews tacked on. Please tell me if you know anybody else who may like to be on this mailing list [5]. Lutz [1] https://lists.fu-berlin.de/pipermail/rqc-interest/2015/msg00003.html [2] http://sciencecareers.sciencemag.org/ [3] http://www.reviewerrecognition.elsevier.com/ [4] https://publons.com/ [5] https://lists.fu-berlin.de/listinfo/rqc-interest (you can subscribe here)