<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>how to survive peer review &#8211; Od Informacji do Wiedzy</title>
	<atom:link href="https://profind.pl/en/tag/how-to-survive-peer-review/feed/" rel="self" type="application/rss+xml" />
	<link>https://profind.pl</link>
	<description>Blog o informacjach na temat informacji i wiedzy</description>
	<lastBuildDate>Tue, 07 Oct 2025 07:57:01 +0000</lastBuildDate>
	<language>en-GB</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.4</generator>
	<item>
		<title>The Anatomy of a Scientific Review: Surviving the Ultimate Judgment in Computer Science and Medical Sciences</title>
		<link>https://profind.pl/en/2025/10/06/the-anatomy-of-a-scientific-review-surviving-the-ultimate-judgment-in-computer-science-and-medical-sciences/</link>
					<comments>https://profind.pl/en/2025/10/06/the-anatomy-of-a-scientific-review-surviving-the-ultimate-judgment-in-computer-science-and-medical-sciences/#respond</comments>
		
		<dc:creator><![CDATA[Marcin]]></dc:creator>
		<pubDate>Mon, 06 Oct 2025 12:15:47 +0000</pubDate>
				<category><![CDATA[Research diary]]></category>
		<category><![CDATA[AI in peer review]]></category>
		<category><![CDATA[how to survive peer review]]></category>
		<category><![CDATA[machine learning peer review]]></category>
		<category><![CDATA[medical sciences SEO]]></category>
		<category><![CDATA[peer review]]></category>
		<category><![CDATA[peer review in computer science]]></category>
		<category><![CDATA[peer review in computer science and medicine]]></category>
		<category><![CDATA[research ethics]]></category>
		<category><![CDATA[research reproducibility]]></category>
		<category><![CDATA[reviewer's checklist]]></category>
		<category><![CDATA[SEO for science]]></category>
		<guid isPermaLink="false">https://profind.pl/?p=1881</guid>

					<description><![CDATA[You hit “Submit” and then… silence. Months of work, hundreds of edits, one click—and the long wait for a verdict from those enigmatic figures: Reviewer 1, Reviewer 2… It’s one of the most stressful moments in a researcher’s life. But what actually happens on the other side? Peer review is the backbone of science—a quality-control [&#8230;]]]></description>
										<content:encoded><![CDATA[<div class="moj-wpis-naukowy">
<p>You hit “Submit” and then… silence. Months of work, hundreds of edits, one click—and the long wait for a verdict from those enigmatic figures: Reviewer 1, Reviewer 2… It’s one of the most stressful moments in a researcher’s life. But what actually happens on the other side? Peer review is the backbone of science—a quality-control system meant to ensure that published work is <strong>important, original, and rigorous</strong> (<a href="https://senseaboutscience.org/activities/peer-review/" target="_blank" rel="noopener">Sense about Science</a>).</p>
<p><span id="more-1881"></span></p>
<h3>The Reviewer’s Mind Map: What gets checked, and how?</h3>
<p>Across fields, reviewers look for answers to the same fundamental questions—but they use different tools to get there. The table below shows how the same principles of rigor are interpreted in the medical/social sciences versus in computer science and <em>machine learning</em>.</p>
<div class="table-wrapper">
<table>
<thead>
<tr style="background-color: #f2f2f2;">
<th style="padding: 12px; border: 1px solid #ddd; text-align: left;">Assessment Aspect / The Reviewer’s Question</th>
<th style="padding: 12px; border: 1px solid #ddd; text-align: left;">Implementation (Medical / Social Sciences)</th>
<th style="padding: 12px; border: 1px solid #ddd; text-align: left;">Implementation (Computer Science / Machine Learning)</th>
</tr>
</thead>
<tbody>
<tr>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>0. Conflicts of Interest and Expertise</strong></td>
<td style="padding: 8px; border: 1px solid #ddd;">Do I have ties to the authors/institution? Do I understand this clinical intervention and the statistical methodology?</td>
<td style="padding: 8px; border: 1px solid #ddd;">Do I have ties to the authors/lab? Is this my subfield (algorithms, model architecture, theory)?</td>
</tr>
<tr>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>1. Originality and Significance</strong></td>
<td style="padding: 8px; border: 1px solid #ddd;">Does the study address an important clinical/social problem? Does it add something genuinely new relative to existing studies and reviews?</td>
<td style="padding: 8px; border: 1px solid #ddd;">Is the problem significant (theoretically or practically)? Is the method truly novel, or merely incremental? <strong>Empirically:</strong> does it improve the <em>state of the art</em> in a meaningful way?</td>
</tr>
<tr>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>2. Methodological Rigor (Reproducibility)</strong></td>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>Protocol and preregistration:</strong> is the study registered (e.g., OSF)? Is the protocol available and followed?</td>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>Artifacts and code:</strong> are code, data, and the environment available (GitHub/Zenodo)? Is there a <code>README</code> to run the experiments? Do we meet <strong>ACM Artifact Badging</strong> criteria (Available/Evaluated/Results Validated)? <small>(Badges assess <em>artifacts and reproducibility</em>, not the scientific quality of the paper itself.)</small> <a href="https://www.acm.org/publications/policies/artifact-review-and-badging-current" target="_blank" rel="noopener">ACM policy</a> · <a href="https://sigir.org/general-information/acm-sigir-artifact-badging/" target="_blank" rel="noopener">SIGIR note</a></td>
</tr>
<tr>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>3. Methodological Rigor (Correctness)</strong></td>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>Reporting standards:</strong> is the appropriate <strong>EQUATOR Network</strong> checklist used—<a href="https://www.equator-network.org/reporting-guidelines/consort/" target="_blank" rel="noopener">CONSORT 2025</a> for RCTs, <a href="https://www.prisma-statement.org/prisma-2020" target="_blank" rel="noopener">PRISMA 2020</a> for systematic reviews, <a href="https://www.equator-network.org/reporting-guidelines/strobe/" target="_blank" rel="noopener">STROBE</a> for observational studies?</td>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>Technical craft:</strong> correctness of proofs and definitions; fair <em>apples-to-apples</em> comparisons; appropriate metrics; <em>ablation</em> to assess key components.</td>
</tr>
<tr>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>4. Results and Analysis</strong></td>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>Statistics:</strong> correct choice of tests; uncertainty reporting (CIs); multiple-testing corrections.</td>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>Experimental credibility:</strong> standard, public datasets; repeatability (different random seeds); valid significance tests for model comparisons.</td>
</tr>
<tr>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>5. Conclusions and Interpretation</strong></td>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>No overclaiming:</strong> conclusions match the data; no population-wide claims from small samples; a candid discussion of limitations.</td>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>Proportionality:</strong> no “we solve X” if the gain is +0.5%; discuss computational cost, scalability, and practical limits.</td>
</tr>
<tr>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>6. Research Ethics</strong></td>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>Participant protection (ICMJE/COPE):</strong> IRB approval, informed consent, data anonymization. <a href="https://www.icmje.org/recommendations/" target="_blank" rel="noopener">ICMJE</a> · <a href="https://publicationethics.org/core-practices" target="_blank" rel="noopener">COPE</a></td>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>Data and model ethics:</strong> licenses, privacy, bias, potential harms/dual-use; transparency about data/model sources.</td>
</tr>
</tbody>
</table>
</div>
<p style="font-size: .9em; font-style: italic; margin-top: 10px;">Simplification: in interdisciplinary work, reviewers combine both perspectives.</p>
<h3>A journey through your manuscript: three reviewer passes</h3>
<h4>Pass 1: Quick triage (15–30 min) – <em>is this worth reading further?</em></h4>
<ul>
<li><strong>Fit:</strong> is the work original and within the venue’s scope?</li>
<li><strong>Logic:</strong> do conclusions broadly follow from the data?</li>
<li><strong>Red flags:</strong>
<ul>
<li><em>CS/ML:</em> a “new” algorithm is a known method under a new name; non-standard, trivial dataset.</li>
<li><em>Medicine:</em> RCT without IRB approval or without a control group.</li>
<li><em>Universal:</em> conclusions contradict the authors’ own results.</li>
</ul>
</li>
</ul>
<h4>Pass 2: Microscope (2–5 h) – <em>can I trust these results?</em></h4>
<ul>
<li>Med/Soc: point-by-point compliance with <strong>CONSORT 2025</strong>/<strong>PRISMA</strong>/<strong>STROBE</strong>.</li>
<li>CS/ML: review repository and artifacts; fair baselines; correctness of claims.</li>
</ul>
<h4>Pass 3: Narrative (1–2 h) – <em>does the story make sense?</em></h4>
<ul>
<li>Do the promises in the introduction materialize later?</li>
<li>Is there an honest <em>Limitations</em> section?</li>
<li>Are figures/tables readable and properly described?</li>
</ul>
<h3>Thinking like a reviewer: pre-submission self-review checklist</h3>
<p>Before you click “Submit,” do a tough, point-by-point self-review.</p>
<h4>Big picture and contribution</h4>
<ul>
<li><strong>Problem:</strong> do I clearly define what I solve and why it matters?</li>
<li><strong>Originality:</strong> where in the text do I demonstrate novelty vs. prior work?</li>
<li><strong>Impact:</strong> will anyone want to cite this in 2–3 years?</li>
</ul>
<h4>Section by section</h4>
<p><strong>Title:</strong> does it accurately reflect the content (neither too broad nor too narrow)?</p>
<p><strong>Abstract:</strong></p>
<ul>
<li><strong>WHAT?</strong> question/goal; <strong>WHY?</strong> significance; <strong>HOW?</strong> method; <strong>RESULTS:</strong> key numbers; <strong>NOVELTY:</strong> one-sentence summary of what’s new.</li>
</ul>
<p><strong>Introduction:</strong> context → gap → goal → novelety → solution; end with clear <em>contributions</em>.</p>
<p><strong>Methods/Approach:</strong></p>
<ul>
<li><strong>Reproducibility:</strong> could a peer in the field replicate the experiment?</li>
<li><strong>Justification:</strong> why these methodological choices?</li>
<li><strong>Details:</strong> (CS/ML) hyperparameters, architecture, datasets, metrics; (Med/Soc) inclusion/exclusion criteria, randomization, sample size.</li>
<li><strong>Transparency:</strong> are code and data available? If not, provide a solid rationale.</li>
</ul>
<p><strong>Results &amp; Discussion:</strong> clear figures; strong baselines; claims supported by data; a separate paragraph on limitations.</p>
<p><strong>Conclusions:</strong> answer the research question + offer broader perspective and future directions.</p>
<hr />
<div style="padding: 12px; border-left: 4px solid #444; background: #fafafa; margin: 24px 0;">
<p><strong>Box: standards and transparency in peer review</strong><br />
Point readers to <strong>ANSI/NISO Z39.106-2023</strong>—a standard terminology for peer review models (single/double anonymized, openness of reports, etc.). <a href="https://www.niso.org/publications/z39106-2023-peerreview" target="_blank" rel="noopener">NISO</a></p>
</div>
<h3>AI in service of peer review: assistant or usurper?</h3>
<p>Large language models (LLMs) have entered our toolbox. Surveys (e.g., <a href="https://www.sciencedirect.com/science/article/abs/pii/S1566253525004051" target="_blank" rel="noopener">Zhuang et al., 2025</a>) outline both potential and risk. Below is a policy-safe minimal practice set.</p>
<h4>What genuinely helps?</h4>
<ul>
<li><strong>Fast triage and summaries:</strong> initial mapping of content, scoping against the venue.</li>
<li><strong>Formal checklists:</strong> matching study type to the right standard (e.g., RCT → CONSORT 2025).</li>
<li><strong>Sanity-checking consistency:</strong> basic mismatches between text and tables/figures.</li>
</ul>
<h4>Where are the boundaries?</h4>
<ul>
<li><strong>No deep expertise:</strong> models can miss the true contribution or deep flaws.</li>
<li><strong>Hallucinations and phantom citations:</strong> risk of fabricated sources/claims.</li>
<li><strong>Bias and score inflation:</strong> studies report a tendency to over-recommend.</li>
</ul>
<h4>The ugly truth: ethics, confidentiality, and publisher policies</h4>
<p><strong>Manuscript confidentiality is sacred.</strong> Uploading an unpublished paper to a public LLM breaches review confidentiality.</p>
<ul>
<li><strong>Elsevier:</strong> reviewers <em>should not</em> upload manuscripts to genAI tools or use genAI for substantive reviewing. <a href="https://www.elsevier.com/about/policies-and-standards/the-use-of-generative-ai-and-ai-assisted-technologies-in-the-review-process" target="_blank" rel="noopener">Policy</a></li>
<li><strong>Springer Nature:</strong> asks reviewers to <em>not upload</em> manuscripts to genAI; confidentiality first. <a href="https://www.springer.com/gp/editorial-policies/peer-review-policy-process" target="_blank" rel="noopener">Policy</a></li>
<li><strong>IEEE:</strong> <em>prohibits</em> using AI to draft or generate reviews. <a href="https://ieeeauthorcenter.ieee.org/wp-content/uploads/ieee-reviewer-guidelines.pdf" target="_blank" rel="noopener">Reviewer Guidelines</a></li>
<li><strong>ACM:</strong> only permits language assistance on <em>your own review text</em>, and only without breaching confidentiality (prefer privacy-preserving enterprise solutions). <a href="https://www.acm.org/publications/policies/peer-review-faq" target="_blank" rel="noopener">ACM Peer Review FAQ</a></li>
<li><strong>ACL/ARR:</strong> <em>do not</em> share confidential content with third-party tools (practically: <em>do not</em> use popular LLMs for review materials). <a href="https://aclrollingreview.org/reviewerguidelines" target="_blank" rel="noopener">ARR Reviewer Guidelines</a></li>
</ul>
<p><strong>Exception: publisher-provided tools.</strong> Some publishers develop <em>closed</em> internal tools to support integrity and completeness checks (e.g., Frontiers—<a href="https://www.frontiersin.org/news/2020/07/01/artificial-intelligence-peer-review-assistant-aira/" target="_blank" rel="noopener">AIRA</a>). These handle pre-screening and technical checks (plagiarism, figures, language); the <strong>final judgment remains human</strong>. Always verify the current policy for your journal/conference.</p>
<div style="padding: 12px; border-left: 4px solid #f39c12; background: #fffaf3; margin: 24px 0;">
<p><strong>Minimum safety rule:</strong> <em>never</em> upload someone else’s manuscript to a public LLM. If you need language help for <em>your own</em> review text, use privacy-preserving solutions or the publisher’s in-house tools.</p>
</div>
<h3>The art of responding to reviews</h3>
<ul>
<li><strong>Professional tone:</strong> thank the reviewers and the editor.</li>
<li><strong>Point-by-point list:</strong> address <em>every</em> comment separately.</li>
<li><strong>Show the changes:</strong> cite exact locations in the manuscript (page/line/section).</li>
<li><strong>When you disagree:</strong> be polite, provide evidence-based arguments.</li>
</ul>
<hr />
<h3>References and further reading</h3>
<h4>Standards and guidelines</h4>
<ul>
<li><a href="https://www.equator-network.org/reporting-guidelines/consort/" target="_blank" rel="noopener">CONSORT 2025</a> (RCTs)—update with parallel publications in BMJ/JAMA/Lancet/Nat Med/PLOS Med.</li>
<li><a href="https://www.prisma-statement.org/prisma-2020" target="_blank" rel="noopener">PRISMA 2020</a> (systematic reviews) · <a href="https://www.equator-network.org/reporting-guidelines/strobe/" target="_blank" rel="noopener">STROBE</a> (observational studies)</li>
<li><a href="https://publicationethics.org/core-practices" target="_blank" rel="noopener">COPE Core Practices</a> · <a href="https://www.icmje.org/recommendations/" target="_blank" rel="noopener">ICMJE Recommendations</a></li>
<li><a href="https://www.niso.org/publications/z39106-2023-peerreview" target="_blank" rel="noopener">ANSI/NISO Z39.106-2023</a>—peer review terminology standard</li>
</ul>
<h4>Reproducibility and artifacts</h4>
<ul>
<li><a href="https://www.acm.org/publications/policies/artifact-review-and-badging-current" target="_blank" rel="noopener">ACM Artifact Review &amp; Badging</a> · <a href="https://sigir.org/general-information/acm-sigir-artifact-badging/" target="_blank" rel="noopener">SIGIR: what badges mean (not a scientific-quality rating)</a></li>
</ul>
<h4>AI policies (selected)</h4>
<ul>
<li><a href="https://www.elsevier.com/about/policies-and-standards/the-use-of-generative-ai-and-ai-assisted-technologies-in-the-review-process" target="_blank" rel="noopener">Elsevier—AI and peer review</a> · <a href="https://www.springer.com/gp/editorial-policies/peer-review-policy-process" target="_blank" rel="noopener">Springer Nature—confidentiality &amp; AI</a> · <a href="https://ieeeauthorcenter.ieee.org/wp-content/uploads/ieee-reviewer-guidelines.pdf" target="_blank" rel="noopener">IEEE—prohibition on AI-written reviews</a></li>
<li><a href="https://www.acm.org/publications/policies/peer-review-faq" target="_blank" rel="noopener">ACM—Peer Review FAQ</a> · <a href="https://aclrollingreview.org/reviewerguidelines" target="_blank" rel="noopener">ACL/ARR—third-party tool restrictions</a></li>
<li><a href="https://www.frontiersin.org/news/2020/07/01/artificial-intelligence-peer-review-assistant-aira/" target="_blank" rel="noopener">Frontiers AIRA</a>—closed, integrity-supporting pre-screening</li>
</ul>
<h4>Introductions and guides</h4>
<ul>
<li><a href="https://senseaboutscience.org/activities/peer-review-the-nuts-and-bolts-2/" target="_blank" rel="noopener">Peer Review: The Nuts and Bolts</a> (Sense about Science)</li>
<li><a href="https://profind.pl/2025/09/20/anatomia-procesu-badawczego-praktyczny-framework-dla-rzetelnej-nauki/" target="_blank" rel="noopener">Anatomy of the research process: a practical framework for rigorous science</a> (your previous post)</li>
</ul>
<h4>LLMs in peer review—surveys and studies</h4>
<ul>
<li>Zhuang, Z. et al. (2025). <a href="https://www.sciencedirect.com/science/article/abs/pii/S1566253525004051" target="_blank" rel="noopener">Large language models for automated scholarly paper review: A survey</a>, <em>Information Fusion</em>.</li>
</ul>
</div>
]]></content:encoded>
					
					<wfw:commentRss>https://profind.pl/en/2025/10/06/the-anatomy-of-a-scientific-review-surviving-the-ultimate-judgment-in-computer-science-and-medical-sciences/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
	</channel>
</rss>
