<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Od Informacji do Wiedzy</title>
	<atom:link href="https://profind.pl/en/feed/" rel="self" type="application/rss+xml" />
	<link>https://profind.pl</link>
	<description>Blog o informacjach na temat informacji i wiedzy</description>
	<lastBuildDate>Fri, 27 Feb 2026 19:17:28 +0000</lastBuildDate>
	<language>en-GB</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.4</generator>
	<item>
		<title>Document Classification Pattern Recognition via Information Fusion: A systematic review of multimodal and multiview representation approaches</title>
		<link>https://profind.pl/en/2026/02/27/document-classification-pattern-recognition-via-information-fusion-a-systematic-review-of-multimodal-and-multiview-representation-approaches-2/</link>
					<comments>https://profind.pl/en/2026/02/27/document-classification-pattern-recognition-via-information-fusion-a-systematic-review-of-multimodal-and-multiview-representation-approaches-2/#respond</comments>
		
		<dc:creator><![CDATA[Marcin]]></dc:creator>
		<pubDate>Fri, 27 Feb 2026 18:58:07 +0000</pubDate>
				<category><![CDATA[Articles]]></category>
		<category><![CDATA[Document classification]]></category>
		<category><![CDATA[information fusion]]></category>
		<category><![CDATA[meta-analysis]]></category>
		<category><![CDATA[Representation learning]]></category>
		<category><![CDATA[systematic review]]></category>
		<guid isPermaLink="false">https://profind.pl/?p=2016</guid>

					<description><![CDATA[Abstract Information fusion is used widely to improve document classification by integrating multiple data sources (multimodal) or multiple representations of the same data (multiview). Yet the literature has been fragmented: there has been no unified framework, no quantitative synthesis of “how much fusion helps,” and limited practitioner-oriented guidance. In our systematic review we analyse 139 [&#8230;]]]></description>
										<content:encoded><![CDATA[
<h2 class="wp-block-heading">Abstract</h2>



<p class="wp-block-paragraph" style="text-align: justify;">Information fusion is used widely to improve document classification by integrating multiple data sources (<strong>multimodal</strong>) or multiple representations of the same data (<strong>multiview</strong>). Yet the literature has been fragmented: there has been no unified framework, no quantitative synthesis of “how much fusion helps,” and limited practitioner-oriented guidance. In our systematic review we analyse <strong>139 primary studies</strong>, propose a formal framework to structure the field, summarise key qualitative trends, and perform a <strong>random-effects meta-analysis</strong> (to our knowledge, the first focused specifically on document classification). The results show that <strong>multimodal fusion</strong> significantly improves accuracy (mean gain <strong>+5.28 percentage points</strong>, <strong>p=0.0016</strong>), while <strong>multiview fusion</strong> yields consistent but modest improvements for accuracy (<strong>+4.67%</strong>), F1-score (<strong>+3.08%</strong>) and recall (all <strong>p&lt;0.05</strong>). We also highlight a reproducibility gap: only <strong>11.8%</strong> (multimodal) and <strong>23.3%</strong> (multiview) of studies report statistical tests. Overall, the key lesson is practical: success depends less on algorithmic complexity and more on <strong>aligning the fusion strategy with the task context</strong> and committing to <strong>rigorous validation</strong>.</p>



<p>]]></content:encoded>
					
					<wfw:commentRss>https://profind.pl/en/2026/02/27/document-classification-pattern-recognition-via-information-fusion-a-systematic-review-of-multimodal-and-multiview-representation-approaches-2/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>From Experiment to Theory: How to Correctly Evaluate ML Model Performance</title>
		<link>https://profind.pl/en/2025/12/12/from-experiment-to-theory-how-to-correctly-evaluate-ml-model-performance/</link>
					<comments>https://profind.pl/en/2025/12/12/from-experiment-to-theory-how-to-correctly-evaluate-ml-model-performance/#respond</comments>
		
		<dc:creator><![CDATA[Marcin]]></dc:creator>
		<pubDate>Fri, 12 Dec 2025 17:10:05 +0000</pubDate>
				<category><![CDATA[Research diary]]></category>
		<category><![CDATA[cross validation]]></category>
		<category><![CDATA[data science]]></category>
		<category><![CDATA[generalization]]></category>
		<category><![CDATA[machine learning]]></category>
		<category><![CDATA[model evaluation]]></category>
		<category><![CDATA[model selection]]></category>
		<guid isPermaLink="false">https://profind.pl/?p=1983</guid>

					<description><![CDATA[In a typical machine learning experiment, we possess data that (in the simplest variant) is split into three disjoint sets: training, validation, and test. The training set allows for the estimation of model parameters (e.g., neural network weights, regression coefficients, distribution parameters in Naive Bayes, etc.) by minimizing a chosen loss function or maximizing likelihood. The goal [&#8230;]]]></description>
										<content:encoded><![CDATA[<p class="mb-4 break-words" style="text-align: justify;">In a typical machine learning experiment, we possess data that (in the simplest variant) is split into three disjoint sets: training, validation, and test. The <strong class="font-bold">training set</strong> allows for the estimation of model parameters (e.g., neural network weights, regression coefficients, distribution parameters in Naive Bayes, etc.) by minimizing a chosen loss function or maximizing likelihood. The goal of learning is not the &#8220;perfect mapping&#8221; of training data per se, but achieving good generalization—i.e., low error on new, unseen examples (assuming they originate from the same distribution as the training data).</p>
<p class="mb-4 break-words" style="text-align: justify;">In practice, during training, the model may start to overfit: the error on the training set continues to decrease, but performance on unseen data stops improving or even deteriorates. Therefore, we introduce a <strong class="font-bold">validation set</strong>, which is not used to fit parameters in a given training run, but rather for model selection and setting the learning process configuration (hyperparameters), as well as for procedures like <strong class="font-bold">early stopping</strong> (halting training when the loss or metric on the validation set stops improving).</p>
<p class="mb-4 break-words" style="text-align: justify;">Selecting the &#8220;best&#8221; model is usually done by minimizing loss or maximizing a chosen metric (Accuracy, F1, AUC, etc.) on the validation set. However, one must remember that intensive tuning (numerous trials of architectures, hyperparameters, random seeds, augmentations, preprocessing decisions) can lead to &#8220;overfitting to validation&#8221; and an overly optimistic evaluation. This is a form of <strong class="font-bold">selection bias</strong> in model selection.</p>
<p class="mb-4 break-words" style="text-align: justify;">The <strong class="font-bold">test set</strong> should be kept &#8220;for the end&#8221; and used only after the entire design process (model selection and hyperparameter tuning) is complete. The result on the test set serves as an estimation of the generalization error for the adopted protocol, but it is still an estimation subject to uncertainty (dependent, among other factors, on the test size and problem variance).</p>
<p class="mb-4 break-words" style="text-align: justify;">When data is scarce, or we require a more stable evaluation, resampling procedures are employed. Classically, <strong class="font-bold"><span class="katex"><span class="katex-html" aria-hidden="true"><span class="base"><span class="mord mathnormal">k</span></span></span></span>-fold cross-validation</strong> (including leave-one-out as an extreme case) and bootstrap methods are used for prediction error estimation. Bootstrap (e.g., the <span class="katex"><span class="katex-html" aria-hidden="true"><span class="base"><span class="mord">.632</span><span class="mord">+</span></span></span></span> variant) is sometimes treated as a &#8220;smoothed&#8221; alternative to CV with a different bias–variance trade-off. If cross-validation serves both for hyperparameter tuning and for quality assessment, a <strong class="font-bold">nested cross-validation</strong> procedure is necessary to avoid optimistic bias. In this approach, the inner loop tunes hyperparameters, while the outer loop estimates the generalization error.</p>
<p class="mb-4 break-words" style="text-align: justify;">Empirical research consists of designing models and learning procedures and comparing them within a controlled protocol. The theoretical current is complementary: it provides formal frameworks where (under certain assumptions) one can obtain guarantees such as &#8220;with high probability, the true risk will not exceed a certain value,&#8221; dependent on the empirical risk and a measure of solution complexity. In particular, <strong class="font-bold">PAC-Bayes</strong> theory provides probabilistic upper bounds on the generalization error (often for stochastic/Gibbs classifiers) as a function of empirical error and a complexity term expressed, among others, by the Kullback-Leibler divergence (<span class="katex"><span class="katex-html" aria-hidden="true"><span class="base"><span class="mord"><span class="mord mathnormal">D</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist"><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">K</span><span class="mord mathnormal mtight">L</span></span></span></span><span class="vlist-s">​</span></span></span></span></span></span></span></span>) between the posterior and prior distributions. This does not imply a simple deterministic relationship like <span class="katex"><span class="katex-html" aria-hidden="true"><span class="base"><span class="mord"><span class="mord mathnormal">E</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist"><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">v</span><span class="mord mathnormal mtight">a</span><span class="mord mathnormal mtight">l</span></span></span></span><span class="vlist-s">​</span></span></span></span></span><span class="mrel">≤</span></span><span class="base"><span class="mord"><span class="mord mathnormal">E</span><span class="msupsub"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist"><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">t</span><span class="mord mathnormal mtight">es</span><span class="mord mathnormal mtight">t</span></span></span></span><span class="vlist-s">​</span></span></span></span></span></span></span></span>; the relationship between validation and test errors depends on the selection protocol and can be biased by the model selection process.</p>
]]></content:encoded>
					
					<wfw:commentRss>https://profind.pl/en/2025/12/12/from-experiment-to-theory-how-to-correctly-evaluate-ml-model-performance/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>5 Surprising Truths About Prompts That Will Change How You Talk to AI</title>
		<link>https://profind.pl/en/2025/11/05/1934/</link>
					<comments>https://profind.pl/en/2025/11/05/1934/#respond</comments>
		
		<dc:creator><![CDATA[Marcin]]></dc:creator>
		<pubDate>Wed, 05 Nov 2025 12:13:03 +0000</pubDate>
				<category><![CDATA[Research diary]]></category>
		<category><![CDATA[best-practices]]></category>
		<category><![CDATA[chain-of-thought]]></category>
		<category><![CDATA[CoT]]></category>
		<category><![CDATA[fine-tuning]]></category>
		<category><![CDATA[Gemma-2]]></category>
		<category><![CDATA[GPT-4o]]></category>
		<category><![CDATA[GSM8K]]></category>
		<category><![CDATA[in-context-learning]]></category>
		<category><![CDATA[LLM-performance]]></category>
		<category><![CDATA[MMLU]]></category>
		<category><![CDATA[PaLM-540B]]></category>
		<category><![CDATA[paraphrase-sensitivity]]></category>
		<category><![CDATA[politeness]]></category>
		<category><![CDATA[prefix-tuning]]></category>
		<category><![CDATA[Prompt Engineering]]></category>
		<category><![CDATA[prompt-structure]]></category>
		<category><![CDATA[prompt-tone]]></category>
		<category><![CDATA[robustness]]></category>
		<category><![CDATA[SCULPT]]></category>
		<category><![CDATA[semantic-stability]]></category>
		<category><![CDATA[step-back-prompting]]></category>
		<category><![CDATA[TimeQA]]></category>
		<category><![CDATA[token-costs]]></category>
		<guid isPermaLink="false">https://profind.pl/?p=1934</guid>

					<description><![CDATA[Interacting with AI via prompts—i.e., text instructions—seems simple, but it hides plenty of surprises. In recent years (2023–2025), researchers have intensively analyzed how the form and style of our queries affect the responses of large language models (LLMs). It turns out that some popular beliefs about the “art of asking questions” need revisiting. Below are [&#8230;]]]></description>
										<content:encoded><![CDATA[


<p class="wp-block-paragraph" style="text-align: justify;">Interacting with AI via <em>prompts</em>—i.e., text instructions—seems simple, but it hides plenty of surprises. In recent years (2023–2025), researchers have intensively analyzed how the form and style of our queries affect the responses of large language models (LLMs). It turns out that some popular beliefs about the “art of asking questions” need revisiting. Below are five surprising, research-backed truths about prompts. These findings give AI conversations a more scientific edge—and might change how you formulate instructions.</p>

<p><span id="more-1934"></span></p>

<h2 class="wp-block-heading">1. Simple vs. complex prompts—more isn’t always better</h2>



<p class="wp-block-paragraph" style="text-align: justify;">Intuitively, one might think that the more detailed and elaborate the prompt, the better the answer. Reality is more nuanced. Early experiments showed that adding an instruction like “Think step by step” can <strong>trigger a model’s reasoning process</strong> and improve performance on logic or math problems—this is <em>Chain-of-Thought</em> (CoT) prompting. As early as 2022, it was shown that a few worked examples prompted step by step enabled <strong>PaLM 540B</strong> to achieve <strong>state-of-the-art on GSM8K</strong>, even surpassing a fine-tuned GPT-3 with a verifier [1]. This illustrates the potential of complex prompts: <em>a well-crafted instruction can “unlock” latent reasoning abilities</em>.</p>
<p style="text-align: justify;">More recent studies suggest tempering universal enthusiasm for complex prompts. <strong>More complicated prompts don’t always yield better results.</strong> A 2025 report (Wharton GAIL) indicates that for newer, “reasoning” LLMs, the gains from explicitly forcing reasoning can be <em>marginal</em>, while time/token costs rise; effects vary by model and task [2]. In other words, overly intricate prompts can introduce unnecessary steps and mislead the model—while larger models may already <em>plan internally</em> without extra instructions.</p>
<p style="text-align: justify;">Additionally, in one experimental survey <strong>Gemma 2 9B</strong> maintained similar effectiveness whether the prompt was simple or multitask—suggesting stability with respect to prompt type in certain tasks [10]. <strong>The surprising truth:</strong> the simplest approach is often as effective as elaborate prose. Clarity is key—if the task doesn’t require a complex structure, a simple, precise prompt may be best.</p>


<hr class="wp-block-separator has-alpha-channel-opacity" />


<h2 class="wp-block-heading">2. Prompt tone and politeness matter—but not always as you expect</h2>



<p class="wp-block-paragraph" style="text-align: justify;">You might expect politeness to aid collaboration with AI. <strong>Recent findings are mixed.</strong> A 2025 study on GPT-4o reports a small advantage for terse, “brusque” instructions over very polite ones (e.g., ~80.8% vs. ~84.8% accuracy on multiple-choice questions) [4]. <em>However,</em> this is preliminary, on a single model and a modest question set. Moreover, an earlier cross-lingual study (2024) showed that <strong>impolite prompts often <em>degrade</em> performance</strong>, with the optimum depending on language and model [3].</p>
<p style="text-align: justify;">Practical takeaway: <strong>experiment with form</strong> (terse/direct vs. polite/expanded), but don’t generalize that “being impolite” always helps. Changing tone can nudge the model toward more concise answers, yet it can also harm quality—especially across languages.</p>


<hr class="wp-block-separator has-alpha-channel-opacity" />


<h2 class="wp-block-heading">3. Structure matters: long prompts benefit from being split and “sculpted”</h2>



<p class="wp-block-paragraph" style="text-align: justify;">The more complex the task, the easier it is to produce a “wall of text.” <strong>Structuring the instruction</strong> (sections, steps, plan → execution) usually helps the model. A good example is <strong>SCULPT</strong>—a method that treats a long prompt as a tree and refines it iteratively (Critic + Actor), improving performance and <strong>robustness to small perturbations</strong>. SCULPT was presented at <strong>ACL 2025</strong> [5].</p>
<p style="text-align: justify;">In practice, <strong>Step-Back Prompting</strong> also works well—first ask the model to identify the problem type and governing principles/plan (abstraction), then generate the solution. In the authors’ tests, the technique improved results by <strong>+7 pp</strong> on MMLU (Physics), <strong>+11 pp</strong> on MMLU (Chemistry), and <strong>+27 pp</strong> on TimeQA (PaLM-2L) [7].</p>


<hr class="wp-block-separator has-alpha-channel-opacity" />


<h2 class="wp-block-heading">4. Fine-tuning—teach the model your style instead of repeating it in prompts</h2>



<p class="wp-block-paragraph" style="text-align: justify;">Hand-tuning prompts has limits. If you need <strong>consistent style/format across an organization</strong>, consider <em>fine-tuning</em>—updating model weights on your data. In August 2023, OpenAI stated that a fine-tuned GPT-3.5-Turbo can, on narrow tasks, <em>match base GPT-4</em> (a vendor claim; always verify locally) [8]. It’s useful to distinguish <em>fine-tuning</em> from <em>prefix/prompt-tuning</em>, which add “virtual” prefix tokens without changing weights (see Li &amp; Liang, 2021) [9].</p>


<hr class="wp-block-separator has-alpha-channel-opacity" />


<h2 class="wp-block-heading">5. Small change, big effect: models are sensitive to prompt nuances</h2>



<p class="wp-block-paragraph" style="text-align: justify;">Two semantically equivalent questions can yield dramatically different results. On the <strong>RobustAlpacaEval</strong> benchmark, spreads of <strong>~45 percentage points</strong> between the best and worst paraphrase for the same model were observed; in extreme cases, the <em>worst</em> variant of Llama-2-70B-chat dropped to <strong>9.38%</strong> [6]. New techniques (e.g., instruction <em>self-denoising</em>) improve robustness to typos and small perturbations [11], but the core issue—semantic instability—remains. <strong>Practice:</strong> favor clarity and test multiple prompt variants.</p>


<hr class="wp-block-separator has-alpha-channel-opacity" />


<h2 class="wp-block-heading">Conclusion</h2>



<p class="wp-block-paragraph" style="text-align: justify;">The era of <em>prompt engineering</em> is evolving rapidly. We’ve seen cases where simplicity beats over-engineering, and cases where an unusual factor (like tone) shifts outcomes. We’ve learned that prompt structure can be optimized much like code, and when that’s not enough—you can change the model itself via fine-tuning. All this leads to one takeaway: <strong>talking to AI is a skill that blends creativity with an understanding of how the model works</strong>. With the above truths in mind, you can better extract what you need—be it precise answers or a preferred style.</p>



<h3 class="wp-block-heading">Practical tips (at a glance)</h3>



<ul class="wp-block-list">
<li style="text-align: justify;"><strong>Start with a simple prompt</strong>; add structure (steps, sections, examples) only if results are weak.</li>
<li style="text-align: justify;"><strong>Test tone and style</strong> (terse/direct vs. polite/expanded)—effects depend on model and language.</li>
<li style="text-align: justify;"><strong>Decompose complex tasks</strong> into steps (plan → execution) and split long context into sections; consider SCULPT/Step-Back.</li>
<li style="text-align: justify;"><strong>Standardize output formats</strong> (e.g., “Return JSON with fields …”)—this improves repeatability.</li>
<li style="text-align: justify;"><strong>Measure cost/tokens</strong>: CoT and plan-and-solve increase latency and cost—use them where they truly improve accuracy.</li>
<li style="text-align: justify;"><strong>If you need consistent style</strong> at organizational scale, consider fine-tuning/prefix-tuning rather than long “super-prompts.”</li>
</ul>



<h2 class="wp-block-heading">References / Bibliography</h2>



<ol class="wp-block-list">
<li style="text-align: justify;">Wei, J. et al. (2022), <em>Chain-of-Thought Prompting Elicits Reasoning in Large Language Models</em>. (PaLM 540B; SOTA on GSM8K). <a href="https://arxiv.org/abs/2201.11903" target="_blank" rel="noopener">arXiv:2201.11903</a>.</li>
<li style="text-align: justify;">Meincke, L., Mollick, E., Mollick, L., Shapiro, D. (2025), <em>Prompting Science Report 2: The Decreasing Value of Chain-of-Thought in Prompting</em>. (Diminishing CoT gains; time/token cost). <a href="https://gail.wharton.upenn.edu/research-and-insights/tech-report-chain-of-thought/" target="_blank" rel="noopener">Wharton GAIL</a>, <a href="https://arxiv.org/abs/2506.07142" target="_blank" rel="noopener">arXiv:2506.07142</a>.</li>
<li style="text-align: justify;">Yin, Z. et al. (2024), <em>Should We Respect LLMs? A Cross-Lingual Study on the Influence of Prompt Politeness on LLM Performance</em>. (Politeness: effects depend on language/model). <a href="https://aclanthology.org/2024.sicon-1.2/" target="_blank" rel="noopener">ACL Anthology</a>, <a href="https://arxiv.org/abs/2402.14531" target="_blank" rel="noopener">arXiv:2402.14531</a>.</li>
<li style="text-align: justify;">Dobariya, O., Kumar, A. (2025), <em>Mind Your Tone: Investigating How Prompt Politeness Affects LLM Accuracy</em>. (GPT-4o: ~80.8% vs. ~84.8%; preliminary, one model). <a href="https://arxiv.org/abs/2510.04950" target="_blank" rel="noopener">arXiv:2510.04950</a>.</li>
<li style="text-align: justify;">Kumar, S. et al. (2024/2025), <em>SCULPT: Systematic Tuning of Long Prompts</em>. (ACL 2025; robustness to perturbations). <a href="https://aclanthology.org/2025.acl-long.730/" target="_blank" rel="noopener">ACL 2025</a>, <a href="https://arxiv.org/abs/2410.20788" target="_blank" rel="noopener">arXiv:2410.20788</a>.</li>
<li style="text-align: justify;">Cao, B. et al. (2024), <em>On the Worst Prompt Performance of Large Language Models</em>. (RobustAlpacaEval; spreads up to ~45 pp; minima ~9.38%). <a href="https://arxiv.org/abs/2406.10248" target="_blank" rel="noopener">arXiv:2406.10248</a>, <a href="https://proceedings.neurips.cc/paper_files/paper/2024/file/7fa5a377b7ffabcce43cd00231bb3f9c-Paper-Conference.pdf" target="_blank" rel="noopener">NeurIPS 2024 (PDF)</a>.</li>
<li style="text-align: justify;">Zheng, H.S. et al. (2023), <em>Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models</em>. (Step-Back: +7/+11 pp MMLU; +27 pp TimeQA on PaLM-2L). <a href="https://arxiv.org/abs/2310.06117" target="_blank" rel="noopener">arXiv:2310.06117</a>.</li>
<li style="text-align: justify;">OpenAI (2023), <em>GPT-3.5-Turbo fine-tuning and API updates</em>. (Claim: fine-tuned 3.5 can match base GPT-4 on narrow tasks). <a href="https://openai.com/index/gpt-3-5-turbo-fine-tuning-and-api-updates/" target="_blank" rel="noopener">openai.com</a>.</li>
<li style="text-align: justify;">Li, X.L., Liang, P. (2021), <em>Prefix-Tuning: Optimizing Continuous Prompts for Generation</em>. (“Virtual” prefixes; no weight updates). <a href="https://arxiv.org/abs/2101.00190" target="_blank" rel="noopener">arXiv:2101.00190</a>, <a href="https://aclanthology.org/2021.acl-long.353/" target="_blank" rel="noopener">ACL 2021</a>.</li>
<li style="text-align: justify;">Gozzi, M., Di Maio, F. (2024), <em>Comparative Analysis of Prompt Strategies for Large Language Models: Single-Task vs. Multitask Prompts</em>. (Gemma 2 9B: small differences across prompt types). <a href="https://www.mdpi.com/2079-9292/13/23/4712" target="_blank" rel="noopener">Electronics 13(23):4712</a>.</li>
<li style="text-align: justify;">Agrawal, R. et al. (2025), <em>Enhancing LLM Robustness to Perturbed Instructions</em>. (Iterative instruction <em>self-denoising</em>). <a href="https://www.doc.ic.ac.uk/~la1520/data/Enhancing_LLM_Robustness_to_Perturbed_Instructions.pdf" target="_blank" rel="noopener">PDF</a>.</li>
</ol>



<p class="wp-block-paragraph" style="text-align: justify;"><em>Note: effects depend on the model, language, and benchmark; test prompt variants locally on your own tasks.</em></p>]]></content:encoded>
					
					<wfw:commentRss>https://profind.pl/en/2025/11/05/1934/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>Taming Long Context in LLMs: 3 Problems, 1 Cohesive Set of Strategies</title>
		<link>https://profind.pl/en/2025/11/04/taming-long-context-in-llms-3-problems-1-cohesive-set-of-strategies/</link>
					<comments>https://profind.pl/en/2025/11/04/taming-long-context-in-llms-3-problems-1-cohesive-set-of-strategies/#respond</comments>
		
		<dc:creator><![CDATA[Marcin]]></dc:creator>
		<pubDate>Tue, 04 Nov 2025 19:13:28 +0000</pubDate>
				<category><![CDATA[Research diary]]></category>
		<category><![CDATA[context compression]]></category>
		<category><![CDATA[cross-encoder reranking]]></category>
		<category><![CDATA[LLMLingua / LongLLMLingua]]></category>
		<category><![CDATA[long context (LLM)]]></category>
		<category><![CDATA[LongRoPE]]></category>
		<category><![CDATA[Lost in the Middle]]></category>
		<category><![CDATA[Ms-PoE (Found in the Middle)]]></category>
		<category><![CDATA[RAG]]></category>
		<category><![CDATA[RULER / OneRULER evaluation]]></category>
		<category><![CDATA[streaming attention (attention sinks)]]></category>
		<guid isPermaLink="false">https://profind.pl/?p=1921</guid>

					<description><![CDATA[In practical LLM systems (QA, analytics, assistants, agentic RAG), three phenomena routinely degrade quality: (1) Lost in the Middle — accuracy drops when the key evidence sits in the middle of a long prompt; (2) Distracting prompts — a few “tempting” sentences derail reasoning; (3) Very large contexts → performance drop — despite advertised 32k+ [&#8230;]]]></description>
										<content:encoded><![CDATA[<article>
<p style="text-align: justify;">In practical LLM systems (QA, analytics, assistants, agentic RAG), three phenomena routinely degrade quality: (1) <em>Lost in the Middle</em> — accuracy drops when the key evidence sits in the middle of a long prompt; (2) <em>Distracting prompts</em> — a few “tempting” sentences derail reasoning; (3) <em>Very large contexts → performance drop</em> — despite advertised 32k+ windows, results and stability degrade. Below: why this happens, what works “right now,” what to implement in the model/pipeline, and how to measure it rigorously.</p>
<hr />
<h2>TL;DR for the impatient</h2>
<ul>
<li style="text-align: justify;"><strong>Instead of stuffing everything into the prompt</strong>: retrieval → cross-encoder reranking → compression → <em>extreme ordering</em> (most important at the beginning <strong>and</strong> end).</li>
<li style="text-align: justify;"><strong>Limit distraction</strong> with a simple instruction + answer format, few-shot with “noise,” <em>self-consistency</em>, and <em>gating/abstention</em> (<em>NO-RESPONSE</em>) at the passage level.</li>
<li style="text-align: justify;"><strong>Stabilise long context</strong> via position scaling (LongRoPE/YaRN), a training regime for long sequences (ProLong), test-time adaptation (LIFT), <em>streaming attention</em> with sink tokens and/or external memory.</li>
<li style="text-align: justify;"><strong>Measure smartly</strong>: not only “needle-in-a-haystack.” Use RULER/ONERULER (also multilingual), multi-needle tests, and real tasks with source citation.</li>
</ul>
</section>
<p><span id="more-1921"></span></p>
<section>
<h2>1. Lost in the Middle — diagnosis and fixes</h2>
<h3>Why it happens</h3>
<p style="text-align: justify;">Models with popular positional schemes (e.g., RoPE and position extrapolation methods) exhibit a <strong>positional bias</strong>: they usually “retain” the beginning and the end of a sequence better, while the middle is often the weakest. When the needle lands in the middle, accuracy drops — even in “long-context” models.</p>
<h3>What works immediately (no training)</h3>
<ol>
<li style="text-align: justify;"><strong>Extreme ordering</strong>: after reranking, place the most important fragments at the <strong>beginning</strong> and <strong>end</strong> of context (use the middle for supporting material).</li>
<li style="text-align: justify;"><strong>Iterative retrieval</strong> (Self-RAG/FLARE): don’t load everything at once — generate in steps and pull missing facts on demand.</li>
<li style="text-align: justify;"><strong>Hierarchical summaries</strong> (map → reduce): condense sections to 1–2 sentences and compose a “map” of arguments.</li>
</ol>
<h3>What works at the model/pipeline level</h3>
<ul>
<li style="text-align: justify;"><strong>Cross-encoder reranking</strong> (cross-encoder/LLM ranker) before arranging context.</li>
<li style="text-align: justify;"><strong>Better positional handling</strong>: LongRoPE/YaRN/PI — more stable windows; and <strong>Ms-PoE</strong> (<em>Found in the Middle</em>) targeted at improving the “middle.”</li>
<li style="text-align: justify;"><strong>SFT with positional debiasing</strong>: rotation/shuffling of order, needles placed at different positions; generally — positional augmentation aligned with the latest long-context guidance.</li>
</ul>
<blockquote><p><strong>Prompt pattern (minimal)</strong></p>
<pre escaped="true"><code>TASK: {question}
RULES: Use only fragments marked [IMPORTANT]. Omit [ADDITIONAL].
FORMAT:
[EVIDENCE] bullet points with a citation --&gt; {passage_id}
[ANSWER] concise conclusion
</code></pre>
</blockquote>
</section>
<hr />
<section>
<h2>2. Distracting prompts and the “tempting” noise</h2>
<h3>Why it happens</h3>
<p style="text-align: justify;">LLMs are sensitive to distractors that are semantically similar to the correct answer. A single irrelevant sentence can steer reasoning off course.</p>
<h3>Tactics without training</h3>
<ol>
<li style="text-align: justify;"><strong>Instruction + answer format</strong>: ask to list <em>evidence/relevant sentences</em> first, then provide the conclusion.</li>
<li style="text-align: justify;"><strong>Few-shot with distractors</strong>: in examples, show the step “filter out irrelevant → solve.”</li>
<li style="text-align: justify;"><strong>Self-Consistency</strong>: several independent reasoning trajectories and majority voting.</li>
<li style="text-align: justify;"><strong>Passage-level gating/abstention</strong>: test each fragment — if it doesn’t contain necessary information, the model should output <code>NO-RESPONSE</code>; apply this filter before final generation.</li>
</ol>
<h3>Noise-robust RAG pipeline</h3>
<p style="text-align: justify;"><em>Retriever → Cross-Encoder Rerank → NLI/entailment filters → Compression (LongLLMLingua/LLMLingua-2) → Extreme ordering → Gating/abstention → Generation with self-consistency.</em></p>
<blockquote><p>Prompt pattern (passage gating)</p>
<pre escaped="true"><code>For each passage do:
1) Does this passage contain information required to answer "{question}"?
2) If NO, return: NO-RESPONSE | {id}
3) If YES, return: EVIDENCE | {id} | {short_quote}
</code></pre>
</blockquote>
</section>
<hr />
<section>
<h2>3. Very large contexts and performance degradation</h2>
<h3>Where degradation comes from</h3>
<ul>
<li style="text-align: justify;"><strong>Positions</strong>: extrapolation beyond the training distribution.</li>
<li style="text-align: justify;"><strong>Optimisation</strong>: long sequences strain the KV cache and attention stability.</li>
<li style="text-align: justify;"><strong>Selection</strong>: the practically useful portion is smaller than the declared window; the rest often is noise.</li>
</ul>
<h3>Layers of defence (from practice to architecture)</h3>
<h4>A. Practice “by tomorrow”</h4>
<ul>
<li style="text-align: justify;"><strong>Context compression</strong> (LongLLMLingua/LLMLingua-2): 2–6× shorter inputs; in many tasks <em>no loss</em>, sometimes <em>improved</em> quality; reported speedups ~1.4–2.9× depending on the task.</li>
<li style="text-align: justify;"><strong>Adaptive retrieval</strong> (Self-RAG/FLARE): “retrieve only when needed,” with self-reflection on sources.</li>
<li style="text-align: justify;"><strong>Positional budget</strong>: aim for ~20–40% of the window for hard evidence; leave the rest for the model’s own reasoning <em>(heuristic — A/B test on your data)</em>.</li>
</ul>
<h4>B. Model and inference</h4>
<ul>
<li style="text-align: justify;"><strong>LongRoPE/YaRN/PI</strong>: stabilisation and extension of 32k+ windows.</li>
<li style="text-align: justify;"><strong>ProLong</strong>: continued pre-training + SFT for long sequences (models up to 512k).</li>
<li style="text-align: justify;"><strong>LIFT (test-time adaptation)</strong>: adapter/learning with long input at test time; improves long-context understanding.</li>
<li style="text-align: justify;"><strong>Streaming + attention sinks</strong>: a fixed pool of anchor tokens + sliding KV window for long streams.</li>
<li style="text-align: justify;"><strong>External memory</strong>: LM-Infinite/InfLLM and related — cheaper than linear prompt growth.</li>
</ul>
<h4>C. Multilinguality</h4>
<ul>
<li style="text-align: justify;"><strong>Language alignment</strong> (instruction ≈ documents) or normalise to a single language.</li>
<li style="text-align: justify;"><strong>Per-language evaluation</strong> — gaps grow with sequence length; use multilingual benchmarks.</li>
</ul>
</section>
<hr />
<section>
<h2>A cohesive production pipeline (sketch)</h2>
<ol>
<li style="text-align: justify;"><strong>Retrieval</strong>: <code>BM25 + bi-encoder (dense)</code>; take top-50.</li>
<li style="text-align: justify;"><strong>Reranking</strong>: cross-encoder/LLM-ranker → top-k (e.g., 8–12).</li>
<li style="text-align: justify;"><strong>Compression</strong>: LongLLMLingua-2/LongLLMLingua targeting ~3–5k tokens.</li>
<li style="text-align: justify;"><strong>Ordering</strong>: <em>extreme ordering</em> — most important at the <strong>beginning and end</strong>; the middle = support.</li>
<li style="text-align: justify;"><strong>Gating</strong>: quick <em>NO-RESPONSE</em> pass over passages.</li>
<li style="text-align: justify;"><strong>Generation</strong>: <em>self-consistency</em> (n=3–7) + critic/reflection over citations.</li>
<li style="text-align: justify;"><strong>64k+ streams</strong>: enable <em>streaming attention</em> with sink tokens and a KV budget.</li>
</ol>
<pre escaped="true"><code class="language-python"># Pseudocode
docs = retriever(q, k=50)
scored = cross_encoder.rank(q, docs)
sel = topk(scored, 10)
compressed = longllmlingua(sel, target_tokens=4000)
ordered = extreme_ordering(compressed)   # front + tail boost
kept = passage_gating(ordered, rule="NO-RESPONSE")
answer = llm.generate(kept, self_consistency=5, cite_sources=True)
</code></pre>
</section>
<hr />
<section>
<h2>How to measure progress rigorously</h2>
<h3>Test suite</h3>
<ul>
<li style="text-align: justify;"><strong>RULER / ONERULER</strong>: lengths 8k→128k, multiple needles, language variants (retrieval, tracking, aggregation).</li>
<li style="text-align: justify;"><strong>“Needle-in-the-Haystack” on steroids</strong>: multiple needles + “hard negatives.”</li>
<li style="text-align: justify;"><strong>Real tasks</strong>: QA/analyses with <em>faithfulness</em> scoring (does it cite the correct sources?).</li>
</ul>
<h3>Metrics</h3>
<ul>
<li style="text-align: justify;"><em>Exact/EM, F1, citation precision/recall</em>, latency, tokens/query cost, % of “abstain” answers.</li>
<li style="text-align: justify;">Curves for <strong>quality vs. length</strong> (8k, 16k, 32k, 64k…) and <strong>quality vs. context budget</strong>.</li>
</ul>
<h3>Simple A/B plan</h3>
<ol>
<li style="text-align: justify;">Dataset of ~200–500 questions (min. 3 domains, 2 languages).</li>
<li style="text-align: justify;">Conditions: baseline, +reranking, +compression, +ordering, +gating, +self-consistency.</li>
<li style="text-align: justify;">Error analysis: separately “middle of context,” “distractors,” “overfill.”</li>
<li style="text-align: justify;">Report: table with cost and quality gains per condition.</li>
</ol>
</section>
<hr />
<section>
<h2>Implementation checklists</h2>
<h3>Lost in the Middle</h3>
<ul>
<li>Cross-encoder reranking before inserting into the prompt.</li>
<li>Extreme ordering (front+tail).</li>
<li>Hierarchical summaries (map→reduce).</li>
<li>If possible: LongRoPE/YaRN/PI + Ms-PoE; positional augmentation during SFT.</li>
</ul>
<h3>Distracting prompts</h3>
<ul>
<li>Instruction “use only necessary information” + format with [EVIDENCE]→[ANS.].</li>
<li>Few-shot with noise and filtering.</li>
<li>Gating/abstention (<code>NO-RESPONSE</code>) at passage level.</li>
<li>Self-consistency (n≥3) for sensitive tasks.</li>
</ul>
<h3>Very large contexts</h3>
<ul>
<li style="text-align: justify;">Compression (LongLLMLingua/LLMLingua-2) before feeding the LLM.</li>
<li style="text-align: justify;">Streaming attention + sink tokens for streams.</li>
<li style="text-align: justify;">External memory instead of “everything in the prompt.”</li>
<li style="text-align: justify;">ProLong/LIFT when you control model/inference.</li>
</ul>
</section>
<hr />
<section>
<h2>Common anti-patterns (and how to fix them)</h2>
<ul>
<li style="text-align: justify;"><strong>“I’ll add more context; it will be better”</strong> → <em>No</em>. First select/compress, then order.</li>
<li style="text-align: justify;"><strong>“We’ll add 20 few-shot examples”</strong> → A shorter, well-curated pattern + format is often more robust.</li>
<li style="text-align: justify;"><strong>“Since the window is 128k, we’ll use 120k”</strong> → Practically effective is often 20–40% of the window. Keep the rest in an index/memory <em>(heuristic → verify via A/B)</em>.</li>
</ul>
</section>
<hr />
<section>
<h2>To wrap up</h2>
<p style="text-align: justify;">Long context is tempting, but not free. The best results come from <em>selection and compression</em>, conscious handling of positional bias (front+tail), and a pipeline that <strong>knows when to be silent</strong> (gating/abstention) and <strong>when to pull more sources</strong> (iterative retrieval). Architectural and training changes can add extra points, but first squeeze the most out of the application layer.</p>
</section>
<section>
<h2 style="text-align: justify;">References and selected sources</h2>
<ol>
<li style="text-align: justify;">Liu, N. F., et al. (2023/2024). <a href="https://arxiv.org/abs/2307.03172" target="_blank" rel="nofollow noopener">Lost in the Middle: How Language Models Use Long Contexts</a> / TACL version: <a href="https://aclanthology.org/2024.tacl-1.9/" target="_blank" rel="nofollow noopener">PDF</a>.</li>
<li style="text-align: justify;">Shi, F., et al. (2023). <a href="https://arxiv.org/abs/2302.00093" target="_blank" rel="nofollow noopener">Large Language Models Can Be Easily Distracted by Irrelevant Context</a> (ICLR). Alt: <a href="https://dl.acm.org/doi/10.5555/3618408.3619699" target="_blank" rel="nofollow noopener">publisher version</a>.</li>
<li style="text-align: justify;">Hsieh, C.-P., et al. (2024). <a href="https://arxiv.org/abs/2404.06654" target="_blank" rel="nofollow noopener">RULER: What’s the Real Context Size of Your Long-Context Language Models?</a> (+ <a href="https://github.com/NVIDIA/RULER" target="_blank" rel="nofollow noopener">repo</a>).</li>
<li style="text-align: justify;">Kim, Y., et al. (2025). <a href="https://arxiv.org/abs/2503.01996" target="_blank" rel="nofollow noopener">OneRULER: Benchmarking multilingual long-context language models</a> (OpenReview: <a href="https://openreview.net/forum?id=3vxxB3Ar9r" target="_blank" rel="nofollow noopener">page</a>).</li>
<li style="text-align: justify;">Hengle, A., et al. (2025). <a href="https://arxiv.org/abs/2504.12845" target="_blank" rel="nofollow noopener">Can LLMs reason over extended multilingual contexts?</a></li>
<li style="text-align: justify;">Ding, Y., et al. (2024). <a href="https://arxiv.org/abs/2402.13753" target="_blank" rel="nofollow noopener">LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens</a>.</li>
<li style="text-align: justify;">Peng, B., et al. (2023). <a href="https://arxiv.org/abs/2309.00071" target="_blank" rel="nofollow noopener">YaRN: Efficient Context Window Extension of Large Language Models</a> (OpenReview: <a href="https://openreview.net/forum?id=wHBfxhZu1u" target="_blank" rel="nofollow noopener">page</a>).</li>
<li style="text-align: justify;">Chen, S., et al. (2023). <a href="https://arxiv.org/abs/2306.15595" target="_blank" rel="nofollow noopener">Position Interpolation (PI)</a>.</li>
<li style="text-align: justify;">Gao, T., Wettig, A., Yen, H., Chen, D. (2024/2025). <a href="https://arxiv.org/abs/2410.02660" target="_blank" rel="nofollow noopener">How to Train Long-Context Language Models (Effectively)</a> (ACL Findings 2025: <a href="https://aclanthology.org/2025.acl-long.366.pdf" target="_blank" rel="nofollow noopener">PDF</a>; ProLong: <a href="https://github.com/princeton-nlp/ProLong" target="_blank" rel="nofollow noopener">repo</a>).</li>
<li style="text-align: justify;">Mao, Y., et al. (2024/2025). <a href="https://arxiv.org/abs/2412.13626" target="_blank" rel="nofollow noopener">LIFT: Improving Long Context Understanding Through Long Input Fine-Tuning</a>.</li>
<li style="text-align: justify;">Xiao, G., et al. (2023/ICLR-2024). <a href="https://arxiv.org/abs/2309.17453" target="_blank" rel="nofollow noopener">Efficient Streaming Language Models with Attention Sinks (StreamingLLM)</a> (GitHub: <a href="https://github.com/mit-han-lab/streaming-llm" target="_blank" rel="nofollow noopener">repo</a>).</li>
<li style="text-align: justify;">Han, C., et al. (2024). <a href="https://aclanthology.org/2024.naacl-long.222/" target="_blank" rel="nofollow noopener">LM-Infinite: Zero-Shot Extreme Length Generalization for LLMs</a>.</li>
<li style="text-align: justify;">Xiao, C., et al. (2024). <a href="https://arxiv.org/abs/2402.04617" target="_blank" rel="nofollow noopener">InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory</a>.</li>
<li style="text-align: justify;">Jiang, H., et al. (2023). <a href="https://arxiv.org/abs/2310.05736" target="_blank" rel="nofollow noopener">LLMLingua</a>; Jiang, H., et al. (2024). <a href="https://arxiv.org/abs/2310.06839" target="_blank" rel="nofollow noopener">LongLLMLingua</a> (ACL-Long 2024: <a href="https://aclanthology.org/2024.acl-long.91.pdf" target="_blank" rel="nofollow noopener">PDF</a>); Pan, Z., et al. (2024). <a href="https://arxiv.org/abs/2403.12968" target="_blank" rel="nofollow noopener">LLMLingua-2</a> (MSR: <a href="https://www.microsoft.com/en-us/research/project/llmlingua/llmlingua-2/" target="_blank" rel="nofollow noopener">page</a>).</li>
<li style="text-align: justify;">Asai, A., et al. (2023). <a href="https://arxiv.org/abs/2310.11511" target="_blank" rel="nofollow noopener">Self-RAG: Learning to Retrieve, Generate, and Critique</a> (site: <a href="https://selfrag.github.io/" target="_blank" rel="nofollow noopener">selfrag.github.io</a>).</li>
<li style="text-align: justify;">Jiang, Z., et al. (2023). <a href="https://arxiv.org/abs/2305.06983" target="_blank" rel="nofollow noopener">FLARE: Active Retrieval Augmented Generation</a> (EMNLP 2023: <a href="https://aclanthology.org/2023.emnlp-main.495/" target="_blank" rel="nofollow noopener">page</a>).</li>
<li style="text-align: justify;">Zhang, Z., et al. (2024). <a href="https://arxiv.org/abs/2403.04797" target="_blank" rel="nofollow noopener">Found in the Middle: Plug-and-Play Positional Encoding (Ms-PoE)</a>.</li>
<li style="text-align: justify;">Amiraz, C., Cuconasu, F., Filice, S., Karnin, Z. (2025). <a href="https://arxiv.org/abs/2505.06914" target="_blank" rel="nofollow noopener">The Distracting Effect: Understanding Irrelevant Passages in RAG</a> (ACL 2025: <a href="https://aclanthology.org/2025.acl-long.892.pdf" target="_blank" rel="nofollow noopener">PDF</a>).</li>
<li style="text-align: justify;">(Reranking) Nogueira, R., Cho, K. (2019). <a href="https://arxiv.org/abs/1901.04085" target="_blank" rel="nofollow noopener">Passage Re-ranking with BERT</a>.</li>
<li style="text-align: justify;">(Dense late interaction) Santhanam, K., et al. (2022). <a href="https://arxiv.org/abs/2112.01488" target="_blank" rel="nofollow noopener">ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction</a> (NAACL 2022: <a href="https://aclanthology.org/2022.naacl-main.272.pdf" target="_blank" rel="nofollow noopener">PDF</a>).</li>
</ol>
</section>
</article>
]]></content:encoded>
					
					<wfw:commentRss>https://profind.pl/en/2025/11/04/taming-long-context-in-llms-3-problems-1-cohesive-set-of-strategies/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>Revolution in the Test Tube: How AI Agents Are Transforming Scientific Research</title>
		<link>https://profind.pl/en/2025/11/03/revolution-in-the-test-tube-how-ai-agents-are-transforming-scientific-research/</link>
					<comments>https://profind.pl/en/2025/11/03/revolution-in-the-test-tube-how-ai-agents-are-transforming-scientific-research/#respond</comments>
		
		<dc:creator><![CDATA[Marcin]]></dc:creator>
		<pubDate>Mon, 03 Nov 2025 12:59:36 +0000</pubDate>
				<category><![CDATA[Research diary]]></category>
		<category><![CDATA[AI agents]]></category>
		<category><![CDATA[AI in science]]></category>
		<category><![CDATA[AlphaEvolve]]></category>
		<category><![CDATA[AlphaFold 3]]></category>
		<category><![CDATA[ATOMICA]]></category>
		<category><![CDATA[biomolecular interactions]]></category>
		<category><![CDATA[Co-Scientist]]></category>
		<category><![CDATA[CrystaLLM]]></category>
		<category><![CDATA[ECMWF AIFS]]></category>
		<category><![CDATA[ESM3]]></category>
		<category><![CDATA[esmGFP]]></category>
		<category><![CDATA[Genie 2]]></category>
		<category><![CDATA[IMO 2025]]></category>
		<category><![CDATA[materials discovery]]></category>
		<category><![CDATA[mathematical reasoning]]></category>
		<category><![CDATA[nanobodies]]></category>
		<category><![CDATA[ORBIT climate model]]></category>
		<category><![CDATA[protein design]]></category>
		<category><![CDATA[robotic synthesis]]></category>
		<category><![CDATA[self-driving lab]]></category>
		<category><![CDATA[virtual lab]]></category>
		<category><![CDATA[weather forecasting]]></category>
		<category><![CDATA[WeatherNext]]></category>
		<category><![CDATA[world models]]></category>
		<guid isPermaLink="false">https://profind.pl/?p=1900</guid>

					<description><![CDATA[From Data Analysis to a Partner in Discovery: Artificial intelligence in science is undergoing a profound transformation. For years it was seen mainly as a tool for analysing large datasets. Today we’re witnessing an evolution from a passive analyst to an active partner in research — AI systems can help formulate hypotheses, design experiments, and [&#8230;]]]></description>
										<content:encoded><![CDATA[<p style="text-align: justify;"><strong>From Data Analysis to a Partner in Discovery:</strong> Artificial intelligence in science is undergoing a profound transformation. For years it was seen mainly as a tool for analysing large datasets. Today we’re witnessing an evolution from a passive analyst to an active partner in research — AI systems can help formulate hypotheses, design experiments, and interpret results (always under human oversight). This trend is well documented in 2025 surveys and reports, including the <a href="https://hai.stanford.edu/ai-index/2025-ai-index-report" target="_blank" rel="noopener">Stanford AI Index 2025</a> and the <a href="https://www.stateof.ai/" target="_blank" rel="noopener">State of AI Report 2025</a>.</p>
<p><span id="more-1900"></span></p>
<h3>1. The new paradigm: concrete uses of AI agents in science</h3>
<p style="text-align: justify;">Specialised “scientist agents” are accelerating the discovery cycle across multiple disciplines. Below are examples from 2024–2025 grounded in peer-reviewed publications and official reports.</p>
<h4>1.1. Biology &amp; medicine: decoding the “language of life”</h4>
<ul>
<li style="text-align: justify;"><strong>Protein and drug design.</strong> The <strong>ESM3</strong> model (EvolutionaryScale) generated an entirely new fluorescent protein, <em>esmGFP</em>, documented in <em>Science</em> (2025) — the work shows large evolutionary distance from natural FPs and experimental validation. See <a href="https://www.science.org/doi/10.1126/science.ads0018" target="_blank" rel="noopener">Science (2025)</a> and the team’s blog <a href="https://www.evolutionaryscale.ai/blog/esm3-release" target="_blank" rel="noopener">EvolutionaryScale</a>.</li>
<li style="text-align: justify;"><strong>Structures and biomolecular interactions.</strong> <strong>AlphaFold 3</strong> (2024) broadened prediction to complexes involving proteins, nucleic acids and small molecules <a href="https://www.nature.com/articles/s41586-024-07487-w" target="_blank" rel="noopener">[Nature, 2024]</a>. Independent analyses also highlight limitations for some RNA cases and edge scenarios <a href="https://pubs.acs.org/doi/10.1021/acs.jcim.5c00921" target="_blank" rel="noopener">[JCIM, 2025]</a>, <a href="https://cen.acs.org/physical-chemistry/computational-chemistry/Researchers-find-weaknesses-AI-structure/103/web/2025/04" target="_blank" rel="noopener">[C&amp;EN, 2025]</a>.</li>
<li style="text-align: justify;"><strong>“Co-Scientist”.</strong> A multi-agent system from Google DeepMind proposed and tested biological mechanisms; results on bacterial gene transfer were published in <em>Cell</em> (2025) <a href="https://www.cell.com/cell/fulltext/S0092-8674(25)00973-0" target="_blank" rel="noopener">[Cell, 2025]</a>. Overview: <a href="https://research.google/blog/accelerating-scientific-breakthroughs-with-an-ai-co-scientist/" target="_blank" rel="noopener">Google Research</a>.</li>
<li style="text-align: justify;"><strong>Virtual lab &amp; nanobodies.</strong> A Stanford-led team demonstrated a “Virtual Lab” — a group of AI agents working with scientists — that designed <em>92</em> nanobodies neutralising SARS-CoV-2 variants; selected constructs were experimentally validated and published in <em>Nature</em> (2025): <a href="https://www.nature.com/articles/s41586-025-09442-9" target="_blank" rel="noopener">Nature (article)</a>. See also summaries by <a href="https://med.stanford.edu/news/all-news/2025/07/virtual-scientist.html" target="_blank" rel="noopener">Stanford Medicine (news)</a>, <a href="https://www.reuters.com/business/healthcare-pharmaceuticals/health-rounds-virtual-labs-with-ai-scientists-produce-promising-result-stanford-2025-07-30/" target="_blank" rel="noopener">Reuters — Health Rounds</a>, and <a href="https://www.nature.com/articles/d41586-024-01684-3" target="_blank" rel="noopener">Nature News</a> on the preprint.</li>
<li style="text-align: justify;"><strong>Universal interaction models.</strong> <em>ATOMICA</em> learns representations of intermolecular interfaces (protein–ligand, protein–RNA, etc.); this is a preprint with experimental validation of selected cases <a href="https://www.biorxiv.org/content/10.1101/2025.04.02.646906v2" target="_blank" rel="noopener">[bioRxiv, 2025]</a> (project page: <a href="https://zitniklab.hms.harvard.edu/projects/ATOMICA/" target="_blank" rel="noopener">Harvard</a>).</li>
</ul>
<h4>1.2. Chemistry &amp; materials: towards autonomous labs</h4>
<ul>
<li style="text-align: justify;"><strong>Robotised synthesis.</strong> A classic reference is Liverpool’s mobile “robot chemist” (Nature, 2020), which planned and ran experiments autonomously <a href="https://www.nature.com/articles/s41586-020-2442-2" target="_blank" rel="noopener">[Nature, 2020]</a>. Newer “self-driving lab” platforms markedly increase throughput; for example, NC State’s lines demonstrate high-rate flow experiments and data intensification reported in <em>Nature Chemical Engineering</em> 2025 <a href="https://www.sciencedaily.com/releases/2025/07/250714052105.htm" target="_blank" rel="noopener">[overview]</a>, <a href="https://www.abolhasanilab.com/2025" target="_blank" rel="noopener">[NC State lab news]</a>.</li>
<li style="text-align: justify;"><strong>New materials from generative models.</strong> Approaches based on LLMs and diffusion (e.g., <em>CrystaLLM</em>) generate stable candidate crystal structures, corroborated by simulation <a href="https://www.nature.com/articles/s41467-024-54639-7" target="_blank" rel="noopener">[Nat. Commun., 2024]</a>. Recent work combines LLMs with diffusion to improve stability and novelty <a href="https://arxiv.org/abs/2510.23040" target="_blank" rel="noopener">[arXiv, 2025]</a>, and LLMs alone can yield stable crystals without finetuning in <em>MatLLMSearch</em> <a href="https://arxiv.org/abs/2502.20933" target="_blank" rel="noopener">[arXiv, 2025]</a>.</li>
</ul>
<h4>1.3. Formal &amp; computational sciences</h4>
<ol>
<li style="text-align: justify;"><strong>Algorithm discovery.</strong> <em>AlphaEvolve</em> (DeepMind) is an evolutionary coding agent that proposed and <em>verified</em> new algorithms. The team reports faster 4×4 matrix multiplication (48 scalar mults) and speedups to selected compute kernels in Google’s stack — see the <a href="https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/" target="_blank" rel="noopener">blog</a> and <a href="https://arxiv.org/abs/2506.13131" target="_blank" rel="noopener">arXiv (2025)</a>.</li>
<li style="text-align: justify;"><strong>Mathematical reasoning.</strong> In 2025, systems from OpenAI and Google achieved “gold-level” performance at the International Mathematical Olympiad, solving 5/6 problems under time constraints — see <a href="https://www.reuters.com/technology/ai-intelligencer-how-ai-won-math-gold-2025-07-24/" target="_blank" rel="noopener">Reuters</a> and <a href="https://deepmind.google/discover/blog/advanced-version-of-gemini-with-deep-think-officially-achieves-gold-medal-standard-at-the-international-mathematical-olympiad/" target="_blank" rel="noopener">DeepMind</a> (with <a href="https://storage.googleapis.com/deepmind-media/gemini/IMO_2025.pdf" target="_blank" rel="noopener">solutions PDF</a>).</li>
<li style="text-align: justify;"><strong>Earth-system &amp; weather models.</strong> <em>ORBIT</em> (ORNL) scales to ~113B parameters on Frontier <a href="https://arxiv.org/abs/2404.14712" target="_blank" rel="noopener">[arXiv, 2024]</a>, <a href="https://www.ornl.gov/news/fine-tuning-forecasts-orbit-brings-long-range-weather-prediction-within-reach" target="_blank" rel="noopener">[ORNL news, 2024]</a>. In weather forecasting, the <em>WeatherNext</em>/GenCast family (Google DeepMind) achieves high-quality probabilistic forecasts up to 15 days — see <a href="https://deepmind.google/discover/blog/gencast-predicts-weather-and-the-risks-of-extreme-conditions-with-sota-accuracy/" target="_blank" rel="noopener">GenCast</a>, the <a href="https://www.nature.com/articles/s41586-024-08252-9" target="_blank" rel="noopener">Nature article</a>, and the public dataset in Earth Engine <a href="https://developers.google.com/earth-engine/datasets/catalog/projects_gcp-public-data-weathernext_assets_126478713_1_0" target="_blank" rel="noopener">WeatherNext Gen</a>. In parallel, <strong>ECMWF</strong> has made its AI Forecast System (<em>AIFS</em>) operational with up-to-15-day forecasts <a href="https://www.ecmwf.int/en/about/media-centre/news/2025/ecmwfs-ai-forecasts-become-operational" target="_blank" rel="noopener">[ECMWF, 2025]</a>.</li>
</ol>
<p style="text-align: justify;">These examples fuel innovation but still demand careful validation.</p>
<h3>2. “Engines of discovery”: technologies powering the revolution</h3>
<h4>2.1. Advanced reasoning &amp; planning</h4>
<p style="text-align: justify;">New models (e.g., o1/o3, Gemini Deep Think) use “think-before-answering” techniques and increased test-time compute. At the same time, research shows brittleness to distractors — models can fail due to small, irrelevant additions to a problem <a href="https://arxiv.org/abs/2503.01781" target="_blank" rel="noopener">[arXiv: Cats Confuse Reasoning LLMs, 2025]</a>.</p>
<h4>2.2. Agent systems &amp; tool use</h4>
<p style="text-align: justify;">A true scientific agent integrates an LLM “brain” with digital and physical tools via function calling/APIs — from structural databases to robotised lab equipment. Practical demonstrations include Co-Scientist <a href="https://research.google/blog/accelerating-scientific-breakthroughs-with-an-ai-co-scientist/" target="_blank" rel="noopener">[Google Research]</a> and virtual labs <a href="https://www.nature.com/articles/s41586-025-09442-9" target="_blank" rel="noopener">[Nature, 2025]</a>.</p>
<h4>2.3. Reinforcement learning with verifiable rewards (RLVR)</h4>
<p style="text-align: justify;">Where correctness can be automatically checked (proofs, program tests, molecular properties), RL with verifiable rewards speeds up learning to reason — yet stability and generalisation still require rigorous evaluation. <em>(See AI Index 2025 for an overview; examples include AlphaEvolve and IMO 2025.)</em> See also DeepSeek-R1 in <em>Nature</em> <a href="https://www.nature.com/articles/s41586-025-09422-z" target="_blank" rel="noopener">[2025]</a> and a recent survey <a href="https://arxiv.org/abs/2506.14245" target="_blank" rel="noopener">[arXiv, 2025]</a>.</p>
<h4>2.4. World models &amp; simulation</h4>
<p style="text-align: justify;">The <strong>Genie</strong> family (DeepMind) creates interactive, controllable worlds for low-cost hypothesis testing <a href="https://deepmind.google/discover/blog/genie-2-a-large-scale-foundation-world-model/" target="_blank" rel="noopener">Genie 2</a>, <a href="https://deepmind.google/discover/blog/genie-3-a-new-frontier-for-world-models/" target="_blank" rel="noopener">Genie 3</a>. Such simulations shorten the “hypothesis → experiment” loop before moving to expensive wet-lab work.</p>
<h3>3. The lab of the future: challenges and prospects</h3>
<h4>3.1. Towards integrated “AI labs”</h4>
<p style="text-align: justify;">The strategic goal is an integrated chain: from literature review and hypothesis generation through planning and running experiments (in silico + robotics) to analysis and manuscript drafting — all with a human in the loop. The concept of <em>open-endedness</em> (constantly raising difficulty and competence) is becoming central.</p>
<h4>3.2. Challenges and limitations</h4>
<table>
<thead>
<tr>
<th>Technical challenge</th>
<th>Implications</th>
</tr>
</thead>
<tbody>
<tr>
<td>Brittleness of reasoning</td>
<td>Susceptibility to distractors and errors in multi-step tasks (<a href="https://arxiv.org/abs/2503.01781" target="_blank" rel="noopener">arXiv, 2025</a>).</td>
</tr>
<tr>
<td>Out-of-distribution generalisation</td>
<td>Quality drops on novel problem classes; examples of limitations in AF3 for RNA/unusual cases (<a href="https://pubs.acs.org/doi/10.1021/acs.jcim.5c00921" target="_blank" rel="noopener">JCIM, 2025</a>; <a href="https://cen.acs.org/physical-chemistry/computational-chemistry/Researchers-find-weaknesses-AI-structure/103/web/2025/04" target="_blank" rel="noopener">C&amp;EN, 2025</a>).</td>
</tr>
<tr>
<td>Real-world validation</td>
<td>Predictions and simulations are hypotheses — drug candidates, new molecules or materials require costly, time-consuming validation in wet labs and/or production (TRL).</td>
</tr>
</tbody>
</table>
<h4>3.3. The evolving role of scientists</h4>
<p style="text-align: justify;">AI will not replace researchers; it will amplify them — automating tedious steps and freeing time for creative questions, cross-disciplinary synthesis, and ethical oversight. The biggest wins come from human<img src="https://s.w.org/images/core/emoji/17.0.2/72x72/2194.png" alt="↔" class="wp-smiley" style="height: 1em; max-height: 1em;" />machine synergy.</p>
<h3>References &amp; selected sources</h3>
<ul>
<li><strong>Stanford AI Index 2025</strong> — <a href="https://hai.stanford.edu/ai-index/2025-ai-index-report" target="_blank" rel="noopener">report page</a></li>
<li><strong>State of AI Report 2025</strong> — <a href="https://www.stateof.ai/" target="_blank" rel="noopener">online</a></li>
<li><strong>ESM3 / esmGFP</strong> — <a href="https://www.science.org/doi/10.1126/science.ads0018" target="_blank" rel="noopener">Science (2025)</a>, <a href="https://www.evolutionaryscale.ai/blog/esm3-release" target="_blank" rel="noopener">EvolutionaryScale blog</a></li>
<li><strong>AlphaFold 3</strong> — <a href="https://www.nature.com/articles/s41586-024-07487-w" target="_blank" rel="noopener">Nature (2024)</a>; limitations: <a href="https://pubs.acs.org/doi/10.1021/acs.jcim.5c00921" target="_blank" rel="noopener">JCIM (2025)</a>, <a href="https://cen.acs.org/physical-chemistry/computational-chemistry/Researchers-find-weaknesses-AI-structure/103/web/2025/04" target="_blank" rel="noopener">C&amp;EN (2025)</a></li>
<li><strong>Virtual Lab / nanobodies</strong> — <a href="https://www.nature.com/articles/s41586-025-09442-9" target="_blank" rel="noopener">Nature (2025)</a>, <a href="https://med.stanford.edu/news/all-news/2025/07/virtual-scientist.html" target="_blank" rel="noopener">Stanford Medicine</a>, <a href="https://www.reuters.com/business/healthcare-pharmaceuticals/health-rounds-virtual-labs-with-ai-scientists-produce-promising-result-stanford-2025-07-30/" target="_blank" rel="noopener">Reuters</a></li>
<li><strong>ATOMICA</strong> — <a href="https://www.biorxiv.org/content/10.1101/2025.04.02.646906v2" target="_blank" rel="noopener">bioRxiv (2025)</a>, <a href="https://zitniklab.hms.harvard.edu/projects/ATOMICA/" target="_blank" rel="noopener">project page</a></li>
<li><strong>AlphaEvolve</strong> — <a href="https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/" target="_blank" rel="noopener">blog</a>, <a href="https://arxiv.org/abs/2506.13131" target="_blank" rel="noopener">arXiv (2025)</a></li>
<li><strong>IMO 2025 — gold level</strong> — <a href="https://www.reuters.com/technology/ai-intelligencer-how-ai-won-math-gold-2025-07-24/" target="_blank" rel="noopener">Reuters</a>, <a href="https://deepmind.google/discover/blog/advanced-version-of-gemini-with-deep-think-officially-achieves-gold-medal-standard-at-the-international-mathematical-olympiad/" target="_blank" rel="noopener">DeepMind</a>, <a href="https://storage.googleapis.com/deepmind-media/gemini/IMO_2025.pdf" target="_blank" rel="noopener">solutions (PDF)</a></li>
<li><strong>ORBIT (113B) / climate models</strong> — <a href="https://arxiv.org/abs/2404.14712" target="_blank" rel="noopener">arXiv (2024)</a>, <a href="https://www.ornl.gov/news/fine-tuning-forecasts-orbit-brings-long-range-weather-prediction-within-reach" target="_blank" rel="noopener">ORNL news</a></li>
<li><strong>WeatherNext / GenCast</strong> — <a href="https://deepmind.google/science/weathernext/" target="_blank" rel="noopener">project page</a>, <a href="https://www.nature.com/articles/s41586-024-08252-9" target="_blank" rel="noopener">Nature</a>, <a href="https://developers.google.com/earth-engine/datasets/catalog/projects_gcp-public-data-weathernext_assets_126478713_1_0" target="_blank" rel="noopener">Earth Engine dataset</a></li>
<li><strong>ECMWF AIFS</strong> — <a href="https://www.ecmwf.int/en/about/media-centre/news/2025/ecmwfs-ai-forecasts-become-operational" target="_blank" rel="noopener">press release (2025)</a></li>
<li><strong>Brittleness of reasoning</strong> — <a href="https://arxiv.org/abs/2503.01781" target="_blank" rel="noopener">arXiv (2025)</a></li>
</ul>
]]></content:encoded>
					
					<wfw:commentRss>https://profind.pl/en/2025/11/03/revolution-in-the-test-tube-how-ai-agents-are-transforming-scientific-research/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>5 Surprising Truths About the AI Revolution</title>
		<link>https://profind.pl/en/2025/11/03/5-surprising-truths-about-the-ai-revolution/</link>
					<comments>https://profind.pl/en/2025/11/03/5-surprising-truths-about-the-ai-revolution/#respond</comments>
		
		<dc:creator><![CDATA[Marcin]]></dc:creator>
		<pubDate>Mon, 03 Nov 2025 10:46:51 +0000</pubDate>
				<category><![CDATA[Research diary]]></category>
		<category><![CDATA[agriculture & AI]]></category>
		<category><![CDATA[AI Act]]></category>
		<category><![CDATA[AI agents]]></category>
		<category><![CDATA[AI CO2 emissions]]></category>
		<category><![CDATA[AI economics]]></category>
		<category><![CDATA[AI energy use]]></category>
		<category><![CDATA[AI geopolitics]]></category>
		<category><![CDATA[AI regulation]]></category>
		<category><![CDATA[AI security]]></category>
		<category><![CDATA[Anduril]]></category>
		<category><![CDATA[artificial intelligence]]></category>
		<category><![CDATA[autonomous vehicles]]></category>
		<category><![CDATA[Carbon Robotics]]></category>
		<category><![CDATA[Chatbot Arena]]></category>
		<category><![CDATA[ChatGPT]]></category>
		<category><![CDATA[data centers]]></category>
		<category><![CDATA[DeepSeek]]></category>
		<category><![CDATA[defense & AI]]></category>
		<category><![CDATA[generative AI]]></category>
		<category><![CDATA[inference costs]]></category>
		<category><![CDATA[jailbreaks]]></category>
		<category><![CDATA[Llama 3]]></category>
		<category><![CDATA[LLMs]]></category>
		<category><![CDATA[prompt injection]]></category>
		<category><![CDATA[Responsible AI]]></category>
		<category><![CDATA[technology adoption]]></category>
		<category><![CDATA[Tesla FSD]]></category>
		<category><![CDATA[US vs China]]></category>
		<category><![CDATA[Waymo]]></category>
		<guid isPermaLink="false">https://profind.pl/?p=1895</guid>

					<description><![CDATA[We live in a time when technological change is happening faster than ever. This sense of acceleration isn’t just a subjective impression—it’s a measurable reality. As early as 1999, Vint Cerf, one of the fathers of the internet, observed that one year in the internet industry was like seven “dog years.” That comparison, once apt [&#8230;]]]></description>
										<content:encoded><![CDATA[<p style="text-align: justify;">We live in a time when technological change is happening faster than ever. This sense of acceleration isn’t just a subjective impression—it’s a measurable reality. As early as 1999, Vint Cerf, one of the fathers of the internet, observed that one year in the internet industry was like seven “dog years.” That comparison, once apt at capturing the pace of innovation, now seems insufficient in the context of artificial intelligence. The speed at which AI is reshaping our world is unprecedented—faster than in previous technology waves, including the internet era. The amount of data and analysis on the topic is overwhelming, and media narratives often swing between utopian excitement and dystopian fear. Yet beneath the headlines lie hard numbers that paint a far more nuanced and fascinating picture.</p>
<p style="text-align: justify;">In this article, I present five of the most surprising and counterintuitive findings from the latest analyses. They help explain the true nature of the AI revolution—its unprecedented speed, paradoxical economics, geopolitical tensions, impact on the physical world, and the fundamental shift in the labor market. These are truths worth knowing to navigate the era ahead with intent.</p>
<p><span id="more-1895"></span></p>
<h3>1. “Dog years” is an understatement: The speed of AI adoption defies belief</h3>
<p style="text-align: justify;">The pace at which AI-based tools have been adopted by users worldwide is unprecedented. The scale becomes clear when we compare how long different platforms needed to reach 100 million <em>monthly active</em> users or subscribers:</p>
<ul>
<li><strong>Netflix:</strong> about 10 years to 100M global subscribers (2017)</li>
<li><strong>Facebook:</strong> ~4.5 years to 100M users (2008)</li>
<li><strong>Instagram:</strong> ~28 months to 100M MAU</li>
<li><strong>TikTok:</strong> ~9 months to 100M users</li>
<li><strong>ChatGPT:</strong> ~2 months to 100M MAU</li>
</ul>
<p style="text-align: justify;">Unlike previous waves, generative AI became a global phenomenon from day one. Lightning-fast adoption shortens feedback loops between users and builders, further accelerating innovation and increasing pressure to monetize from the very first day.</p>
<h3>2. AI’s economic paradox: The more powerful it becomes, the stranger its business gets</h3>
<p style="text-align: justify;">The AI business model combines very high upfront investment with rapidly falling unit prices for services:</p>
<ul>
<li style="text-align: justify;"><strong>Training costs are very high:</strong> estimates in model cards and academic publications show that emissions associated with training the newest, largest models reach thousands of tons of CO₂e (e.g., <code>Llama 3.1 405B</code> ~8,930 tons CO₂e per its model card). For older models such as <code>GPT-3</code>, academic papers report on the order of several hundred tons of CO₂e.</li>
<li style="text-align: justify;"><strong>Inference costs are dropping precipitously:</strong> since late 2022, prices for executing queries of comparable quality have fallen by more than <strong>280×</strong>, dramatically lowering barriers to entry for builders and end users.</li>
</ul>
<p style="text-align: justify;">The result? AI is becoming cheaper and more accessible for developers and users, yet monetization for model creators is hard—they incur multibillion-dollar investments in technologies whose unit usage costs quickly fall.</p>
<blockquote><p>It’s different this time, we’ll make it up on volume, and we’ll figure out how to monetize our users in the future are typically three of the biggest danger statements in business.</p></blockquote>
<p style="text-align: justify;"><em>This is an editorial comment on business-model risks, not a conclusion drawn from the reports.</em></p>
<h4>Hardware and accelerator performance</h4>
<p style="text-align: justify;">Inference cost declines are driven by scale, rising accelerator performance, increasingly effective model optimizations, and energy efficiency improvements across entire compute stacks.</p>
<h4>Energy, water, and power grids</h4>
<p style="text-align: justify;">The scale of AI investment is forcing infrastructure expansion. Forecasts indicate that global electricity demand from data centers could double around 2026, and many countries are seeing growing pressure on water resources for cooling and on distribution grid build-out.</p>
<h3>3. A new space race with two leaders and an open flank: Geopolitics and AI philosophy</h3>
<p style="text-align: justify;">Andrew Bosworth, CTO of <code>Meta</code>, likened the current AI phase to a “space race.” Today, competition runs on two fronts in parallel:</p>
<ul>
<li style="text-align: justify;"><strong>Geopolitical front (U.S. vs. China):</strong> in 2024, the U.S. accounted for more “breakthrough” models than China (roughly 40 vs. 15), but the performance gap among top systems is shrinking quickly. New players such as <code>DeepSeek</code> are growing rapidly in their domestic market and increasingly assertive globally.</li>
<li style="text-align: justify;"><strong>Philosophical front (closed vs. open-weight):</strong> the quality advantage of closed systems has visibly narrowed—on <code>Chatbot Arena</code> the difference between leaders of both approaches has shrunk to ~<strong>1.7%</strong>.</li>
</ul>
<p style="text-align: justify;">These two fronts are intertwined. The decision to release strong models under an <strong>open-weight</strong> regime (e.g., <code>Llama 3.1</code>) is also strategic—it strengthens the global developer community and makes it harder for single vendors to monopolize the ecosystem.</p>
<h4>Regulation: The AI Act and a global patchwork of law</h4>
<p style="text-align: justify;">The EU has adopted a comprehensive legal framework (the AI Act), which <strong>entered into force on 1 August 2024</strong>. Application is phased: some prohibitions apply sooner, while the full risk requirements come into effect progressively through 2026. Requirements for general-purpose models (GPAI) also enter into force under transitional arrangements.</p>
<h4>Responsible AI and incidents</h4>
<p style="text-align: justify;">In parallel, the number of documented AI-related incidents is rising (errors, reputational harm, privacy violations). Organizations are implementing Responsible AI policies and <em>model governance</em> processes, but there remains a gap between declarations and actual enforcement.</p>
<h4>Technical security: prompt injection, jailbreaks, and cyber</h4>
<p style="text-align: justify;">LLM-based systems are vulnerable to <em>prompt injection</em> and jailbreaks, among other threats. However, emerging best-practice catalogs and standards (e.g., the OWASP Top 10 for LLMs, NIST guidelines) help harden them (filters, sandboxing, output validation, vulnerability testing, defense in depth).</p>
<h4>How we measure AI quality: Evaluations 2.0</h4>
<p style="text-align: justify;">Alongside classic benchmarks, the importance of factuality, robustness, and complex planning/agent tests is growing. At the same time, awareness of test-set contamination is rising, creating a need for credible, dynamic evaluation methods.</p>
<h3>4. AI leaves the cloud: How AI is reshaping our physical world</h3>
<p style="text-align: justify;">The AI revolution isn’t happening in the cloud alone. The deepest change is the transformation of real-world assets—cars, agricultural machinery, defense systems—into intelligent software endpoints.</p>
<ul>
<li style="text-align: justify;"><strong>Autonomous vehicles:</strong> the scale of miles driven is reaching the billions. Tesla’s <code>FSD (supervised)</code> surpassed about <strong>4.5 billion miles</strong> in mid-2025. In robotaxi, <code>Waymo</code> already has a clear share of rides in San Francisco (about <strong>22%</strong> in 2024), and independent analyses rate its safety as comparable to or better than the average human driver.</li>
<li style="text-align: justify;"><strong>Defense:</strong> <code>Anduril</code> is rapidly scaling autonomous defense systems (drones, surveillance, C2). Market estimates and press reports indicate that in 2024 the company doubled revenue to around <strong>$1B</strong>, reflecting rising demand for intelligent technologies in the defense sector.</li>
<li style="text-align: justify;"><strong>Agriculture:</strong> <code>Carbon Robotics</code> uses AI for laser “weeding,” which in practice can significantly reduce herbicide use (case studies cite savings of around <strong>~80–90%</strong> in weeding costs) and improve efficiency.</li>
</ul>
<h4>Safety of autonomy on public roads</h4>
<p style="text-align: justify;">Analyses from pilot markets suggest that “driverless” systems may have lower collision rates than the average driver. Independent reviews classify <code>Waymo</code> as one of the “safest drivers” on available data, though methodology and scope remain subject to debate.</p>
<h4>Agentic AI systems (agents)</h4>
<p style="text-align: justify;">Combining LLMs with planning, tools, and memory enables multi-step tasks in both digital and physical worlds. This opens new business models (process automation, customer support, fleet management) while creating fresh challenges in control, safety, and accountability.</p>
<h4>AI in scientific discovery</h4>
<p style="text-align: justify;">“AI labs” that combine hypothesis generation, experiment design, and data analysis are shortening the discovery cycle in the natural sciences—confirmed by a growing number of peer-reviewed publications and real-world deployments.</p>
<h3>5. It’s not whether AI will take your job—it’s whether you’ll learn to work with it</h3>
<p style="text-align: justify;">Cognitive automation is changing the nature of work. Labor-market data show a rapid rise in demand for AI skills: in 2024, the share of U.S. job postings requiring AI skills reached about <strong>1.7–1.8%</strong> of all listings, and mentions of <em>generative AI</em> skills grew by more than <strong>threefold</strong> year over year.</p>
<blockquote><p>“You’re not going to lose your job to AI, you’re going to lose your job to somebody who uses AI.” — Jensen Huang</p></blockquote>
<p style="text-align: justify;">Companies are adapting their practices. At <code>Shopify</code>, “reflexive use of AI” has become a <strong>baseline expectation</strong> for employees (disclosed in a publicly available internal memo). It’s a signal that working with AI is no longer a plus—it’s a basic competency.</p>
<p style="text-align: justify;">The AI revolution does not so much eliminate the need for work as shift the center of gravity toward skills that amplify human creativity and judgment with computational power. The future belongs to those who treat AI not as a threat, but as the most powerful tool in their arsenal.</p>
<h3>Conclusion</h3>
<p style="text-align: justify;">The facts presented here portray a revolution that is faster, more complex, and broader in scope than any other technological shift in history. Unprecedented global adoption pressures a paradoxical economics—driven by high training costs and falling inference prices. This dynamic intersects with geopolitical and philosophical rivalry for dominance, growing impact on the physical world, and a fundamental transformation of the labor market—key, interlocking pieces of the puzzle.</p>
<p style="text-align: justify;">Understanding these phenomena is essential to move beyond simplistic narratives. AI is neither utopia nor apocalypse—it’s a powerful force already redefining the rules of the game in business, society, and international affairs. The challenge is to understand its dynamics and shape the future intentionally.</p>
<p style="text-align: justify;"><em>If AI is revolutionizing technology, business, and work at such breakneck speed, which fundamental truth about our society will it challenge next?</em></p>
<h3>References and further reading</h3>
<ul>
<li><strong>Artificial Intelligence Index Report 2025</strong> — Stanford HAI:<br />
<a href="https://hai.stanford.edu/ai-index/2025-ai-index-report" target="_blank" rel="noopener">report page</a> · <a href="https://hai.stanford.edu/assets/files/hai_ai_index_report_2025.pdf" target="_blank" rel="noopener">full PDF</a></li>
<li><strong>State of AI Report 2025</strong> — Air Street / State of AI:<br />
<a href="https://www.stateof.ai/" target="_blank" rel="noopener">online version</a> · <a href="https://press.airstreet.com/p/the-state-of-ai-report-2025" target="_blank" rel="noopener">overview / note</a></li>
<li><strong>Trends – Artificial Intelligence (2025)</strong> — Mary Meeker / BOND:<br />
<a href="https://www.bondcap.com/report/tai/" target="_blank" rel="noopener">report page / PDF</a> · <a href="https://www.slideshare.net/slideshow/trends-artificial-intelligence-mary-meeker/279946168" target="_blank" rel="noopener">slides on SlideShare</a></li>
</ul>
]]></content:encoded>
					
					<wfw:commentRss>https://profind.pl/en/2025/11/03/5-surprising-truths-about-the-ai-revolution/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>The Anatomy of a Scientific Review: Surviving the Ultimate Judgment in Computer Science and Medical Sciences</title>
		<link>https://profind.pl/en/2025/10/06/the-anatomy-of-a-scientific-review-surviving-the-ultimate-judgment-in-computer-science-and-medical-sciences/</link>
					<comments>https://profind.pl/en/2025/10/06/the-anatomy-of-a-scientific-review-surviving-the-ultimate-judgment-in-computer-science-and-medical-sciences/#respond</comments>
		
		<dc:creator><![CDATA[Marcin]]></dc:creator>
		<pubDate>Mon, 06 Oct 2025 12:15:47 +0000</pubDate>
				<category><![CDATA[Research diary]]></category>
		<category><![CDATA[AI in peer review]]></category>
		<category><![CDATA[how to survive peer review]]></category>
		<category><![CDATA[machine learning peer review]]></category>
		<category><![CDATA[medical sciences SEO]]></category>
		<category><![CDATA[peer review]]></category>
		<category><![CDATA[peer review in computer science]]></category>
		<category><![CDATA[peer review in computer science and medicine]]></category>
		<category><![CDATA[research ethics]]></category>
		<category><![CDATA[research reproducibility]]></category>
		<category><![CDATA[reviewer's checklist]]></category>
		<category><![CDATA[SEO for science]]></category>
		<guid isPermaLink="false">https://profind.pl/?p=1881</guid>

					<description><![CDATA[You hit “Submit” and then… silence. Months of work, hundreds of edits, one click—and the long wait for a verdict from those enigmatic figures: Reviewer 1, Reviewer 2… It’s one of the most stressful moments in a researcher’s life. But what actually happens on the other side? Peer review is the backbone of science—a quality-control [&#8230;]]]></description>
										<content:encoded><![CDATA[<div class="moj-wpis-naukowy">
<p>You hit “Submit” and then… silence. Months of work, hundreds of edits, one click—and the long wait for a verdict from those enigmatic figures: Reviewer 1, Reviewer 2… It’s one of the most stressful moments in a researcher’s life. But what actually happens on the other side? Peer review is the backbone of science—a quality-control system meant to ensure that published work is <strong>important, original, and rigorous</strong> (<a href="https://senseaboutscience.org/activities/peer-review/" target="_blank" rel="noopener">Sense about Science</a>).</p>
<p><span id="more-1881"></span></p>
<h3>The Reviewer’s Mind Map: What gets checked, and how?</h3>
<p>Across fields, reviewers look for answers to the same fundamental questions—but they use different tools to get there. The table below shows how the same principles of rigor are interpreted in the medical/social sciences versus in computer science and <em>machine learning</em>.</p>
<div class="table-wrapper">
<table>
<thead>
<tr style="background-color: #f2f2f2;">
<th style="padding: 12px; border: 1px solid #ddd; text-align: left;">Assessment Aspect / The Reviewer’s Question</th>
<th style="padding: 12px; border: 1px solid #ddd; text-align: left;">Implementation (Medical / Social Sciences)</th>
<th style="padding: 12px; border: 1px solid #ddd; text-align: left;">Implementation (Computer Science / Machine Learning)</th>
</tr>
</thead>
<tbody>
<tr>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>0. Conflicts of Interest and Expertise</strong></td>
<td style="padding: 8px; border: 1px solid #ddd;">Do I have ties to the authors/institution? Do I understand this clinical intervention and the statistical methodology?</td>
<td style="padding: 8px; border: 1px solid #ddd;">Do I have ties to the authors/lab? Is this my subfield (algorithms, model architecture, theory)?</td>
</tr>
<tr>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>1. Originality and Significance</strong></td>
<td style="padding: 8px; border: 1px solid #ddd;">Does the study address an important clinical/social problem? Does it add something genuinely new relative to existing studies and reviews?</td>
<td style="padding: 8px; border: 1px solid #ddd;">Is the problem significant (theoretically or practically)? Is the method truly novel, or merely incremental? <strong>Empirically:</strong> does it improve the <em>state of the art</em> in a meaningful way?</td>
</tr>
<tr>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>2. Methodological Rigor (Reproducibility)</strong></td>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>Protocol and preregistration:</strong> is the study registered (e.g., OSF)? Is the protocol available and followed?</td>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>Artifacts and code:</strong> are code, data, and the environment available (GitHub/Zenodo)? Is there a <code>README</code> to run the experiments? Do we meet <strong>ACM Artifact Badging</strong> criteria (Available/Evaluated/Results Validated)? <small>(Badges assess <em>artifacts and reproducibility</em>, not the scientific quality of the paper itself.)</small> <a href="https://www.acm.org/publications/policies/artifact-review-and-badging-current" target="_blank" rel="noopener">ACM policy</a> · <a href="https://sigir.org/general-information/acm-sigir-artifact-badging/" target="_blank" rel="noopener">SIGIR note</a></td>
</tr>
<tr>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>3. Methodological Rigor (Correctness)</strong></td>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>Reporting standards:</strong> is the appropriate <strong>EQUATOR Network</strong> checklist used—<a href="https://www.equator-network.org/reporting-guidelines/consort/" target="_blank" rel="noopener">CONSORT 2025</a> for RCTs, <a href="https://www.prisma-statement.org/prisma-2020" target="_blank" rel="noopener">PRISMA 2020</a> for systematic reviews, <a href="https://www.equator-network.org/reporting-guidelines/strobe/" target="_blank" rel="noopener">STROBE</a> for observational studies?</td>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>Technical craft:</strong> correctness of proofs and definitions; fair <em>apples-to-apples</em> comparisons; appropriate metrics; <em>ablation</em> to assess key components.</td>
</tr>
<tr>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>4. Results and Analysis</strong></td>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>Statistics:</strong> correct choice of tests; uncertainty reporting (CIs); multiple-testing corrections.</td>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>Experimental credibility:</strong> standard, public datasets; repeatability (different random seeds); valid significance tests for model comparisons.</td>
</tr>
<tr>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>5. Conclusions and Interpretation</strong></td>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>No overclaiming:</strong> conclusions match the data; no population-wide claims from small samples; a candid discussion of limitations.</td>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>Proportionality:</strong> no “we solve X” if the gain is +0.5%; discuss computational cost, scalability, and practical limits.</td>
</tr>
<tr>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>6. Research Ethics</strong></td>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>Participant protection (ICMJE/COPE):</strong> IRB approval, informed consent, data anonymization. <a href="https://www.icmje.org/recommendations/" target="_blank" rel="noopener">ICMJE</a> · <a href="https://publicationethics.org/core-practices" target="_blank" rel="noopener">COPE</a></td>
<td style="padding: 8px; border: 1px solid #ddd;"><strong>Data and model ethics:</strong> licenses, privacy, bias, potential harms/dual-use; transparency about data/model sources.</td>
</tr>
</tbody>
</table>
</div>
<p style="font-size: .9em; font-style: italic; margin-top: 10px;">Simplification: in interdisciplinary work, reviewers combine both perspectives.</p>
<h3>A journey through your manuscript: three reviewer passes</h3>
<h4>Pass 1: Quick triage (15–30 min) – <em>is this worth reading further?</em></h4>
<ul>
<li><strong>Fit:</strong> is the work original and within the venue’s scope?</li>
<li><strong>Logic:</strong> do conclusions broadly follow from the data?</li>
<li><strong>Red flags:</strong>
<ul>
<li><em>CS/ML:</em> a “new” algorithm is a known method under a new name; non-standard, trivial dataset.</li>
<li><em>Medicine:</em> RCT without IRB approval or without a control group.</li>
<li><em>Universal:</em> conclusions contradict the authors’ own results.</li>
</ul>
</li>
</ul>
<h4>Pass 2: Microscope (2–5 h) – <em>can I trust these results?</em></h4>
<ul>
<li>Med/Soc: point-by-point compliance with <strong>CONSORT 2025</strong>/<strong>PRISMA</strong>/<strong>STROBE</strong>.</li>
<li>CS/ML: review repository and artifacts; fair baselines; correctness of claims.</li>
</ul>
<h4>Pass 3: Narrative (1–2 h) – <em>does the story make sense?</em></h4>
<ul>
<li>Do the promises in the introduction materialize later?</li>
<li>Is there an honest <em>Limitations</em> section?</li>
<li>Are figures/tables readable and properly described?</li>
</ul>
<h3>Thinking like a reviewer: pre-submission self-review checklist</h3>
<p>Before you click “Submit,” do a tough, point-by-point self-review.</p>
<h4>Big picture and contribution</h4>
<ul>
<li><strong>Problem:</strong> do I clearly define what I solve and why it matters?</li>
<li><strong>Originality:</strong> where in the text do I demonstrate novelty vs. prior work?</li>
<li><strong>Impact:</strong> will anyone want to cite this in 2–3 years?</li>
</ul>
<h4>Section by section</h4>
<p><strong>Title:</strong> does it accurately reflect the content (neither too broad nor too narrow)?</p>
<p><strong>Abstract:</strong></p>
<ul>
<li><strong>WHAT?</strong> question/goal; <strong>WHY?</strong> significance; <strong>HOW?</strong> method; <strong>RESULTS:</strong> key numbers; <strong>NOVELTY:</strong> one-sentence summary of what’s new.</li>
</ul>
<p><strong>Introduction:</strong> context → gap → goal → novelety → solution; end with clear <em>contributions</em>.</p>
<p><strong>Methods/Approach:</strong></p>
<ul>
<li><strong>Reproducibility:</strong> could a peer in the field replicate the experiment?</li>
<li><strong>Justification:</strong> why these methodological choices?</li>
<li><strong>Details:</strong> (CS/ML) hyperparameters, architecture, datasets, metrics; (Med/Soc) inclusion/exclusion criteria, randomization, sample size.</li>
<li><strong>Transparency:</strong> are code and data available? If not, provide a solid rationale.</li>
</ul>
<p><strong>Results &amp; Discussion:</strong> clear figures; strong baselines; claims supported by data; a separate paragraph on limitations.</p>
<p><strong>Conclusions:</strong> answer the research question + offer broader perspective and future directions.</p>
<hr />
<div style="padding: 12px; border-left: 4px solid #444; background: #fafafa; margin: 24px 0;">
<p><strong>Box: standards and transparency in peer review</strong><br />
Point readers to <strong>ANSI/NISO Z39.106-2023</strong>—a standard terminology for peer review models (single/double anonymized, openness of reports, etc.). <a href="https://www.niso.org/publications/z39106-2023-peerreview" target="_blank" rel="noopener">NISO</a></p>
</div>
<h3>AI in service of peer review: assistant or usurper?</h3>
<p>Large language models (LLMs) have entered our toolbox. Surveys (e.g., <a href="https://www.sciencedirect.com/science/article/abs/pii/S1566253525004051" target="_blank" rel="noopener">Zhuang et al., 2025</a>) outline both potential and risk. Below is a policy-safe minimal practice set.</p>
<h4>What genuinely helps?</h4>
<ul>
<li><strong>Fast triage and summaries:</strong> initial mapping of content, scoping against the venue.</li>
<li><strong>Formal checklists:</strong> matching study type to the right standard (e.g., RCT → CONSORT 2025).</li>
<li><strong>Sanity-checking consistency:</strong> basic mismatches between text and tables/figures.</li>
</ul>
<h4>Where are the boundaries?</h4>
<ul>
<li><strong>No deep expertise:</strong> models can miss the true contribution or deep flaws.</li>
<li><strong>Hallucinations and phantom citations:</strong> risk of fabricated sources/claims.</li>
<li><strong>Bias and score inflation:</strong> studies report a tendency to over-recommend.</li>
</ul>
<h4>The ugly truth: ethics, confidentiality, and publisher policies</h4>
<p><strong>Manuscript confidentiality is sacred.</strong> Uploading an unpublished paper to a public LLM breaches review confidentiality.</p>
<ul>
<li><strong>Elsevier:</strong> reviewers <em>should not</em> upload manuscripts to genAI tools or use genAI for substantive reviewing. <a href="https://www.elsevier.com/about/policies-and-standards/the-use-of-generative-ai-and-ai-assisted-technologies-in-the-review-process" target="_blank" rel="noopener">Policy</a></li>
<li><strong>Springer Nature:</strong> asks reviewers to <em>not upload</em> manuscripts to genAI; confidentiality first. <a href="https://www.springer.com/gp/editorial-policies/peer-review-policy-process" target="_blank" rel="noopener">Policy</a></li>
<li><strong>IEEE:</strong> <em>prohibits</em> using AI to draft or generate reviews. <a href="https://ieeeauthorcenter.ieee.org/wp-content/uploads/ieee-reviewer-guidelines.pdf" target="_blank" rel="noopener">Reviewer Guidelines</a></li>
<li><strong>ACM:</strong> only permits language assistance on <em>your own review text</em>, and only without breaching confidentiality (prefer privacy-preserving enterprise solutions). <a href="https://www.acm.org/publications/policies/peer-review-faq" target="_blank" rel="noopener">ACM Peer Review FAQ</a></li>
<li><strong>ACL/ARR:</strong> <em>do not</em> share confidential content with third-party tools (practically: <em>do not</em> use popular LLMs for review materials). <a href="https://aclrollingreview.org/reviewerguidelines" target="_blank" rel="noopener">ARR Reviewer Guidelines</a></li>
</ul>
<p><strong>Exception: publisher-provided tools.</strong> Some publishers develop <em>closed</em> internal tools to support integrity and completeness checks (e.g., Frontiers—<a href="https://www.frontiersin.org/news/2020/07/01/artificial-intelligence-peer-review-assistant-aira/" target="_blank" rel="noopener">AIRA</a>). These handle pre-screening and technical checks (plagiarism, figures, language); the <strong>final judgment remains human</strong>. Always verify the current policy for your journal/conference.</p>
<div style="padding: 12px; border-left: 4px solid #f39c12; background: #fffaf3; margin: 24px 0;">
<p><strong>Minimum safety rule:</strong> <em>never</em> upload someone else’s manuscript to a public LLM. If you need language help for <em>your own</em> review text, use privacy-preserving solutions or the publisher’s in-house tools.</p>
</div>
<h3>The art of responding to reviews</h3>
<ul>
<li><strong>Professional tone:</strong> thank the reviewers and the editor.</li>
<li><strong>Point-by-point list:</strong> address <em>every</em> comment separately.</li>
<li><strong>Show the changes:</strong> cite exact locations in the manuscript (page/line/section).</li>
<li><strong>When you disagree:</strong> be polite, provide evidence-based arguments.</li>
</ul>
<hr />
<h3>References and further reading</h3>
<h4>Standards and guidelines</h4>
<ul>
<li><a href="https://www.equator-network.org/reporting-guidelines/consort/" target="_blank" rel="noopener">CONSORT 2025</a> (RCTs)—update with parallel publications in BMJ/JAMA/Lancet/Nat Med/PLOS Med.</li>
<li><a href="https://www.prisma-statement.org/prisma-2020" target="_blank" rel="noopener">PRISMA 2020</a> (systematic reviews) · <a href="https://www.equator-network.org/reporting-guidelines/strobe/" target="_blank" rel="noopener">STROBE</a> (observational studies)</li>
<li><a href="https://publicationethics.org/core-practices" target="_blank" rel="noopener">COPE Core Practices</a> · <a href="https://www.icmje.org/recommendations/" target="_blank" rel="noopener">ICMJE Recommendations</a></li>
<li><a href="https://www.niso.org/publications/z39106-2023-peerreview" target="_blank" rel="noopener">ANSI/NISO Z39.106-2023</a>—peer review terminology standard</li>
</ul>
<h4>Reproducibility and artifacts</h4>
<ul>
<li><a href="https://www.acm.org/publications/policies/artifact-review-and-badging-current" target="_blank" rel="noopener">ACM Artifact Review &amp; Badging</a> · <a href="https://sigir.org/general-information/acm-sigir-artifact-badging/" target="_blank" rel="noopener">SIGIR: what badges mean (not a scientific-quality rating)</a></li>
</ul>
<h4>AI policies (selected)</h4>
<ul>
<li><a href="https://www.elsevier.com/about/policies-and-standards/the-use-of-generative-ai-and-ai-assisted-technologies-in-the-review-process" target="_blank" rel="noopener">Elsevier—AI and peer review</a> · <a href="https://www.springer.com/gp/editorial-policies/peer-review-policy-process" target="_blank" rel="noopener">Springer Nature—confidentiality &amp; AI</a> · <a href="https://ieeeauthorcenter.ieee.org/wp-content/uploads/ieee-reviewer-guidelines.pdf" target="_blank" rel="noopener">IEEE—prohibition on AI-written reviews</a></li>
<li><a href="https://www.acm.org/publications/policies/peer-review-faq" target="_blank" rel="noopener">ACM—Peer Review FAQ</a> · <a href="https://aclrollingreview.org/reviewerguidelines" target="_blank" rel="noopener">ACL/ARR—third-party tool restrictions</a></li>
<li><a href="https://www.frontiersin.org/news/2020/07/01/artificial-intelligence-peer-review-assistant-aira/" target="_blank" rel="noopener">Frontiers AIRA</a>—closed, integrity-supporting pre-screening</li>
</ul>
<h4>Introductions and guides</h4>
<ul>
<li><a href="https://senseaboutscience.org/activities/peer-review-the-nuts-and-bolts-2/" target="_blank" rel="noopener">Peer Review: The Nuts and Bolts</a> (Sense about Science)</li>
<li><a href="https://profind.pl/2025/09/20/anatomia-procesu-badawczego-praktyczny-framework-dla-rzetelnej-nauki/" target="_blank" rel="noopener">Anatomy of the research process: a practical framework for rigorous science</a> (your previous post)</li>
</ul>
<h4>LLMs in peer review—surveys and studies</h4>
<ul>
<li>Zhuang, Z. et al. (2025). <a href="https://www.sciencedirect.com/science/article/abs/pii/S1566253525004051" target="_blank" rel="noopener">Large language models for automated scholarly paper review: A survey</a>, <em>Information Fusion</em>.</li>
</ul>
</div>
]]></content:encoded>
					
					<wfw:commentRss>https://profind.pl/en/2025/10/06/the-anatomy-of-a-scientific-review-surviving-the-ultimate-judgment-in-computer-science-and-medical-sciences/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>A Researcher&#8217;s Roadmap: A Practical Framework for Rigorous Science</title>
		<link>https://profind.pl/en/2025/09/20/a-researchers-roadmap-a-practical-framework-for-rigorous-science/</link>
					<comments>https://profind.pl/en/2025/09/20/a-researchers-roadmap-a-practical-framework-for-rigorous-science/#respond</comments>
		
		<dc:creator><![CDATA[Marcin]]></dc:creator>
		<pubDate>Sat, 20 Sep 2025 18:11:49 +0000</pubDate>
				<category><![CDATA[Research diary]]></category>
		<category><![CDATA[academic publishing]]></category>
		<category><![CDATA[academic research]]></category>
		<category><![CDATA[AI in research]]></category>
		<category><![CDATA[ChatGPT for research]]></category>
		<category><![CDATA[literature review]]></category>
		<category><![CDATA[LLMs in science]]></category>
		<category><![CDATA[meta-analysis]]></category>
		<category><![CDATA[NLP]]></category>
		<category><![CDATA[open science]]></category>
		<category><![CDATA[pre-registration]]></category>
		<category><![CDATA[replicability]]></category>
		<category><![CDATA[reproducibility]]></category>
		<category><![CDATA[research design]]></category>
		<category><![CDATA[research ethics]]></category>
		<category><![CDATA[research framework]]></category>
		<category><![CDATA[research methodology]]></category>
		<category><![CDATA[research process]]></category>
		<category><![CDATA[rigorous science]]></category>
		<category><![CDATA[scientific method]]></category>
		<category><![CDATA[scientific writing]]></category>
		<category><![CDATA[systematic review]]></category>
		<category><![CDATA[threats to validity]]></category>
		<category><![CDATA[transparency in science]]></category>
		<guid isPermaLink="false">https://profind.pl/?p=1839</guid>

					<description><![CDATA[After many years spent in research, the scientific process—from idea to publication—becomes second nature. However, this intuition, though invaluable, deserves to be structured. The desire to describe this workflow stems not only from a need to better understand my own work but also from the desire to create a map that can help others navigate [&#8230;]]]></description>
										<content:encoded><![CDATA[<div class="moj-wpis-naukowy">
<p>After many years spent in research, the scientific process—from idea to publication—becomes second nature. However, this intuition, though invaluable, deserves to be structured. The desire to describe this workflow stems not only from a need to better understand my own work but also from the desire to create a map that can help others navigate this complex terrain.</p>
<p>One inspiration was a humorous but accurate list from the book &#8220;We Have No Idea: A Guide to the Unknown Universe&#8221; by Jorge Cham and Daniel Whiteson:</p>
<blockquote>
<ol>
<li>Organize what you know</li>
<li>Look for patterns</li>
<li>Ask questions</li>
<li>Buy a tweed jacket with elbow patches</li>
</ol>
</blockquote>
<p>However, scientific work is, above all, the art of asking the right questions. It&#8217;s not about &#8220;beating the baseline&#8221; but about understanding a phenomenon. The question &#8220;why?&#8221; is a researcher&#8217;s compass. In turn, understanding often means the ability to reconstruct a mechanism (e.g., by implementing code or a formal proof), although in some areas of mathematics, a complete, verifiable line of reasoning is sufficient.</p>
<p>I have noticed that whether I am writing an empirical paper in Natural Language Processing (NLP) or a systematic review with a meta-analysis, a common skeleton lies beneath the surface. The result of these observations is the working framework below, which attempts to visualize this skeleton.</p>
<p><span id="more-1839"></span></p>
<h3>A Practical Sketch of the Research Process</h3>
<p>The following framework is a map, not a dogma. It shows the key stages and their specifics depending on the type of work, including reporting standards and the cautious use of modern tools like Large Language Models (LLMs).</p>
<div class="table-wrapper">
<table>
<thead>
<tr>
<th>Research Stage (Phase)</th>
<th>General Description and Goal</th>
<th>Empirical Paper (e.g., NLP, Observational Studies)</th>
<th>Theoretical Paper (e.g., Mathematics, Computer Science)</th>
<th>Systematic Review (+ Meta-analysis)</th>
<th>Role and Application of LLMs (always with expert validation)</th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>0. Planning, Ethics, and Pre-registration</strong></td>
<td>Establishing ethical and methodological frameworks <em>before</em> research begins. Ensuring transparency.</td>
<td>Ethics committee (IRB) approval. Pre-registration of the research plan (e.g., OSF). Data Management Plan (DMP) (compliance with GDPR, HIPAA).</td>
<td>Declaration of conflicting interests. Defining rules for using others&#8217; work and proofs.</td>
<td>Protocol registration in a database (e.g., PROSPERO for health; OSF, INPLASY for other fields).</td>
<td>Support in formulating a data management plan. Assistance in identifying potential ethical risks for further analysis.</td>
</tr>
<tr>
<td><strong>1. Conceptualization and Problem Identification</strong></td>
<td>Broad reading, observation, discussions. The goal is to find a significant scientific question or a knowledge gap.</td>
<td>Observing the limitations of existing models, identifying a new, practical problem.</td>
<td>Noticing a gap in existing theory or the possibility of generalizing a proof.</td>
<td>Observing conflicting research results, lack of a clear answer to an important clinical/practical question.</td>
<td><strong>Sparring partner:</strong> Generating ideas, summarizing fields <em>for verification</em>, identifying trends and potential literature gaps.</td>
</tr>
<tr>
<td><strong>2. Formulating a Question and Hypothesis</strong></td>
<td>Focused literature review. The goal is to formulate a precise, verifiable research question.</td>
<td>Formulating a hypothesis. Defining novelty (new method or new problem).</td>
<td>Formulating a specific thesis to be proven or disproven (conjecture).</td>
<td>Formulating a question in PICO/SPIDER format. Defining inclusion/exclusion criteria.</td>
<td><strong>Literature review support:</strong> Suggesting keywords, summarizing abstracts <em>for researcher&#8217;s assessment</em>, thematic grouping of papers.</td>
</tr>
<tr>
<td><strong>3. Designing the Methodology</strong></td>
<td>A detailed plan for answering the research question. Justification for the choice of methods (why these and not others?).</td>
<td><strong>Protocol according to EQUATOR standards:</strong> e.g., STROBE, CONSORT, SPIRIT, TRIPOD. Selection of metrics, baselines.</td>
<td>Defining the proof strategy, necessary lemmas. Considering formal verification (e.g., using proof assistants like Lean/Coq).</td>
<td><strong>Review protocol:</strong> Compliant with PRISMA 2020. Finalizing search strategies (Boolean/MeSH), planning the risk of bias assessment (e.g., RoB 2, ROBINS-I; or PROBAST for predictive models).</td>
<td><strong>Design assistant:</strong> Generating <em>draft</em> code, proposing libraries, assisting in formulating <em>preliminary</em> search strategies.</td>
</tr>
<tr>
<td><strong>4. Implementation and Data Analysis</strong></td>
<td>Protocol implementation. Collecting results and their statistical analysis.</td>
<td>Model training, validation. Pre-defined ablation plan, stability tests. Corrections for multiple comparisons (e.g., Benjamini–Hochberg FDR control).</td>
<td>Working on the proof. Implementing an algorithm to test the theory.</td>
<td>Dual, independent screening with agreement assessment (e.g., Cohen&#8217;s κ coefficient). Data extraction. Synthesis, heterogeneity analysis (I², τ²), publication bias (Egger&#8217;s test), prediction intervals.</td>
<td><strong>Analyst&#8217;s assistant:</strong> Writing and <em>debugging</em> scripts, generating visualizations <em>for verification</em>. <strong>Important:</strong> inclusion/exclusion decisions in a review are always made by a human.</td>
</tr>
<tr>
<td><strong>5. Interpretation and Synthesis</strong></td>
<td>Answering the &#8220;so what?&#8221; question. What do the results mean, what are their implications and limitations?</td>
<td>Analysis of results in the context of the hypothesis. Discussion of limitations, threats to validity (see below), and future research directions.</td>
<td>Understanding the implications of the proof. What new questions does it open?</td>
<td>Qualitative and quantitative synthesis (e.g., using the Hartung–Knapp adjustment), assessment of the certainty of evidence (e.g., GRADE, GRADE-CERQual).</td>
<td><strong>Critical sparring partner:</strong> Proposing alternative explanations for the results, helping to identify weak points in the argumentation.</td>
</tr>
<tr>
<td><strong>6. Communicating Results (Writing)</strong></td>
<td>Structuring the results into a paper according to reporting standards (e.g., IMRaD).</td>
<td>Writing according to the IMRaD structure and relevant standard extensions (e.g., CONSORT-AI, MI-CLAIM). &#8220;Limitations&#8221; section.</td>
<td>Flexible structure (definitions, lemmas, proof, conclusions). Precision and coherence are key.</td>
<td>Writing according to PRISMA 2020 and PRISMA-S standards. Including a PRISMA 2020 flow diagram and checklist.</td>
<td><strong>Writing assistant:</strong> Language and style correction, paraphrasing, bibliography formatting, generating a <em>draft</em> abstract.</td>
</tr>
<tr>
<td><strong>7. Sharing and Archiving</strong></td>
<td>Ensuring replicability and transparency. <em>Science is a conversation</em>.</td>
<td><code>Data/Code Availability Statement</code>. Publication of code, data, and environment (repository + DOI, with an explicit license, e.g., MIT/BSD-3 for code; CC BY 4.0 for data).</td>
<td>Archiving the proof on platforms like arXiv.</td>
<td>Sharing extraction forms, data, and analytical scripts.</td>
<td>Assistance in preparing code documentation (<code>README.md</code>). Archiving prompts (including the system prompt), settings (temperature, top_p, seed), and model version.</td>
</tr>
</tbody>
</table>
</div>
<h3>Key Aspects of Research Rigor</h3>
<p>A framework is one thing, but true rigor lies in the details.</p>
<h4>Threats to the Validity of Conclusions</h4>
<p>Every study is subject to the risk of error. Awareness of these threats is a sign of research maturity. It is always worth asking about four types of validity:</p>
<ul>
<li><strong>Internal:</strong> Is the observed effect really caused by our intervention and not by another factor?</li>
<li><strong>External:</strong> Can the results be generalized to other populations, contexts, or times?</li>
<li><strong>Construct:</strong> Do our measurement tools (metrics, questionnaires) actually measure what we intend to measure?</li>
<li><strong>Statistical:</strong> Are the statistical conclusions correct (e.g., did we have sufficient power, did we handle multiple testing correctly)?</li>
</ul>
<h4>The Role of AI: A Tool, Not an Authority</h4>
<p>Large language models are powerful accelerators, but their use requires discipline. Risks such as hallucinations and perpetuating biases are well-documented (see Bibliography: Weidinger et al., 2021). On my team, LLMs serve as an assistant and a critical sparring partner. They support brainstorming, literature organization, and code drafting, but <strong>every output and suggestion is verified by a human</strong>: through tests, replication, and source review. We treat them as productivity tools, not as oracles.</p>
<blockquote><p><em>Example of an AI usage declaration:</em></p>
<p>A large language model (GPT-4, August 2025 version) was used in the preparation of this manuscript to support the following tasks: (1) stylistic and grammatical correction, (2) generating draft R code for data visualization. All generated code snippets were manually verified, tested, and refactored by the authors. The model was not involved in data analysis, formulating conclusions, or making decisions about the inclusion/exclusion of studies in the systematic review. The prompts used, including the system prompt, and key parameters (temperature=0.5, top_p=1.0) have been archived in the project repository.</p>
</blockquote>
<h3>Checklist and the Culture of Openness</h3>
<p>This framework is a living document. At the end of each project, however, it&#8217;s worth conducting a simple rigor test. The following checklist, though not exhaustive, helps maintain a high standard and aligns with the ongoing shift in scientific culture towards full transparency, manifested in formats like <em>Registered Reports</em>.</p>
<ol>
<li><strong>Transparency:</strong> Is there a pre-registration or public protocol? Where?</li>
<li><strong>Replicability:</strong> Are the code, data, environment, and random seeds shared (repository + DOI, license, instructions)?</li>
<li><strong>Statistical Rigor:</strong> Are metrics reported with measures of uncertainty (e.g., confidence/credibility intervals)? Were sensitivity analyses conducted?</li>
<li><strong>Self-criticism:</strong> Does the paper include an explicit &#8220;Threats to Validity&#8221; and/or &#8220;Limitations&#8221; section?</li>
<li><strong>Tool Accountability:</strong> Is the role of AI and the methods for its verification clearly described?</li>
</ol>
<hr />
<h3>Bibliography and Further Reading</h3>
<ol>
<li>Cham, J., &amp; Whiteson, D. (2017). <em>We Have No Idea: A Guide to the Unknown Universe</em>. Riverhead Books.</li>
<li>Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., &#8230; &amp; Gabriel, I. (2021). <em>Ethical and social risks of harm from Language Models</em>. <em>arXiv</em>. <a href="https://doi.org/10.48550/arXiv.2112.04359" target="_blank" rel="noopener">https://doi.org/10.48550/arXiv.2112.04359</a></li>
<li>Willis, L. D. (2023). Formulating the Research Question and Framing the Hypothesis (FINER, PICO). <a href="https://pubmed.ncbi.nlm.nih.gov/37041024/" target="_blank" rel="noopener">PubMed</a></li>
<li>Mark Xu (2022). How to do theoretical research, a personal perspective. <a href="https://www.lesswrong.com/posts/zbFGRRWPwxBwjwknY/how-to-do-theoretical-research-a-personal-perspective-1" target="_blank" rel="noopener">LessWrong</a></li>
<li>Cory-Wright, R., Cornelio, C., Dash, S., El Khadir, B., … Horesh, L. (2024). Evolving scientific discovery by unifying data and background knowledge with AI Hilbert. <a href="https://www.nature.com/articles/s41467-024-50074-w" target="_blank" rel="noopener">Nature Communications</a></li>
<li>What is the difference between research methods, research methodology and research approach? <a href="https://www.quora.com/What-is-the-difference-between-research-methods-research-methodology-and-research-approach" target="_blank" rel="noopener">Quora (Q&#038;A)</a></li>
<li>Nikola Balić (2025). AI Ate Its Own Tail, and I Learned Something About Writing. <a href="https://www.nibzard.com/ai-ate-its-tail/" target="_blank" rel="noopener">nibzard.com (blog)</a></li>
<li>Robort Gabriel (2025). AI Tools for Productivity: Boost Workflow &amp; Output. <a href="https://dev.to/coding180/ai-tools-for-productivity-boost-workflow-output-3c83" target="_blank" rel="noopener">DEV Community</a></li>
<li>hashcollision (Substack). Merging with AI. <a href="https://hashcollision.substack.com/p/merging-with-ai" target="_blank" rel="noopener">Substack (essay)</a></li>
<li>What are the types of scientific research? <a href="https://poland.cochrane.org/news/teoria" target="_blank" rel="noopener">Cochrane</a></li>
</ol>
<p><em>Note:</em> Treat items from blogs/Q&#038;A as inspirations and commentaries, not as evidentiary sources. It is better to support &#8216;hard&#8217; claims in a scientific text with methodological/peer-reviewed literature.</p>
</div>
]]></content:encoded>
					
					<wfw:commentRss>https://profind.pl/en/2025/09/20/a-researchers-roadmap-a-practical-framework-for-rigorous-science/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>Transformer &#8211; Roadmap</title>
		<link>https://profind.pl/en/2025/08/04/transformer-roadmap/</link>
					<comments>https://profind.pl/en/2025/08/04/transformer-roadmap/#respond</comments>
		
		<dc:creator><![CDATA[Marcin]]></dc:creator>
		<pubDate>Mon, 04 Aug 2025 17:08:41 +0000</pubDate>
				<category><![CDATA[Research diary]]></category>
		<category><![CDATA[large language models]]></category>
		<category><![CDATA[LLMs]]></category>
		<category><![CDATA[transformers]]></category>
		<guid isPermaLink="false">https://profind.pl/?p=1769</guid>

					<description><![CDATA[Introduction Welcome to my postsseries on the inner workings of Transformers, with a focus on their application in large language models (LLMs). Over time, I’ll break down key components—like self-attention, positional encoding, and layer normalization—highlighting how they contribute to the remarkable capabilities of modern LLMs. Each part of this post will explore one element in [&#8230;]]]></description>
										<content:encoded><![CDATA[
<div class="wp-block-cover is-light"><img fetchpriority="high" decoding="async" width="1024" height="571" class="wp-block-cover__image-background wp-image-1793 size-large" alt="" src="https://profind.pl/wp-content/uploads/2025/08/LLM-Mind-Map-LinkedIn-1024x571.jpg" data-object-fit="cover" srcset="https://profind.pl/wp-content/uploads/2025/08/LLM-Mind-Map-LinkedIn-1024x571.jpg 1024w, https://profind.pl/wp-content/uploads/2025/08/LLM-Mind-Map-LinkedIn-300x167.jpg 300w, https://profind.pl/wp-content/uploads/2025/08/LLM-Mind-Map-LinkedIn-768x428.jpg 768w, https://profind.pl/wp-content/uploads/2025/08/LLM-Mind-Map-LinkedIn-1536x856.jpg 1536w, https://profind.pl/wp-content/uploads/2025/08/LLM-Mind-Map-LinkedIn-2048x1142.jpg 2048w, https://profind.pl/wp-content/uploads/2025/08/LLM-Mind-Map-LinkedIn-624x348.jpg 624w" sizes="(max-width: 1024px) 100vw, 1024px" /><span aria-hidden="true" class="wp-block-cover__background has-background-dim" style="background-color:#f7f8f8"></span><div class="wp-block-cover__inner-container is-layout-constrained wp-block-cover-is-layout-constrained">
<p class="has-text-align-center has-large-font-size wp-block-paragraph"></p>
</div></div>



<h2 class="wp-block-heading">Introduction</h2>



<p class="wp-block-paragraph">Welcome to my postsseries on the inner workings of Transformers, with a focus on their application in large language models (LLMs). Over time, I’ll break down key components—like self-attention, positional encoding, and layer normalization—highlighting how they contribute to the remarkable capabilities of modern LLMs. Each part of this post will explore one element in depth, grounded in both theoretical understanding and practical relevance. The goal is to make these complex systems more transparent and accessible, especially for those interested in research, development, or curious exploration. Let’s begin this journey into the architecture behind today’s most powerful AI models.</p>


]]></content:encoded>
					
					<wfw:commentRss>https://profind.pl/en/2025/08/04/transformer-roadmap/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
		<item>
		<title>Unveiling Dual Quality in Product Reviews: An NLP-Based Approach</title>
		<link>https://profind.pl/en/2025/06/04/unveiling-dual-quality-in-product-reviews-an-nlp-based-approach/</link>
					<comments>https://profind.pl/en/2025/06/04/unveiling-dual-quality-in-product-reviews-an-nlp-based-approach/#respond</comments>
		
		<dc:creator><![CDATA[Marcin]]></dc:creator>
		<pubDate>Wed, 04 Jun 2025 13:08:55 +0000</pubDate>
				<category><![CDATA[Articles]]></category>
		<category><![CDATA[AI for fairness]]></category>
		<category><![CDATA[consumer protection]]></category>
		<category><![CDATA[dual quality]]></category>
		<category><![CDATA[multilingual models]]></category>
		<category><![CDATA[NLP]]></category>
		<category><![CDATA[product reviews]]></category>
		<category><![CDATA[sentiment analysis]]></category>
		<guid isPermaLink="false">https://profind.pl/?p=1785</guid>

					<description><![CDATA[Abstract Consumers often face inconsistent product quality, particularly when identical products vary between markets, a situation known as the dual quality problem. To identify and address this issue, automated techniques are needed. This paper explores how natural language processing (NLP) can aid in detecting such discrepancies and presents the full process of developing a solution. [&#8230;]]]></description>
										<content:encoded><![CDATA[
<h2 class="wp-block-heading">Abstract</h2>


<p style="text-align: justify;">Consumers often face inconsistent product quality, particularly when identical products vary between markets, a situation known as the dual quality problem. To identify and address this issue, automated techniques are needed. This paper explores how natural language processing (NLP) can aid in detecting such discrepancies and presents the full process of developing a solution. First, we describe in detail the creation of a new Polish-language dataset with 1,957 reviews, 540 highlighting dual quality issues. We then discuss experiments with various approaches like SetFit with sentence-transformers, transformer-based encoders, and LLMs, including error analysis and robustness verification. Additionally, we evaluate multilingual transfer using a subset of opinions in English, French, and German. The paper concludes with insights on deployment and practical applications.</p>

]]></content:encoded>
					
					<wfw:commentRss>https://profind.pl/en/2025/06/04/unveiling-dual-quality-in-product-reviews-an-nlp-based-approach/feed/</wfw:commentRss>
			<slash:comments>0</slash:comments>
		
		
			</item>
	</channel>
</rss>
