# Ai2 Blog

Latest updates

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Theorizer comes to Asta

DevFeed: [Theorizer comes to Asta](<https://devfeed.tech/articles/theorizer-comes-to-asta-144176.md>)

Original publisher: [Read original article](<https://allenai.org/blog/theorizer-in-asta>)

Published: 2026-11-19T08:00:00Z

Content type: release

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Tool](<https://devfeed.tech/topics/tool.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [ai2](<https://devfeed.tech/tags/ai2.md>), [brain](<https://devfeed.tech/tags/brain.md>), [cancer](<https://devfeed.tech/tags/cancer.md>), [cancer-research](<https://devfeed.tech/tags/cancer-research.md>), [climate](<https://devfeed.tech/tags/climate.md>), [code](<https://devfeed.tech/tags/code.md>), [crop](<https://devfeed.tech/tags/crop.md>), [data](<https://devfeed.tech/tags/data.md>), [data-management](<https://devfeed.tech/tags/data-management.md>), [data-scientist](<https://devfeed.tech/tags/data-scientist.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [develop](<https://devfeed.tech/tags/develop.md>), [developing](<https://devfeed.tech/tags/developing.md>), [literature](<https://devfeed.tech/tags/literature.md>), [test](<https://devfeed.tech/tags/test.md>)

### AI overview

Theorizer searches scientific papers, extracts findings relevant to a researcher's question, and combines them into proposed explanations linked to their source studies. Researchers can inspect the evidence and test hypotheses; the tool can miss relevant papers or misjudge whether an idea is original, so its claims require review. In a lung cancer investigation, its hypotheses helped focus analysis of tumor and cell-line data, which produced preliminary support for a possible CDK2 dependency that still needs experimental validation. In Peru, a proposed seasonal explanation for implausible dew-point values was ruled out, but led researchers to investigate how the dataset handled areas far from weather stations. The article also describes literature synthesis in neuroscience and generating possible explanations for a historical change in recorded potato varieties, whose cause remains unresolved. Theorizer is available through Asta, with token costs covered up to a limit, and its code is open source for local use and adaptation.

### Source excerpt

Theorizer in Asta connects scientific literature to proposed explanations that researchers can examine and test.

## Ai2 and Taylor Geospatial partner to make geospatial AI easier to compare, run, and build on

DevFeed: [Ai2 and Taylor Geospatial partner to make geospatial AI easier to compare, run, and build on](<https://devfeed.tech/articles/ai2-and-taylor-geospatial-partner-to-make-geospatial-ai-easier-to-compare-run-and-build-on-141078.md>)

Original publisher: [Read original article](<https://allenai.org/blog/taylor-geospatial-partnership>)

Published: 2026-11-18T08:00:00Z

Content type: news

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [geospatial](<https://devfeed.tech/topics/geospatial.md>), [AI Foundation Models](<https://devfeed.tech/topics/ai-foundation-models.md>), [AI Models](<https://devfeed.tech/topics/ai-models.md>), [Digital Public Good](<https://devfeed.tech/topics/digital-public-goods.md>), [test](<https://devfeed.tech/topics/test.md>), [human feedback](<https://devfeed.tech/topics/human-feedback.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [accelerate](<https://devfeed.tech/tags/accelerate.md>), [accessible](<https://devfeed.tech/tags/accessible.md>), [agriculture](<https://devfeed.tech/tags/agriculture.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai2](<https://devfeed.tech/tags/ai2.md>), [applications](<https://devfeed.tech/tags/applications.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [build](<https://devfeed.tech/tags/build.md>), [compare](<https://devfeed.tech/tags/compare.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [geospatial](<https://devfeed.tech/tags/geospatial.md>), [good](<https://devfeed.tech/tags/good.md>), [imagery](<https://devfeed.tech/tags/imagery.md>), [models](<https://devfeed.tech/tags/models.md>), [open](<https://devfeed.tech/tags/open.md>), [partner](<https://devfeed.tech/tags/partner.md>), [public](<https://devfeed.tech/tags/public.md>), [run](<https://devfeed.tech/tags/run.md>), [satellite](<https://devfeed.tech/tags/satellite.md>)

### AI overview

Ai2 and Taylor Geospatial are partnering to provide open resources that help practitioners use and evaluate geospatial AI for conservation, agriculture, and humanitarian work. The effort combines Benchmarks of The World for comparing models, Ai2's OlmoEarth open Earth-observation model family with a no-code platform to fine-tune and run models globally, and an expansion of Fields of The World to improve farm-field boundary precision and add crop type and yield. The partners say datasets, benchmarks, and model weights will be free under open licenses on Source Cooperative and GitHub. Fields of The World already uses a 13-member practitioner advisory group; the partners want similar input to shape shared benchmarks around field needs.

### Source excerpt

Ai2 and Taylor Geospatial are partnering to make geospatial AI more accessible and useful for the public good through open models, shared benchmarks, and improved satellite imagery datasets for applications in conservation, agriculture, and humanitarian work.

## XYBench: Helping with the problem behind the question

DevFeed: [XYBench: Helping with the problem behind the question](<https://devfeed.tech/articles/xybench-helping-with-the-problem-behind-the-question-132645.md>)

Original publisher: [Read original article](<https://allenai.org/blog/xybench>)

Published: 2026-11-13T08:00:00Z

Content type: article

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [Benchmark](<https://devfeed.tech/topics/benchmark.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [about](<https://devfeed.tech/tags/about.md>), [act](<https://devfeed.tech/tags/act.md>), [advice](<https://devfeed.tech/tags/advice.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-assistants](<https://devfeed.tech/tags/ai-assistants.md>), [ai-systems](<https://devfeed.tech/tags/ai-systems.md>), [all](<https://devfeed.tech/tags/all.md>), [also](<https://devfeed.tech/tags/also.md>), [alternatives](<https://devfeed.tech/tags/alternatives.md>), [approach](<https://devfeed.tech/tags/approach.md>), [automated](<https://devfeed.tech/tags/automated.md>), [award](<https://devfeed.tech/tags/award.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [llms](<https://devfeed.tech/tags/llms.md>), [misconceptions](<https://devfeed.tech/tags/misconceptions.md>), [question](<https://devfeed.tech/tags/question.md>), [user](<https://devfeed.tech/tags/user.md>), [with](<https://devfeed.tech/tags/with.md>), [xybench](<https://devfeed.tech/tags/xybench.md>)

### AI overview

XYBench is a benchmark of 8,115 questions testing whether AI assistants notice mistaken assumptions and prioritize users' underlying goals. It separates whether a response offers a solution to the stated request or intended goal, which goal it emphasizes, and whether it redirects away from a misconception. In a single-turn evaluation, five models more often addressed the literal request than the underlying goal; on Stack Overflow, they emphasized the underlying goal in 35-41% of answers, versus the stated request in 47-57%. Giving Sonnet 4.6 the intended goal increased goal-solution mentions from 59% to 67% but reduced misconception identification from 47% to 34%, suggesting that recognizing intent does not ensure acting on it. Models selected goal-focused answers 71-84% of the time in a separate comparison, despite producing them less often. The benchmark currently covers English queries, mainly procedural problems.

### Source excerpt

XYBench is an 8,115-question benchmark testing whether LLMs spot misconceptions in user requests and redirect toward the user's underlying goal.

## ToyBlocks: A controlled world for robot learning and evaluation

DevFeed: [ToyBlocks: A controlled world for robot learning and evaluation](<https://devfeed.tech/articles/toyblocks-a-controlled-world-for-robot-learning-and-evaluation-122104.md>)

Original publisher: [Read original article](<https://allenai.org/blog/toyblocks>)

Published: 2026-11-06T08:00:00Z

Content type: article

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [Robot Manipulation](<https://devfeed.tech/topics/robot-manipulation.md>), [Physical AI](<https://devfeed.tech/topics/physical-ai.md>), [Dataset development](<https://devfeed.tech/topics/dataset-development.md>), [Reinforcement learning](<https://devfeed.tech/topics/reinforcement-learning.md>), [World models](<https://devfeed.tech/topics/world-models.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [Isaac](<https://devfeed.tech/topics/isaac.md>), [image recognition](<https://devfeed.tech/topics/image-recognition.md>), [Fine-tuning](<https://devfeed.tech/topics/fine-tuning.md>), [robot sense of touch](<https://devfeed.tech/topics/robot-sense-of-touch.md>)

Tags: [automated](<https://devfeed.tech/tags/automated.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [framework](<https://devfeed.tech/tags/framework.md>), [generation](<https://devfeed.tech/tags/generation.md>), [learning](<https://devfeed.tech/tags/learning.md>), [open](<https://devfeed.tech/tags/open.md>), [robot](<https://devfeed.tech/tags/robot.md>), [simulation](<https://devfeed.tech/tags/simulation.md>), [tabletop](<https://devfeed.tech/tags/tabletop.md>), [training](<https://devfeed.tech/tags/training.md>), [world](<https://devfeed.tech/tags/world.md>)

### AI overview

ToyBlocks is a controlled tabletop simulation for studying how robot training data and methods affect performance. It automatically generates task demonstrations with reinforcement learning, motion planning, and a task-and-motion planner, and supports evaluation beyond success rates, including speed, force, and task progress. Researchers can vary demonstration mixtures and test held-out tasks, spatial generalization, and long-horizon planning; privileged simulator state helps generate skills, while perception models act from sensor observations. Reported tests include 63% success or correct infeasibility judgments in search-and-spell tasks, and zero-shot transfer of ManiFlow atomic skills with similar success in simulation and reality (80.0% and 81.1%). The hierarchical Molmo 2 + ManiFlow system succeeded in 55.5% of simulated and 50.0% of real episodes, with the authors noting more trials are needed for statistical confidence. The initial release includes 85,336 long-horizon demonstrations, training and evaluation code, simulation assets, and two trained models; integrated RL training is described as a future addition.

### Source excerpt

ToyBlocks is an open simulation framework for studying robot learning through automated task generation, training demonstrations, and evaluation in a controlled tabletop world.

## Ai2 and Google collaborate on a new generation of climate models

DevFeed: [Ai2 and Google collaborate on a new generation of climate models](<https://devfeed.tech/articles/ai2-and-google-collaborate-on-a-new-generation-of-climate-models-118979.md>)

Original publisher: [Read original article](<https://allenai.org/blog/google-climate-partnership>)

Published: 2026-11-04T08:00:00Z

Content type: news

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [Climate / Weather / Ocean Modeling](<https://devfeed.tech/topics/climate-weather-ocean-modeling.md>), [Open Source Models & Datasets](<https://devfeed.tech/topics/open-source-models-datasets.md>), [Google](<https://devfeed.tech/topics/google.md>)

Tags: [ace](<https://devfeed.tech/tags/ace.md>), [ai-modeling](<https://devfeed.tech/tags/ai-modeling.md>), [ai2](<https://devfeed.tech/tags/ai2.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [climate](<https://devfeed.tech/tags/climate.md>), [climate-modeling](<https://devfeed.tech/tags/climate-modeling.md>), [decades](<https://devfeed.tech/tags/decades.md>), [errors](<https://devfeed.tech/tags/errors.md>), [experiments](<https://devfeed.tech/tags/experiments.md>), [frameworks](<https://devfeed.tech/tags/frameworks.md>), [funding](<https://devfeed.tech/tags/funding.md>), [google](<https://devfeed.tech/tags/google.md>), [google-org](<https://devfeed.tech/tags/google-org.md>), [google-research](<https://devfeed.tech/tags/google-research.md>), [ocean](<https://devfeed.tech/tags/ocean.md>), [open-source](<https://devfeed.tech/tags/open-source.md>), [physics](<https://devfeed.tech/tags/physics.md>)

### AI overview

Ai2 plans to extend Google Research's NeuralGCM into NeuralGCM+, an open-source model for long-term climate projections. NeuralGCM combines physical equations for large-scale atmospheric dynamics with AI for local processes such as clouds and rainfall; Ai2 aims to couple it with ocean and sea-ice models to simulate decades and capture ocean heat redistribution. The work is supported by multimillion-dollar, three-year Google.org funding and Google Research technical support. Ai2 will test past regional temperature and rainfall errors against conventional models, improve projections for warmer conditions outside training, and release documented tools for researchers to adapt and extend, with better GPU performance. The approach may help address limits of conventional models' computational cost and of learned models' ability to generalize, but reliable future projections remain a central research challenge. The text distinguishes Google's nearer-term forecast focus from Ai2's years-to-decades climate focus.

### Source excerpt

With funding from Google.org and technical support from Google Research, Ai2 is developing NeuralGCM+, an open-source model designed to combine physics and AI to simulate the atmosphere and ocean together for faster, more reliable long-term climate projections.

## StateBench: Testing how language models track state changes over long documents

DevFeed: [StateBench: Testing how language models track state changes over long documents](<https://devfeed.tech/articles/statebench-testing-how-language-models-track-state-changes-over-long-documents-116367.md>)

Original publisher: [Read original article](<https://allenai.org/blog/statebench>)

Published: 2026-11-03T08:00:00Z

Content type: article

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [llm-reasoning](<https://devfeed.tech/topics/llm-reasoning.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [scaling laws](<https://devfeed.tech/topics/scaling-laws.md>)

Tags: [architectures](<https://devfeed.tech/tags/architectures.md>), [attention](<https://devfeed.tech/tags/attention.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [hybrid](<https://devfeed.tech/tags/hybrid.md>), [language-models](<https://devfeed.tech/tags/language-models.md>), [models](<https://devfeed.tech/tags/models.md>), [recurrent](<https://devfeed.tech/tags/recurrent.md>), [testing](<https://devfeed.tech/tags/testing.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

StateBench compares language models on four levels of state tracking, from retrieving an unchanged value to following dependencies among variables. In tests of pretrained models, most handled simple updates but struggled as inputs grew; frontier models fell to 13% or lower on the longest general finite problems. When trained directly on the tasks, Gated DeltaNet and hybrid models generalized almost perfectly on the first three levels up to 64,000 assignments, while transformers degraded on long sequences and never learned the hardest level. A more flexible Gated DeltaNet variant stayed near perfect on general finite tasks to about ten times its maximum training length. These results distinguish training effects from architectural limits, though the benchmark assumes a fixed number of variables and leaves some capabilities of constrained Gated DeltaNet unresolved.

### Source excerpt

StateBench tests how language models track changing information across long documents at four difficulty levels, revealing where models struggle and how targeted training helps recurrent and hybrid architectures handle longer sequences.

## How Olmo 3 earned the highest ethical rating from Nextcloud

DevFeed: [How Olmo 3 earned the highest ethical rating from Nextcloud](<https://devfeed.tech/articles/how-olmo-3-earned-the-highest-ethical-rating-from-nextcloud-113445.md>)

Original publisher: [Read original article](<https://allenai.org/blog/nextcloud>)

Published: 2026-11-02T08:00:00Z

Content type: news

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [Nextcloud](<https://devfeed.tech/topics/nextcloud.md>), [Local AI](<https://devfeed.tech/topics/local-ai.md>), [Self Hosted AI Tools](<https://devfeed.tech/topics/self-hosted-ai-tools.md>), [Self-hosted](<https://devfeed.tech/topics/self-hosted.md>), [ai-ethics](<https://devfeed.tech/topics/ai-ethics.md>)

Tags: [agencies](<https://devfeed.tech/tags/agencies.md>), [ai](<https://devfeed.tech/tags/ai.md>), [consumer](<https://devfeed.tech/tags/consumer.md>), [local-ai](<https://devfeed.tech/tags/local-ai.md>), [nextcloud](<https://devfeed.tech/tags/nextcloud.md>), [open](<https://devfeed.tech/tags/open.md>), [practical](<https://devfeed.tech/tags/practical.md>), [rating](<https://devfeed.tech/tags/rating.md>), [self-hosted](<https://devfeed.tech/tags/self-hosted.md>)

### AI overview

Nextcloud gave Olmo 3 its highest Ethical AI rating, green, because it meets the platform's requirements for open-source training and inference software, a self-hostable trained model, and public training data. The 7B model also suited Nextcloud's goal of running local AI without assuming users have 16 or 32 GB of GPU memory. Through llm2, administrators can configure Olmo centrally and make it available in Nextcloud Assistant and supported apps, for tasks such as drafting text, summarizing email threads, and querying facts in documents. Nextcloud sees this as a fit for organizations that want to keep data on infrastructure they control. The text notes that larger models require more compute and that desired future capabilities include stronger tool use and multimodal support.

### Source excerpt

Nextcloud chose Olmo 3 for fully open, self-hosted AI, earning its highest Ethical AI rating while making local AI practical on consumer hardware.

## How researchers are adapting olmOCR 2 for historical and archaeological archives

DevFeed: [How researchers are adapting olmOCR 2 for historical and archaeological archives](<https://devfeed.tech/articles/how-researchers-are-adapting-olmocr-2-for-historical-and-archaeological-archives-105925.md>)

Original publisher: [Read original article](<https://allenai.org/blog/olmocr-2-historical-archives>)

Published: 2026-10-28T08:00:00Z

Content type: news

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [AI Models](<https://devfeed.tech/topics/ai-models.md>), [peft](<https://devfeed.tech/topics/peft.md>), [Advanced AI workflows](<https://devfeed.tech/topics/advanced-ai-workflows.md>), [Self Hosted Archiving and Digital Preservation (DP)](<https://devfeed.tech/topics/self-hosted-archiving-and-digital-preservation-dp.md>)

Tags: [adapting](<https://devfeed.tech/tags/adapting.md>), [ai2](<https://devfeed.tech/tags/ai2.md>), [annotations](<https://devfeed.tech/tags/annotations.md>), [apis](<https://devfeed.tech/tags/apis.md>), [archive](<https://devfeed.tech/tags/archive.md>), [austria](<https://devfeed.tech/tags/austria.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benefit](<https://devfeed.tech/tags/benefit.md>), [code](<https://devfeed.tech/tags/code.md>), [collection](<https://devfeed.tech/tags/collection.md>), [compare](<https://devfeed.tech/tags/compare.md>), [complex](<https://devfeed.tech/tags/complex.md>), [computer](<https://devfeed.tech/tags/computer.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [data](<https://devfeed.tech/tags/data.md>), [decades](<https://devfeed.tech/tags/decades.md>), [development](<https://devfeed.tech/tags/development.md>), [libraries](<https://devfeed.tech/tags/libraries.md>), [records](<https://devfeed.tech/tags/records.md>), [transcription](<https://devfeed.tech/tags/transcription.md>)

### AI overview

Researchers adapted open model olmOCR 2 in two archival settings. ArtificeOCR connects locally run models to historian Maurice Casey's Tropy photo library, making thousands of images searchable while preserving existing metadata and keeping entrusted materials on his computer. Casey reports strengths with varied layouts, cursive English, and typescript, but warns that fluent output can hide errors, especially for handwriting in languages less represented in training. The tool supports scan comparison, correction, and review of processing changes; it remains in early development. For archaeological records from Carnuntum, the CENTURIA team evaluated 507 annotated documents and fine-tuned olmOCR 2 with LoRA on 57 examples. On the other 450 records, transcription error dropped from 15.58% to 1.11%. The team aims to extend this approach to an archive of roughly 70,000 pottery records. The examples show local deployment and fine-tuning as options, with expert review important to checking transcriptions.

### Source excerpt

Researchers are adapting olmOCR 2 to make historical photo libraries searchable locally and improve transcription of handwritten archaeological records.

## Can we trust LLMs to judge whether a research idea is novel?

DevFeed: [Can we trust LLMs to judge whether a research idea is novel?](<https://devfeed.tech/articles/can-we-trust-llms-to-judge-whether-a-research-idea-is-novel-103563.md>)

Original publisher: [Read original article](<https://allenai.org/blog/novelty-bench>)

Published: 2026-10-27T08:00:00Z

Content type: release

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [Parallel reasoning](<https://devfeed.tech/topics/parallel-reasoning.md>), [benchmark overfitting machine learning](<https://devfeed.tech/topics/benchmark-overfitting-machine-learning.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [about](<https://devfeed.tech/tags/about.md>), [absolute](<https://devfeed.tech/tags/absolute.md>), [acceptance](<https://devfeed.tech/tags/acceptance.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-generated-research](<https://devfeed.tech/tags/ai-generated-research.md>), [ai-systems](<https://devfeed.tech/tags/ai-systems.md>), [can](<https://devfeed.tech/tags/can.md>), [changes](<https://devfeed.tech/tags/changes.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [idea](<https://devfeed.tech/tags/idea.md>), [ideas](<https://devfeed.tech/tags/ideas.md>), [llms](<https://devfeed.tech/tags/llms.md>), [novel](<https://devfeed.tech/tags/novel.md>), [problems](<https://devfeed.tech/tags/problems.md>), [research](<https://devfeed.tech/tags/research.md>), [setup](<https://devfeed.tech/tags/setup.md>), [small](<https://devfeed.tech/tags/small.md>), [tests](<https://devfeed.tech/tags/tests.md>)

### AI overview

The study finds that LLM novelty judgments are highly sensitive to evaluation setup, especially for AI-generated research ideas. Across six judges, changing only a few prompt words shifted GPT-5.4 accuracy on human-generated idea pairs from 93.5% to 40.9%, reversing verdicts on over half the same pairs. Prompt effects varied by model, and retrieval or extra reasoning did not reliably improve performance. On the tested Human+Generated benchmark, a Claude Opus 4.6 prompt with low reasoning and three-call majority voting reached 0.854 macro-F1 for $2.59, outperforming the tested Idea Novelty Checker (0.642; $30.66) and AI Scientist evaluator (0.772; $82.05). An extra-high-reasoning GPT-5.6 Sol agent using iterative arXiv search scored 0.78 macro-F1 and cost roughly ten times more per decision than simpler setups. The benchmark uses ICLR 2026 peer-review signals for human ideas; AI-generated ideas carry weak labels based on an assumption, so the authors also examine instability independent of label correctness. They present the benchmark and data pipeline as starting tools, while concluding that reliable novelty evaluation remains unresolved.

### Source excerpt

Old Ideas, Novel Problems tests how reliably LLMs judge the novelty of research ideas, finding that small changes to the evaluation setup can produce very different verdicts.

## LitPivot: Developing research ideas through dynamic contextualization within the literature landscape

DevFeed: [LitPivot: Developing research ideas through dynamic contextualization within the literature landscape](<https://devfeed.tech/articles/litpivot-developing-research-ideas-through-dynamic-contextualization-within-the-literature-landscape-98250.md>)

Original publisher: [Read original article](<https://allenai.org/blog/litpivot>)

Published: 2026-10-23T08:00:00Z

Content type: article

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [Business, Industry, Usability, User Experience, User Research, Web Strategy](<https://devfeed.tech/topics/business-industry-usability-user-experience-user-research-web-strategy.md>), [Tool](<https://devfeed.tech/topics/tool.md>), [Claude](<https://devfeed.tech/topics/claude.md>)

Tags: [2026](<https://devfeed.tech/tags/2026.md>), [about](<https://devfeed.tech/tags/about.md>), [agentic](<https://devfeed.tech/tags/agentic.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-chat](<https://devfeed.tech/tags/ai-chat.md>), [ai-tool](<https://devfeed.tech/tags/ai-tool.md>), [ai2](<https://devfeed.tech/tags/ai2.md>), [all](<https://devfeed.tech/tags/all.md>), [also](<https://devfeed.tech/tags/also.md>), [alter](<https://devfeed.tech/tags/alter.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [assessment](<https://devfeed.tech/tags/assessment.md>), [automated](<https://devfeed.tech/tags/automated.md>), [critiques](<https://devfeed.tech/tags/critiques.md>), [developing](<https://devfeed.tech/tags/developing.md>), [helps](<https://devfeed.tech/tags/helps.md>), [ideas](<https://devfeed.tech/tags/ideas.md>), [literature](<https://devfeed.tech/tags/literature.md>), [research](<https://devfeed.tech/tags/research.md>), [revisions](<https://devfeed.tech/tags/revisions.md>)

### AI overview

LitPivot connects research idea revision with the choice of literature used to assess it. Researchers draft a proposal divided into problem, contribution, and evaluation facets; for each, they select relevant papers, review critiques linked to supporting excerpts, and accept, edit, or reject suggested revisions. The literature is reranked as a facet changes, and the system flags other passages that may need updating. In a study with 17 researchers, proposals developed with LitPivot were rated by blinded domain experts as better grounded in relevant literature than proposals made with a text editor and an AI tool for asking questions about selected papers, using the same paper collection. Participants also reported greater understanding of the literature and stronger perceived gains in novelty and usefulness. A follow-up with five researchers found uses for both vetting new ideas and reframing more mature ones; these studies do not establish effects beyond the reported settings.

### Source excerpt

LitPivot helps researchers refine early research ideas through literature-supported critiques and revisions as their proposals evolve.

## A crop-mapping benchmark for the tropics

DevFeed: [A crop-mapping benchmark for the tropics](<https://devfeed.tech/articles/a-crop-mapping-benchmark-for-the-tropics-95956.md>)

Original publisher: [Read original article](<https://allenai.org/blog/planteur>)

Published: 2026-10-22T08:00:00Z

Content type: article

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [Benchmark](<https://devfeed.tech/topics/benchmark.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [Ground truth / benchmark quality](<https://devfeed.tech/topics/ground-truth-benchmark-quality.md>)

Tags: [also](<https://devfeed.tech/tags/also.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [building](<https://devfeed.tech/tags/building.md>), [can](<https://devfeed.tech/tags/can.md>), [challenges](<https://devfeed.tech/tags/challenges.md>), [changes](<https://devfeed.tech/tags/changes.md>), [classes](<https://devfeed.tech/tags/classes.md>), [community](<https://devfeed.tech/tags/community.md>), [complex](<https://devfeed.tech/tags/complex.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [crop](<https://devfeed.tech/tags/crop.md>), [data](<https://devfeed.tech/tags/data.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [develop](<https://devfeed.tech/tags/develop.md>), [geospatial](<https://devfeed.tech/tags/geospatial.md>), [mapping](<https://devfeed.tech/tags/mapping.md>), [models](<https://devfeed.tech/tags/models.md>), [regions](<https://devfeed.tech/tags/regions.md>), [systems](<https://devfeed.tech/tags/systems.md>)

### AI overview

PLANTEUR extends the PASTIS crop-mapping benchmark from metropolitan France to six tropical regions: Réunion, Guadeloupe, Martinique, Guyane, Mayotte, and Saint-Martin. It contains 2,750 aligned Sentinel-2, Sentinel-1, and Landsat-8 time series and parcel-derived labels spanning 4,506 km² and seven crop classes. Baselines cover standard and leave-one-island-out splits, few-shot learning, and training that includes PASTIS samples. In the evaluated methods, scores were substantially lower on PLANTEUR than on PASTIS, and accuracy fell by roughly half when testing on an island absent from training. The benchmark is intended to help evaluate crop mapping across tropical agricultural systems and geographic transfer.

### Source excerpt

PLANTEUR extends the PASTIS crop-mapping benchmark to six tropical regions to help evaluate geospatial models across more diverse agricultural systems.

## OlmoEarth v1.3: State-of-the-art embeddings for the whole planet

DevFeed: [OlmoEarth v1.3: State-of-the-art embeddings for the whole planet](<https://devfeed.tech/articles/olmoearth-v1-3-state-of-the-art-embeddings-for-the-whole-planet-95747.md>)

Original publisher: [Read original article](<https://allenai.org/blog/olmoearth-v1-3>)

Published: 2026-10-22T08:00:00Z

Content type: release

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [Embeddings](<https://devfeed.tech/topics/embeddings.md>), [vector embeddings in AI](<https://devfeed.tech/topics/vector-embeddings-in-ai.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai2](<https://devfeed.tech/tags/ai2.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [analyze](<https://devfeed.tech/tags/analyze.md>), [archive](<https://devfeed.tech/tags/archive.md>), [benchmark](<https://devfeed.tech/tags/benchmark.md>), [browser](<https://devfeed.tech/tags/browser.md>), [carbon](<https://devfeed.tech/tags/carbon.md>), [change](<https://devfeed.tech/tags/change.md>), [code](<https://devfeed.tech/tags/code.md>), [compute](<https://devfeed.tech/tags/compute.md>), [computing](<https://devfeed.tech/tags/computing.md>), [coverage](<https://devfeed.tech/tags/coverage.md>), [crop](<https://devfeed.tech/tags/crop.md>), [data](<https://devfeed.tech/tags/data.md>), [embeddings](<https://devfeed.tech/tags/embeddings.md>), [explorer](<https://devfeed.tech/tags/explorer.md>), [global](<https://devfeed.tech/tags/global.md>), [olmoearth](<https://devfeed.tech/tags/olmoearth.md>), [satellite](<https://devfeed.tech/tags/satellite.md>)

### AI overview

OlmoEarth v1.3 provides reusable 10-meter satellite embeddings for mapping and change analysis, with precomputed coverage shared for community use. It combines annual Sentinel-1 radar, Sentinel-2 and Landsat time series into 128-dimensional pixel vectors; the first 64 dimensions offer a smaller option. Across 27 evaluations, the 128-dimensional embeddings led on 20 and ranked highest overall, while the 64-dimensional version led on 18. Users can compare years without training a model, or train a small classifier from labeled points in the browser-based Embeddings Explorer and export a GeoTIFF. The Explorer needs no account or GPU; OlmoEarth Studio remains available to run the model on chosen dates, modalities and resolution, or to fine-tune. Coverage is global for 2023-2025, while 2017-2022 covers selected countries. Provenance tables link tiles to the satellite scenes used.

### Source excerpt

OlmoEarth v1.3 brings state-of-the-art satellite embeddings, precomputed global data, and a browser-based Embeddings Explorer for mapping and analyzing the planet.

## FairStress: A deeper evaluation of fairness in LLM decision-making

DevFeed: [FairStress: A deeper evaluation of fairness in LLM decision-making](<https://devfeed.tech/articles/fairstress-a-deeper-evaluation-of-fairness-in-llm-decision-making-91166.md>)

Original publisher: [Read original article](<https://allenai.org/blog/fairstress>)

Published: 2026-10-20T08:00:00Z

Content type: article

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [responsible-ai](<https://devfeed.tech/topics/responsible-ai.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [decision-making](<https://devfeed.tech/topics/decision-making.md>), [Data analysis](<https://devfeed.tech/topics/data-analysis.md>), [Chain-of-thought](<https://devfeed.tech/topics/chain-of-thought.md>), [benchmark overfitting machine learning](<https://devfeed.tech/topics/benchmark-overfitting-machine-learning.md>)

Tags: [benchmark](<https://devfeed.tech/tags/benchmark.md>), [decision-making](<https://devfeed.tech/tags/decision-making.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [fairness](<https://devfeed.tech/tags/fairness.md>), [llm](<https://devfeed.tech/tags/llm.md>)

### AI overview

FairStress finds that matching final choices can mask demographic differences in how readily LLMs abandon well-supported decisions, explain them, and represent them internally. Across 15 models, the average gap in abandoning correct answers was 6.9 percentage points with implied identity and 9.3 points with explicit labels; the authors call the pattern anti-stereotypical over-correction. FairStress tests 300 scenarios across 29 domains, varying identity cues and matched persuasive prompts, with a 50K-item Core subset for faster evaluation. The authors caution that the benchmark reflects an equality-centered definition of fairness, not every fairness framework. In comparisons with BBQ, measured bias direction changed with question format and scoring method. Of nine tested debiasing interventions, counterfactual chain-of-thought--reconsidering a case with identities swapped--reduced asymmetry across three model families while largely preserving accuracy, though it did not remove minority-candidate preference in ambiguous cases. Other methods were unreliable, and one steering method sharply reduced accuracy.

### Source excerpt

FairStress is a new benchmark that reveals demographic differences in how LLMs defend and explain their decisions, even when their final choices appear fair.

## HiRO-ACEv2: Simulating extreme weather at 3 km resolution for a fraction of the cost

DevFeed: [HiRO-ACEv2: Simulating extreme weather at 3 km resolution for a fraction of the cost](<https://devfeed.tech/articles/hiro-acev2-simulating-extreme-weather-at-3-km-resolution-for-a-fraction-of-the-cost-87146.md>)

Original publisher: [Read original article](<https://allenai.org/blog/hiro-acev2>)

Published: 2026-10-14T08:00:00Z

Content type: article

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [Climate / Weather / Ocean Modeling](<https://devfeed.tech/topics/climate-weather-ocean-modeling.md>), [LLM observability](<https://devfeed.tech/topics/llm-observability.md>), [AI Factory](<https://devfeed.tech/topics/ai-factory.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [climate](<https://devfeed.tech/tags/climate.md>), [extreme-weather](<https://devfeed.tech/tags/extreme-weather.md>), [inference](<https://devfeed.tech/tags/inference.md>), [open-source-ai](<https://devfeed.tech/tags/open-source-ai.md>), [simulations](<https://devfeed.tech/tags/simulations.md>), [weather](<https://devfeed.tech/tags/weather.md>)

### AI overview

HiRO-ACEv2 combines a coarse-resolution AI climate model with a downscaling model to generate global climate data at 3 km resolution. Its updated model reproduces rare extreme weather events, and the system costs about 1/300th as much to run as X-SHiELD. The article describes current limits and planned work to support other climate conditions and additional outputs.

### Source excerpt

HiRO-ACEv2 captures extreme rain, wind, and pressure events at 3 km resolution for about 1/300th the cost of X-SHiELD, making detailed global climate simulations more accessible to researchers.

## How Molmo 2's open release helped an MIT researcher rethink fish tracking

DevFeed: [How Molmo 2's open release helped an MIT researcher rethink fish tracking](<https://devfeed.tech/articles/how-molmo-2-s-open-release-helped-an-mit-researcher-rethink-fish-tracking-86240.md>)

Original publisher: [Read original article](<https://allenai.org/blog/molmo2fish>)

Published: 2026-10-13T08:00:00Z

Content type: news

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [lunar foundation model](<https://devfeed.tech/topics/lunar-foundation-model.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>), [canary release](<https://devfeed.tech/topics/canary-release.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [computer-vision](<https://devfeed.tech/tags/computer-vision.md>), [model](<https://devfeed.tech/tags/model.md>)

### AI overview

Kai Van Brunt adapted the open Molmo 2 vision-language model into Molmo2Fish, which tracks salmon in sonar video and can correct some errors when given clear language instructions. It performed roughly as well as a specialized fish-tracking system before corrections, with results varying across rivers. The project remains research and has not been deployed.

### Source excerpt

Molmo 2's open release helped MIT researcher Kai Van Brunt develop Molmo2Fish, an experimental model that tracks salmon in sonar footage and corrects some tracking errors using everyday language.

## Ai2 at COLM 2026: Open research, from models to agents for science

DevFeed: [Ai2 at COLM 2026: Open research, from models to agents for science](<https://devfeed.tech/articles/ai2-at-colm-2026-open-research-from-models-to-agents-for-science-84459.md>)

Original publisher: [Read original article](<https://allenai.org/blog/colm-2026-recap>)

Published: 2026-10-09T08:00:00Z

Content type: article

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [Open Source Models & Datasets](<https://devfeed.tech/topics/open-source-models-datasets.md>), [AI for science](<https://devfeed.tech/topics/ai-for-science.md>), [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Cloud and systems](<https://devfeed.tech/topics/cloud-and-systems.md>), [Hacktoberfest](<https://devfeed.tech/topics/hacktoberfest.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [community](<https://devfeed.tech/tags/community.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [models](<https://devfeed.tech/tags/models.md>), [published](<https://devfeed.tech/tags/published.md>), [research](<https://devfeed.tech/tags/research.md>), [training](<https://devfeed.tech/tags/training.md>)

### AI overview

Ai2 recaps its work presented at COLM 2026, including Olmo Hybrid's combination of attention and recurrent layers, training infrastructure for mixture-of-experts models, the Asta scientific-work platform, and the byte-based Bolmo language model. The recap describes open research materials and reports that Olmo Hybrid reached Olmo 3 7B's MMLU accuracy using 49% fewer training tokens.

### Source excerpt

Ai2 joined COLM 2026 to share advances in open language models, training infrastructure, and AI for science, including Olmo Hybrid, Asta, and Bolmo, while connecting with the broader research community.

## Impactful scheduling for GPU clusters

DevFeed: [Impactful scheduling for GPU clusters](<https://devfeed.tech/articles/impactful-scheduling-for-gpu-clusters-84278.md>)

Original publisher: [Read original article](<https://allenai.org/blog/impactful-scheduling>)

Published: 2026-10-09T08:00:00Z

Content type: article

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [scheduling](<https://devfeed.tech/topics/scheduling.md>), [NVLink](<https://devfeed.tech/topics/nvlink.md>)

Tags: [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [clusters](<https://devfeed.tech/tags/clusters.md>), [gpu-clusters](<https://devfeed.tech/tags/gpu-clusters.md>), [scheduler](<https://devfeed.tech/tags/scheduler.md>)

### AI overview

Ai2 describes replacing priority-based GPU scheduling with hierarchical time budgets, fair-share allocation, and time-slicing. The new system maintained 98% cluster occupancy, delivered 98% of owed GPU hours during a 30-day test, and reduced debug-workload p90 queue waits to 30 seconds, while creating challenges for long-running interactive sessions.

### Source excerpt

We explain how Ai2's new GPU scheduler uses time budgets, fair-share allocation, and time-slicing to prioritize high-impact research, shorten queue waits, and keep GPUs busy.

## Now in Nature: Retrofitting language models to operate over bytes

DevFeed: [Now in Nature: Retrofitting language models to operate over bytes](<https://devfeed.tech/articles/now-in-nature-retrofitting-language-models-to-operate-over-bytes-82637.md>)

Original publisher: [Read original article](<https://allenai.org/blog/bolmo-nature>)

Published: 2026-10-07T08:00:00Z

Content type: news

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [LLM Techniques](<https://devfeed.tech/topics/llm-techniques.md>), [Open Source Models & Datasets](<https://devfeed.tech/topics/open-source-models-datasets.md>), [llama](<https://devfeed.tech/topics/llama.md>), [qwen](<https://devfeed.tech/topics/qwen.md>)

Tags: [language-models](<https://devfeed.tech/tags/language-models.md>), [open-source-ai](<https://devfeed.tech/tags/open-source-ai.md>)

### AI overview

Ai2 reports that its Bolmo research has been published in Nature and announces new byte-level model checkpoints. Its byteifying technique converts existing subword models through a relatively short additional training run. Applied to Qwen 3 8B and Llama 3 8B, it produces Bwen 8B and Blama 8B, which reportedly approach their source models' performance. Stage 1 checkpoints also provide a starting point for architecture experiments.

### Source excerpt

The technique behind Bolmo, Ai2's fully open byte-level language models, is now published in Nature, with new checkpoints showing the approach generalizes beyond Olmo to other model families.

## Open-sourcing AstaBrief, the fast report-generation model in Asta

DevFeed: [Open-sourcing AstaBrief, the fast report-generation model in Asta](<https://devfeed.tech/articles/open-sourcing-astabrief-the-fast-report-generation-model-in-asta-82634.md>)

Original publisher: [Read original article](<https://allenai.org/blog/astabrief>)

Published: 2026-10-02T08:00:00Z

Content type: article

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [post-training](<https://devfeed.tech/topics/post-training.md>), [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [Retrieval Augmented Generation (RAG)](<https://devfeed.tech/topics/retrieval-augmented-generation-rag.md>), [quantization](<https://devfeed.tech/topics/quantization.md>)

Tags: [ai-infrastructure](<https://devfeed.tech/tags/ai-infrastructure.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [download](<https://devfeed.tech/tags/download.md>), [model](<https://devfeed.tech/tags/model.md>), [open](<https://devfeed.tech/tags/open.md>)

### AI overview

Ai2 introduces AstaBrief 8B, an open-weights model that generates cited scientific reports from research questions and retrieved literature. The article describes its supervised fine-tuning and preference training, citation-focused data filtering, and evaluation against Asta's Claude-powered pipeline and other systems. In the reported tests, Fast mode averaged 51.1 seconds per report versus 178.5 seconds for Thinking mode; the authors qualify the comparisons as reflecting systems available during development, not today's frontier models.

### Source excerpt

We're releasing AstaBrief, an 8B open-weights model for generating cited scientific reports, available in Asta's Fast mode or to download and run on your own infrastructure.

## Introducing Olmo-core 3: Open, scalable training infrastructure for large MoEs

DevFeed: [Introducing Olmo-core 3: Open, scalable training infrastructure for large MoEs](<https://devfeed.tech/articles/introducing-olmo-core-3-open-scalable-training-infrastructure-for-large-moes-82650.md>)

Original publisher: [Read original article](<https://allenai.org/blog/olmocore3>)

Published: 2026-10-01T08:00:00Z

Content type: release

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [Mixture of Experts (MoE)](<https://devfeed.tech/topics/mixture-of-experts-moe.md>), [AI Infrastructure](<https://devfeed.tech/topics/ai-infrastructure.md>), [scaling laws](<https://devfeed.tech/topics/scaling-laws.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [developers](<https://devfeed.tech/tags/developers.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [mixture-of-experts](<https://devfeed.tech/tags/mixture-of-experts.md>), [scaling](<https://devfeed.tech/tags/scaling.md>)

### AI overview

Olmo-core 3 is an open training framework designed to scale mixture-of-experts language models. It replaces the earlier FSDP-based approach with distributed data parallelism and combines expert and pipeline parallelism, distributed optimization, routing improvements, and MXFP8. Benchmarks report higher throughput in specific tests, including a 1.2-trillion-parameter configuration; those system-performance tests do not measure trained model quality.

### Source excerpt

Olmo-core 3 introduces a redesigned, fully open training stack for efficiently scaling mixture-of-experts models into the trillion-parameter range.

## Crowdsourced Steering Arena reveals weaknesses in Olmo 3 prosocial behavior evaluation

DevFeed: [Crowdsourced Steering Arena reveals weaknesses in Olmo 3 prosocial behavior evaluation](<https://devfeed.tech/articles/what-a-crowdsourced-game-revealed-about-steering-olmo-3-82646.md>)

Original publisher: [Read original article](<https://allenai.org/blog/olmo-arena>)

Published: 2026-09-17T08:00:00Z

Content type: article

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [LLM evaluation / benchmarking](<https://devfeed.tech/topics/llm-evaluation-benchmarking.md>), [rachels-ramblings](<https://devfeed.tech/topics/rachels-ramblings.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [ai-evaluations](<https://devfeed.tech/tags/ai-evaluations.md>), [ai-models](<https://devfeed.tech/tags/ai-models.md>), [ai-research](<https://devfeed.tech/tags/ai-research.md>), [crowdsourcing](<https://devfeed.tech/tags/crowdsourcing.md>), [designers](<https://devfeed.tech/tags/designers.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>), [evaluations](<https://devfeed.tech/tags/evaluations.md>)

### AI overview

A crowdsourced game called Steering Arena invited players to submit text that would steer Olmo 3 toward a prosocial behavior pattern. The top submissions were unreadable token strings, and one participant used automated optimization to search for higher-scoring entries. The project showed how exposing an evaluation metric can turn it into an optimization target, while Olmo 3's open internals helped researchers investigate the model behaviors behind the scores.

### Source excerpt

A crowdsourced game built on Olmo 3 showed how people can exploit unexpected model behaviors to stress-test prosocial AI evaluations--and how open access to a model's internals can help researchers understand why those tests break.

## Teaching future scientists to interrogate AI tools for scientific discovery

DevFeed: [Teaching future scientists to interrogate AI tools for scientific discovery](<https://devfeed.tech/articles/teaching-future-scientists-to-interrogate-ai-tools-for-scientific-discovery-82635.md>)

Original publisher: [Read original article](<https://allenai.org/blog/autodiscovery-student-challenge>)

Published: 2026-09-14T08:00:00Z

Content type: article

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [AI for science](<https://devfeed.tech/topics/ai-for-science.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>)

Tags: [ai](<https://devfeed.tech/tags/ai.md>), [bayesian](<https://devfeed.tech/tags/bayesian.md>), [teaching](<https://devfeed.tech/tags/teaching.md>), [validation](<https://devfeed.tech/tags/validation.md>)

### AI overview

University of Washington students tested AutoDiscovery, an AI system that analyzes scientific datasets, proposes hypotheses, and ranks results by Bayesian surprise. Across projects in battery aging, polymers, proteins, and reactor physics, the tool surfaced some useful leads but performed poorly on synthetic data. Students had to check its evidence, compare findings with research, and assess whether results were sound and worth pursuing.

### Source excerpt

University of Washington students put Ai2's AutoDiscovery to the test, showing how AI can surface promising scientific leads while making human judgment, domain expertise, and rigorous validation more important than ever.

## How Goodfire used Ai2's open post-training stack to trace unwanted model behavior

DevFeed: [How Goodfire used Ai2's open post-training stack to trace unwanted model behavior](<https://devfeed.tech/articles/how-goodfire-used-ai2-s-open-post-training-stack-to-trace-unwanted-model-behavior-82641.md>)

Original publisher: [Read original article](<https://allenai.org/blog/goodfire-olmo>)

Published: 2026-09-09T08:00:00Z

Content type: article

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [post-training](<https://devfeed.tech/topics/post-training.md>), [Open Source](<https://devfeed.tech/topics/open-source.md>)

Tags: [ai-safety](<https://devfeed.tech/tags/ai-safety.md>), [benchmarks](<https://devfeed.tech/tags/benchmarks.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [debugging](<https://devfeed.tech/tags/debugging.md>)

### AI overview

Goodfire used Ai2's open Olmo post-training pipeline to predict behavioral shifts, trace unwanted model behavior to individual preference-training examples, and test targeted changes. The work identified increased compliance with harmful requests as well as an unexpected shift in fan-fiction behavior, while checking whether fixes preserved broader capability gains.

### Source excerpt

Goodfire used Ai2's fully open post-training stack to predict LLM behavioral changes, trace unwanted model behavior back to individual training examples, and test targeted fixes without sacrificing broader capability gains.

## The hard parts of AI-assisted science

DevFeed: [The hard parts of AI-assisted science](<https://devfeed.tech/articles/the-hard-parts-of-ai-assisted-science-82655.md>)

Original publisher: [Read original article](<https://allenai.org/blog/swedish-autodiscovery-recap>)

Published: 2026-09-01T08:00:00Z

Content type: article

Language: en

Sources: [Ai2 Blog](<https://devfeed.tech/sources/ai2-blog.md>)

Topics: [AI Engineering](<https://devfeed.tech/topics/ai-engineering.md>), [Artificial Intelligence](<https://devfeed.tech/topics/ai.md>), [human in the loop agent](<https://devfeed.tech/topics/human-in-the-loop-agent.md>), [human review](<https://devfeed.tech/topics/human-review.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-assisted](<https://devfeed.tech/tags/ai-assisted.md>), [evaluation](<https://devfeed.tech/tags/evaluation.md>)

### AI overview

Researchers at an Ai2 event discussed five challenges for AI-assisted science: preserving human judgment, keeping agents steerable as investigations evolve, deciding which work to delegate, preventing AI from amplifying weak research, and connecting AI more closely with experiments. The article describes needs for adaptable systems that incorporate new evidence and let researchers guide and verify their work.

### Source excerpt

At an Ai2 event marking our expanded collaboration with Providence Swedish, researchers explored the hardest problems in AI-assisted science: keeping systems steerable, grounded in human judgment and sound methods, and responsive to new evidence and experiments.

[Next page](<https://devfeed.tech/sources/ai2-blog.md?cursor=WyIyMDI2LTA5LTAxVDA4OjAwOjAwKzAwOjAwIiwgImRkNDE3NzYxLTIzZTctNGU0Mi04YTkyLWU0OGVkNzliNTQ4ZCJd>)