From the Research Universities edition of October 13, 2026
Nature Computational Science (Jena/Intel Labs): LLMs fine-tuned for materials property prediction use atom types but not atom positions, and scaling to 70B parameters or 2 million structures does not fix it
Friedrich Schiller University Jena (Jablonka lab) with Intel Labs; published in Nature Computational Science · AI for materials science and scientific machine learning (academic research) · Germany (Jena) and United States (Intel Labs); open benchmark
- Publisher
- Nature Computational Science
- Original publication
- Nature Computational Science, open access, published 12 October 2026; accompanying Research Briefing (s43588-026-01044-3) published 13 October 2026
- Source retrieved
- 2026-10-14
What happened
Using synthetic datasets with tunable contributions from coordinates and categories, plus the MatText benchmarking framework, the authors report that LLMs consistently fail to learn from coordinate information in coordinate–category data while learning category patterns well. The paper says this persists regardless of model size (up to 70 billion parameters), dataset scale (up to 2 million structures) or text representation. An analysis of the MatBench leaderboard across six property tasks shows graph neural networks consistently outperforming language-model approaches (the authors' 'GNN-LM wall'). The authors recommend using LLMs for composition-dominated tasks and geometric architectures for structure-dominated ones. The companion Research Briefing (13 Oct) summarises the result as LLMs learning which atoms are present, not where they are.
Why it matters
University materials, chemistry and AI-for-science groups, and the research computing teams that fund GPU time for them, are being asked to back general-purpose LLMs for scientific prediction. This peer-reviewed result gives a specific reason to match model architecture to the problem and to test LLM approaches against geometric baselines before committing compute.
Evidence and measured results
Opened the open-access Nature Computational Science article via HTTPS (curl; full HTML text extracted and read, including abstract, GNN-LM wall analysis, Discussion, Limitations, data/code availability and affiliations). Also opened the 13 Oct Research Briefing page (s43588-026-01044-3); its body is subscription-only, so only its visible summary was used.
Limitations and uncertainty
Three properties (shear modulus, bulk modulus, perovskite formation energy) from a limited set of benchmarks; only BERT-style and Llama-2 architectures evaluated; scaling study used parameter-efficient fine-tuning up to 70B, so full fine-tuning or pretraining on coordinate-rich corpora is untested; multimodal models with explicit coordinate embeddings were not evaluated; leaderboard comparison is descriptive. One author is from Intel Labs (now Lila Sciences).
Put this evidence to work
Lighthouse Advisory interpretation, grounded in this source. Enriched 2026-10-13; this does not change the original publication date. Each role follows the same checklist so takeaways compare across sources.
Sales
Role takeaway
- Customer problem
- Research groups and computing centres are spending GPU budget on LLM approaches to scientific prediction without evidence they suit the task.
- Stakeholders
- Research computing director
- Materials and chemistry PIs
- AI-for-science institute lead
- Research software engineering lead
- Discovery questions
- Which prediction tasks depend on 3D structure or coordinates?
- Do projects report geometric baselines before LLM fine-tuning?
- How is GPU time allocated to AI-for-science projects?
- Value hypothesis
- An architecture-selection review may avoid spending fine-tuning budgets on models that cannot use structural information; savings are not quantified in the study.
- Potential engagement
- Three-week benchmarking review of one or two AI-for-science projects using the open MatText code.
- Unsupported claims
- LLMs fail on all scientific prediction tasks
- Multimodal or coordinate-aware models share the limitation
- Quantified compute savings
Pre-sales engineering
Role takeaway
- Fit
- Property prediction from crystal structures, molecular geometries or other coordinate–category data.
- Architecture and integration
- Use geometric neural networks as baseline or primary models for structure-dominated tasks and LLMs for composition- or text-dominated tasks, evaluated in the same benchmark harness (MatText, per the paper).
- Prerequisites
- Labelled structure–property dataset
- Geometric baseline model
- GPU time for fine-tuning comparisons
- Constraints
- Validated on three materials properties
- Only BERT and Llama-2 families tested
- Parameter-efficient fine-tuning only
- Multimodal coordinate-embedding models untested
- Security
- Public datasets and MIT-licensed code; confidentiality concerns arise only with proprietary structures.
- Proof of value
- Rerun the coordinate-sensitivity test on an in-house task
- Compare an LLM against a GNN baseline on held-out data
- Test one coordinate-aware or multimodal model the study did not cover
Delivery
Role takeaway
- Work
- Add a model-selection and baseline evaluation step to AI-for-science project plans and allocation requests.
- Dependencies
- Benchmark datasets
- Research software engineering support
- GPU allocation
- Ownership
- Principal investigator with a research software or ML engineer; research computing reviews allocations.
- Skills and adoption
- Train researchers on inductive bias, baselines and coordinate-sensitivity testing.
- Governance checkpoints
- Baseline reported before large fine-tuning allocation
- Model-choice rationale documented
- Results reviewed against held-out data
- Acceptance criteria
- Proposed: every structure-dependent project reports a geometric baseline
- Proposed: coordinate-sensitivity test documented
- Proposed: model-choice rationale recorded in the allocation request
- Risks
- Overgeneralising beyond materials tasks
- Ignoring untested multimodal designs
- Added evaluation time delaying projects
Implementation considerations
Lighthouse Advisory interpretation across the operating dimensions a public-sector buyer must settle before this evidence becomes a design. Each note answers the question under its heading for this specific source.
Architecture and integration
What must connect, and where does the AI sit in the workflow?
Pipelines for structure-dependent prediction should keep geometric models (for example GNNs) as the primary or baseline model and use LLMs where composition or text dominates.
Governance
Who approves, reviews and stays accountable for outcomes?
Research computing allocation reviews could ask applicants to report a geometric baseline before granting large LLM fine-tuning budgets for property prediction.
Security and privacy
What data, permissions and controls need testing?
No personal data; public benchmark datasets and MIT-licensed code.
Accessibility and workforce
Who is affected, and what skills or accommodations follow?
Researchers need enough ML literacy to understand inductive bias and choose architectures, not only prompt or fine-tune LLMs.
Procurement
What should contracts, pricing and exit terms secure?
Supports budgeting for specialised model development and GPU time for GNNs, rather than assuming larger LLMs will close accuracy gaps.
Operating model
Which teams own the service once it runs?
Argues for an architecture-selection step in AI-for-science project design.
Publication history
- 2026-10-13Research Universities · Issue 364 resources
Stable resource ID: jena-intel-llms-blind-to-atom-coordinates-nature-comp-sci-20261012