AGE-ML: Agentic Architecture for Data-Centric AutoML

The 30th European Conference on Advances in Databases and Information Systems

Mattia Matteini\(^1\), Matteo Magnini\(^2\), Matteo Francia\(^1\), Giovanni Ciatto\(^1\), Andrea Omicini\(^1\)

\(^1\) University of Bologna, \(^2\) University of Luxembourg

Automating Data-Centric AI in the agentic era

Data-Centric AI shifts attention from tuning models to improving the data choices that shape model behavior (Zha et al. 2025).

Which role should agents and generative AI play in this process?

Vibe coding is a reality: Agents are powerful (black box) code generators
However, DC-AI requires domain and quality assumptions to be inspectable, reusable, and traceable
… automation should reduce routine work without hiding critical decisions

What about vibe data analysis?

Help data scientists improve their data-driven tasks by also injecting domain knowledge and constraints
… still, unconstrained generation is hard to trust and can bring to biased results

Idea: augment the generation process with specifications and constraints to make generated pipelines inspectable and accountable.

AutoML, Agents, and Data Scientists

AutoML gives algorithmic search

Automates (only) model selection and hyperparameter optimization (He et al. 2021)
- Classic systems include TPOT, Auto-WEKA, Auto-Sklearn, FLAML (Olson and Moore 2019; Thornton et al. 2013; Feurer et al. 2022; Wang et al. 2022)
- Missing domnain knowledge injection and inspectability

Agentic AI gives generation and adaptation

Uses LLM-backed agents for planning, code generation, validation, and repair
- Single LLM agents can generate pipelines, but they are not constrained and can hallucinate.
- More complex approaches include DS-Agent, MLZero, and AutoML-Agent (Guo et al. 2024; Fang et al. 2025; Trirat et al. 2025)
Missing specifications, validation, and guardrails to stay auditable

Data scientists augment knowledge to drive selection and optimization, how can we keep them in the loop?

End-to-end AutoML (Francia et al. 2023, 2024) and LLM-driven agents (Trirat et al. 2025)
Open to doman-knowledge injection but validation and inspectability remain open problems.

AGE-ML: Specifications Define the Search Space

“Knowing that gender and location are sensitive attributes, predict the dataset outcome and verify the results should not depend on them”

A warm-start standard specification (written in YAML) is provided, including classic ML steps
LLM-powered agents distill and inject additional assumptions and constraints.
- E.g., possible solutions are: feature depletion, data augmentation or pseudonymization, fairness-aware models, or post-hoc metrics.

AGE-ML: Specifications Keep Experts in the Loop

“Wait! I know that this dataset is unbalanced and this might introduce bias. Make sure the solution is robust to this.”

E.g. prefer some algorithms over others, or require mandatory specific preprocessing steps.

AGE-ML: Agents Work Inside the Specification

Planner, executor, and validator agents generate, validate, and run pipelines under the constraints imposed by the specification.

Agents synthesize and adapt (Python) code inside an explicit search space and return a governed ML process.
AGE-ML returns pipelines together with metrics, artifacts, and natural-language description for audit and reuse.

Specification In, Governed Pipelines Out

Agentic architecture

Planner

Specification to admissible pipelines (admissibility as a finite-domain constraint satisfaction problem, Z3 to compute admissible pipelines).

Executor

From pipelines to code, runs, and explanations

Catalog

Collect metrics, artifacts, traces

Evaluator

Return the best model under the budget, but all the process is logged and searchable through the catalog

Validation: Generated Code Must Prove Itself

Generated code must pass the following gates:

Structure validation: verifies that the code is syntactically correct and follows the expected specification.
Semantic validation: implements pipelines that do not violate the specification and conflicting constraints (e.g., feature depletion and post-hoc metrics on depleted features).
Runtime validation: intercept runtime errors (e.g., a feature is breaking typing assumptions).

Failures become feedback for bounded repair attempts.

Data Catalog: Every Decision Becomes Audit Evidence

Logging hierarchy

Every Decision Becomes Audit Evidence

Logging hierarchy

Every Decision Becomes Audit Evidence

Logging hierarchy

What We Learn: agents generate pipelines, specifications keep them accountable.

Specifications define the admissible search space.
- The specification acts as a control surface.
- Data scientists are kept into the loop.
Validation makes generated code prove it implements the intended pipeline.
- Generated code might be wrong for different reasons (e.g., incorrect parameters, logical errors).
- Validation is structured in three layers: structure, semantics, and runtime.
Traceability turns every decision into audit evidence.
- As in CRISP-DM, different solutions can highlight different aspects of the problem.
- Although the best result is returned, the entire process is logged and inspectable.
Results are competitive while keeping the process auditable and inspectable.
- Datasets: 6 (Classification) + 3 (Regression)
- Budget: 30 pipelines, 10 generation attempts per pipeline, 2 hours per run
- Competitors: HAMLET (end-to-end pipeline optimization) / FLAML (AutoML) / Kaggle notebooks (shared knowledge)
- AGE-ML (w/ Gemini 3.1 Flash Lite) beats the competitors on 67% of the datasets

Limitations and Future Work

Should we consider different specifications for different problems/datasets?

One coherent specification is assumed, but specifications evolve over time and must be revised.
A unified specification seems a good solution but could bring to a combinatorial explosion of admissible pipelines.

How do we tune the budget?

LLM generation can fail under strict time and retry budgets.
Still the budget is limited, depending on the task and dataset size, different budgeting is needed.

How can we improve user role?

AGE-ML can return insights as the search progresses, users should steer the exploration while running.

How can we improve user knowledge?

Promising pipelines can be used to suggest new constraints and assumptions to refine the specification.
Suggest specification extensions and refinements after pipeline execution.

Results are promising but user impact is limitedly evaluated.

AB testing to understand the effectiveness of AGE-ML with respect to vibe-coding applications.

Thank You

Specification as Control Surface

pipeline:
  steps:
    normalization:
      candidates:
        min-max:
          params: {feature_range: [[0, 1]]}
    classification:
      mandatory: true
      candidates:
        knn:
          params:
            n_neighbors: [5, 11]
            weights: [uniform, distance]
ordering:
  - sequence: [imputation, normalization, features]
  - {before: features, after: classification}
constraints:
  - if: {step: {classification: knn}}
    require: [normalization, features]
budgets:
  pipelines: 5
  generation_attempts: 10

Feature Comparison

Feature	HAMLET (Francia et al. 2023)	FLAML (Wang et al. 2022)	DS-Agent (Guo et al. 2024)	AutoML-Agent (Trirat et al. 2025)	MLZero (Fang et al. 2025)	AGE-ML
End-to-end pipeline	✅	❌	✅	✅	✅	✅
Domain-knowledge injection	✅	❌	❌	❌	❌	✅
Constrained generation	✅	✅	❌	❌	❌	✅
Pipeline validation	✅	❌	✅	✅	✅	✅
Interpretable pipeline	❌	❌	✅	✅	✅	✅
Zero-data cold start	✅	✅	❌	❌	❌	✅
MLOps oriented	❌	❌	❌	❌	❌	✅

AGE-ML is positioned as the approach that combines end-to-end automation, explicit constraints, pipeline validation, and MLOps-oriented traceability.

Competitive Performance with a Governed Process

Evaluation setup

Datasets: 6 (Classification) + 3 (Regression)
Metrics: balanced accuracy for classification; RMSE for regression
Budget: 30 pipelines, 10 generation attempts per pipeline, 2 hours per run
Baselines: HAMLET for classification (closest approach for domain knowledge injection); Kaggle notebooks for regression

Task	Dataset	Gemini 3.1 FL	Baseline	Gemini 2.5 F
Classification	Wilt	0.983	0.975	0.966
	Texture	0.992	0.997	0.995
	Madelon	0.863	0.875	0.863
	HAR	0.987	0.959	0.981
	Adult	0.822	0.805	0.813
	MNIST_784	0.972	0.972	0.966
Regression	California Housing	67,675	69,838	69,331
	Ames Housing	30,189	29,660.363	31,970
	Auto MPG	2.148	2.508	2.283

References

Fang, Haoyang, Boran Han, Nick Erickson, et al. 2025. “MLZero: A Multi-Agent System for End-to-End Machine Learning Automation.” CoRR abs/2505.13941. https://doi.org/10.48550/ARXIV.2505.13941.

Feurer, Matthias, Katharina Eggensperger, Stefan Falkner, Marius Lindauer, and Frank Hutter. 2022. “Auto-Sklearn 2.0: Hands-Free AutoML via Meta-Learning.” J. Mach. Learn. Res. 23: 261:1–61. https://jmlr.org/papers/v23/21-0992.html.

Francia, Matteo, Joseph Giovanelli, and Matteo Golfarelli. 2024. “AutoClues: Exploring Clustering Pipelines via AutoML and Diversification.” In Advances in Knowledge Discovery and Data Mining - 28th Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2024, Taipei, Taiwan, May 7-10, 2024, Proceedings, Part I, edited by De-Nian Yang, Xing Xie, Vincent S. Tseng, Jian Pei, Jen-Wei Huang, and Jerry Chun-Wei Lin, vol. 14645. Lecture Notes in Computer Science. Springer. https://doi.org/10.1007/978-981-97-2242-6_20.

Francia, Matteo, Joseph Giovanelli, and Giuseppe Pisano. 2023. “HAMLET: A Framework for Human-Centered AutoML via Structured Argumentation.” Future Gener. Comput. Syst. 142: 182–94. https://doi.org/10.1016/J.FUTURE.2022.12.035.

Guo, Siyuan, Cheng Deng, Ying Wen, Hechang Chen, Yi Chang, and Jun Wang. 2024. “DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning.” In Forty-First International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024, edited by Ruslan Salakhutdinov, Zico Kolter, Katherine A. Heller, et al. Proceedings of Machine Learning Research. PMLR / OpenReview.net. https://proceedings.mlr.press/v235/guo24b.html.

He, Xin, Kaiyong Zhao, and Xiaowen Chu. 2021. “AutoML: A Survey of the State-of-the-Art.” Knowl. Based Syst. 212: 106622. https://doi.org/10.1016/J.KNOSYS.2020.106622.

Olson, Randal S., and Jason H. Moore. 2019. “TPOT: A Tree-Based Pipeline Optimization Tool for Automating Machine Learning.” In Automated Machine Learning - Methods, Systems, Challenges, edited by Frank Hutter, Lars Kotthoff, and Joaquin Vanschoren. The Springer Series on Challenges in Machine Learning. Springer. https://doi.org/10.1007/978-3-030-05318-5_8.

Thornton, Chris, Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown. 2013. “Auto-WEKA: Combined Selection and Hyperparameter Optimization of Classification Algorithms.” In The 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2013, Chicago, IL, USA, August 11-14, 2013, edited by Inderjit S. Dhillon, Yehuda Koren, Rayid Ghani, et al. ACM. https://doi.org/10.1145/2487575.2487629.

Trirat, Patara, Wonyong Jeong, and Sung Ju Hwang. 2025. “AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML.” In Forty-Second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025, edited by Aarti Singh, Maryam Fazel, Daniel Hsu, et al. Proceedings of Machine Learning Research. PMLR / OpenReview.net. https://proceedings.mlr.press/v267/trirat25a.html.

Wang, Chi, Qingyun Wu, Xueqing Liu, and Luis Quintanilla. 2022. “Automated Machine Learning & Tuning with FLAML.” In KDD ’22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, August 14 - 18, 2022, edited by Aidong Zhang and Huzefa Rangwala. ACM. https://doi.org/10.1145/3534678.3542636.

Zha, Daochen, Zaid Pervaiz Bhat, Kwei-Herng Lai, et al. 2025. “Data-Centric Artificial Intelligence: A Survey.” ACM Comput. Surv. 57 (5): 129:1–42. https://doi.org/10.1145/3711118.