Foundation models use large datasets to build an effective representation of data that can be deployed on diverse downstream tasks. Previous research developed the omnilearn foundation model for jet physics, using unique properties of particle physics, and showed that it could significantly advance discovery potential across collider experiments. This paper introduces a major upgrade, resulting in the omnilearned framework. This framework has three new elements: (1) updates to the model architecture and training, (2) using over <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow><mn>1</mn><mo>×</mo><msup><mrow><mn>10</mn></mrow><mrow><mn>9</mn></mrow></msup></mrow></math> jets used for training, and (3) providing well-documented software for accessing all datasets and models. We demonstrate omnilearned with three representative tasks: top-quark jet tagging with the community delphes-based benchmark dataset, b tagging with ATLAS full simulation, and anomaly detection with CMS experimental data. In each case, omnilearned is the state of the art, further expanding the discovery potential of past, current, and future collider experiments.

United States

Wahid Bhimji

Chris Harris

Japan

Vinicius Mikuni

Benjamin Nachman

Foundation model framework for all tasks involving jet physics

We extend the particle-flow neural assisted simulations (parnassus) framework of fast simulation and reconstruction to entire collider events. In particular, we use two generative artificial intelligence tools, continuous normalizing flows and diffusion models, to create a set of reconstructed particle-flow objects conditioned on truth-level particles from CMS Open Simulations. While previous work focused on jets, our updated methods now can accommodate all particle-flow objects in an event along with particle-level attributes like particle type and production vertex coordinates. This approach is fully automated, entirely written in Python, and GPU-compatible. Using a variety of physics processes at the LHC, we show that the extended parnassus is able to generalize beyond the training dataset and outperforms the standard, public tool delphes.

Israel

Etienne Dreyer

Eilam Gross

Dmitrii Kobylianskii

Conditional deep generative models for simultaneous simulation and reconstruction of entire events

Unfolding, in the context of high-energy particle physics, refers to the process of removing detector distortions in experimental data. The resulting unfolded measurements are straightforward to use for direct comparisons between experiments and a wide variety of theoretical predictions. For decades, popular unfolding strategies were designed to operate on data formatted as one or more binned histograms. In recent years, new strategies have emerged that use machine learning to unfold datasets in an unbinned manner, allowing for higher-dimensional analyses and more flexibility for current and future users of the unfolded data. This guide comprises recommendations and practical considerations from researchers across a number of major particle physics experiments who have recently put these techniques into practice on real data.

Switzerland

Florencia Canelli

Kyle Cormier

Andrew Cudd

Canada

Dag Gillberg

Roger Huang

Weijie Jin

Sookhyun Lee

Laura Miller

Jingjing Pan

Tanmay Pani

Mariel Pettee

Youqi Song

Fernando Acosta

A practical guide to unbinned unfolding

We introduce Resonant Anomaly Detection with Optimal Transport (RAD-OT), a method for generating signal templates in resonant anomaly detection searches. RAD-OT leverages the fact that the samples from the conditional probability density of the target features vary approximately linearly along the optimal transport path connecting the resonant feature. This does not assume that the conditional density itself is linear with the resonant feature, allowing RAD-OT to efficiently capture multimodal relationships, changes in resolution, etc. By solving the optimal transport problem, RAD-OT can quickly build a template by interpolating between the background distributions in two sideband regions. We demonstrate the performance of RAD-OT using the LHC Olympics R&D dataset, where we find comparable sensitivity and improved stability with respect to deep learning-based approaches.

Matthew Leigh

Debajyoti Sengupta

Tobias Golling

Accelerating template generation in resonant anomaly detection searches with optimal transport

A growing number of applications in particle physics and beyond use neural networks as unbinned likelihood ratio estimators applied to real or simulated data. Precision requirements on the inference tasks demand a high-level of stability from these networks, which are affected by the stochastic nature of training. We show how physics concepts can be used to stabilize network training through a physics-inspired optimizer. In particular, the energy conserving descent (ECD) optimization framework uses classical Hamiltonian dynamics on the space of network parameters to reduce the dependence on the initial conditions while also stabilizing the result near the minimum of the loss function. We develop a version of this optimizer known as <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow><msub><mrow><mi mathvariant="sans-serif">ECD</mi></mrow><mrow><mi>q</mi><mo>=</mo><mn>1</mn></mrow></msub></mrow></math>, which has few free hyperparameters with limited ranges guided by physical reasoning. We apply <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><msub><mrow><mi mathvariant="sans-serif">ECD</mi></mrow><mrow><mi>q</mi><mo>=</mo><mn>1</mn></mrow></msub></math> to representative likelihood-ratio estimation tasks in particle physics and find on average that it out-performs the widely used Adam optimizer. We expect that ECD will be a useful tool for wide array of data-limited problems, where it is computationally expensive to exhaustively optimize hyperparameters and mitigate fluctuations with ensembling.

G. Bruno De Luca

Eva Silverstein

Henry Zheng

Optimizers for stabilizing likelihood-free inference

Identifying the origin of high-energy hadronic jets (jet tagging) has been a critical benchmark problem for machine learning in particle physics. Jets are ubiquitous at colliders and are complex objects that serve as prototypical examples of collections of particles to be categorized. Over the last decade, machine learning-based classifiers have replaced classical observables as the state of the art in jet tagging. Increasingly complex machine learning models are leading to increasingly more effective tagger performance. Our goal is to address the question of convergence—are we getting close to the fundamental limit on jet tagging or is there still potential for computational, statistical, and physical insights for further improvements? We address this question using state-of-the-art generative models to create a realistic, synthetic dataset with a known jet tagging optimum. Various state-of-the-art taggers are deployed on this dataset, showing that there is a significant gap between their performance and the optimum. Our dataset and software are made public to provide a benchmark task for future developments in jet tagging and other areas of particle physics.

Germany

Joep Geuskens

Nishank Gite

Michael Krämer

Alexander Mück

Humberto Reyes-González

Fundamental limit of jet tagging

Monte Carlo simulations are an essential tool in particle physics data analysis. Events are typically generated alongside weights that redistribute the cross section of the simulated process across the phase space. These weights can be negative, and several post hoc methods have been developed to eliminate or mitigate the negative values. All of these methods share the common strategy of approximating the average weight as a function of phase space. We introduce an alternative approach, which, instead of reweighting to the average, refines the initial weights with a scaling transformation, utilizing a phase space-dependent factor. Since this new refinement method does not need to model the full weight distribution, it can be more accurate. High-dimensional and unbinned phase space is processed using neural networks for the refinement method. In addition to the refinement method, we introduce a new resampling protocol, which can be used in conjunction with any weight transformation to not only preserve the average weight but also the statistical uncertainties of the initial distribution. Using both realistic and synthetic examples, we show that the new neural refinement method is able to match or exceed the accuracy of similar weight transformations and that the new resampling protocol is simpler in implementation than previous methods while exhibiting equivalent statistical properties.

Dennis Noll

Neural refinement of sample weights

Modern machine learning has enabled parameter inference from event-level data without the need to first summarize all events with a histogram. All of these unbinned inference methods make use of the fact that the events are statistically independent so that the log likelihood is a sum over events. However, this assumption is not valid for unbinned inference on unfolded data, where the deconvolution process induces a correlation between events. We explore the impact of event correlations on downstream inference tasks in the context of the OmniFold unbinned unfolding method. We find that uncertainties may be significantly underestimated when event correlations are excluded from uncertainty quantification.

Krish Desai

Owen Long

Unbinned inference with correlated events

We present the first study of anti-isolated Upsilon decays to two muons (<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow><mi mathvariant="normal">ϒ</mi><mo stretchy="false">→</mo><msup><mrow><mi>μ</mi></mrow><mrow><mo>+</mo></mrow></msup><msup><mrow><mi>μ</mi></mrow><mrow><mo>−</mo></mrow></msup></mrow></math>) in proton-proton collisions at the Large Hadron Collider. Using a machine learning (ML)-based anomaly detection strategy, we “rediscover” the <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow><mi mathvariant="normal">ϒ</mi></mrow></math> in 13 TeV CMS Open Data from 2016, despite overwhelming anti-isolated backgrounds. We elevate the signal significance to <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow><mn>6.4</mn><mi>σ</mi></mrow></math> using these methods, starting from <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow><mn>1.6</mn><mi>σ</mi></mrow></math> using the dimuon mass spectrum alone. Moreover, we demonstrate improved sensitivity from using an ML-based estimate of the multifeature likelihood compared to traditional “cut-and-count” methods. This is the first ever detection of anti-isolated Upsilons, which can be useful in the study of heavy-flavor fragmentation in quantum chromodynamics. Our Letter demonstrates that it is possible and practical to find real signals in experimental collider data using ML-based anomaly detection, and we distill a readily accessible benchmark dataset from the CMS Open Data to facilitate future anomaly detection developments.

Rikab Gambhir

Radha Mastandrea

Jesse Thaler

Isolating Unisolated Upsilons with Anomaly Detection in CMS Open Data

We propose a new machine-learning-based anomaly detection strategy for comparing data with a background-only reference (a form of weak supervision). The sensitivity of previous strategies degrades significantly when the signal is too rare or there are many unhelpful features. Our prior-assisted weak supervision (PAWS) method incorporates information from a class of signal models to significantly enhance the search sensitivity of weakly supervised approaches. As long as the true signal is in the prespecified class, PAWS matches the sensitivity of a dedicated, fully supervised method without specifying the exact parameters ahead of time. On the benchmark LHC Olympics anomaly detection dataset, our mix of semisupervised and weakly supervised learning is able to extend the sensitivity over previous methods by a factor of 10 in cross section. Furthermore, if we add irrelevant (noise) dimensions to the inputs, classical methods degrade by another factor of 10 in cross section while PAWS remains insensitive to noise. This new approach could be applied in a number of scenarios and pushes the frontier of sensitivity between completely model-agnostic approaches and fully model-specific searches.

Chi Lung Cheng

Gup Singh

Incorporating Physical Priors into Weakly Supervised Anomaly Detection