A growing number of applications in particle physics and beyond use neural networks as unbinned likelihood ratio estimators applied to real or simulated data. Precision requirements on the inference tasks demand a high-level of stability from these networks, which are affected by the stochastic nature of training. We show how physics concepts can be used to stabilize network training through a physics-inspired optimizer. In particular, the energy conserving descent (ECD) optimization framework uses classical Hamiltonian dynamics on the space of network parameters to reduce the dependence on the initial conditions while also stabilizing the result near the minimum of the loss function. We develop a version of this optimizer known as <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow><msub><mrow><mi mathvariant="sans-serif">ECD</mi></mrow><mrow><mi>q</mi><mo>=</mo><mn>1</mn></mrow></msub></mrow></math>, which has few free hyperparameters with limited ranges guided by physical reasoning. We apply <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><msub><mrow><mi mathvariant="sans-serif">ECD</mi></mrow><mrow><mi>q</mi><mo>=</mo><mn>1</mn></mrow></msub></math> to representative likelihood-ratio estimation tasks in particle physics and find on average that it out-performs the widely used Adam optimizer. We expect that ECD will be a useful tool for wide array of data-limited problems, where it is computationally expensive to exhaustively optimize hyperparameters and mitigate fluctuations with ensembling.

United States

G. Bruno De Luca

Benjamin Nachman

Eva Silverstein

Henry Zheng

Optimizers for stabilizing likelihood-free inference

Identifying the origin of high-energy hadronic jets (jet tagging) has been a critical benchmark problem for machine learning in particle physics. Jets are ubiquitous at colliders and are complex objects that serve as prototypical examples of collections of particles to be categorized. Over the last decade, machine learning-based classifiers have replaced classical observables as the state of the art in jet tagging. Increasingly complex machine learning models are leading to increasingly more effective tagger performance. Our goal is to address the question of convergence—are we getting close to the fundamental limit on jet tagging or is there still potential for computational, statistical, and physical insights for further improvements? We address this question using state-of-the-art generative models to create a realistic, synthetic dataset with a known jet tagging optimum. Various state-of-the-art taggers are deployed on this dataset, showing that there is a significant gap between their performance and the optimum. Our dataset and software are made public to provide a benchmark task for future developments in jet tagging and other areas of particle physics.

Germany

Joep Geuskens

Nishank Gite

Michael Krämer

Vinicius Mikuni

Alexander Mück

Humberto Reyes-González

Fundamental limit of jet tagging

Monte Carlo simulations are an essential tool in particle physics data analysis. Events are typically generated alongside weights that redistribute the cross section of the simulated process across the phase space. These weights can be negative, and several post hoc methods have been developed to eliminate or mitigate the negative values. All of these methods share the common strategy of approximating the average weight as a function of phase space. We introduce an alternative approach, which, instead of reweighting to the average, refines the initial weights with a scaling transformation, utilizing a phase space-dependent factor. Since this new refinement method does not need to model the full weight distribution, it can be more accurate. High-dimensional and unbinned phase space is processed using neural networks for the refinement method. In addition to the refinement method, we introduce a new resampling protocol, which can be used in conjunction with any weight transformation to not only preserve the average weight but also the statistical uncertainties of the initial distribution. Using both realistic and synthetic examples, we show that the new neural refinement method is able to match or exceed the accuracy of similar weight transformations and that the new resampling protocol is simpler in implementation than previous methods while exhibiting equivalent statistical properties.

Dennis Noll

Neural refinement of sample weights

Modern machine learning has enabled parameter inference from event-level data without the need to first summarize all events with a histogram. All of these unbinned inference methods make use of the fact that the events are statistically independent so that the log likelihood is a sum over events. However, this assumption is not valid for unbinned inference on unfolded data, where the deconvolution process induces a correlation between events. We explore the impact of event correlations on downstream inference tasks in the context of the OmniFold unbinned unfolding method. We find that uncertainties may be significantly underestimated when event correlations are excluded from uncertainty quantification.

Krish Desai

Owen Long

Unbinned inference with correlated events

We present the first study of anti-isolated Upsilon decays to two muons (<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow><mi mathvariant="normal">ϒ</mi><mo stretchy="false">→</mo><msup><mrow><mi>μ</mi></mrow><mrow><mo>+</mo></mrow></msup><msup><mrow><mi>μ</mi></mrow><mrow><mo>−</mo></mrow></msup></mrow></math>) in proton-proton collisions at the Large Hadron Collider. Using a machine learning (ML)-based anomaly detection strategy, we “rediscover” the <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow><mi mathvariant="normal">ϒ</mi></mrow></math> in 13 TeV CMS Open Data from 2016, despite overwhelming anti-isolated backgrounds. We elevate the signal significance to <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow><mn>6.4</mn><mi>σ</mi></mrow></math> using these methods, starting from <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mrow><mn>1.6</mn><mi>σ</mi></mrow></math> using the dimuon mass spectrum alone. Moreover, we demonstrate improved sensitivity from using an ML-based estimate of the multifeature likelihood compared to traditional “cut-and-count” methods. This is the first ever detection of anti-isolated Upsilons, which can be useful in the study of heavy-flavor fragmentation in quantum chromodynamics. Our Letter demonstrates that it is possible and practical to find real signals in experimental collider data using ML-based anomaly detection, and we distill a readily accessible benchmark dataset from the CMS Open Data to facilitate future anomaly detection developments.

Rikab Gambhir

Radha Mastandrea

Jesse Thaler

Isolating Unisolated Upsilons with Anomaly Detection in CMS Open Data

We propose a new machine-learning-based anomaly detection strategy for comparing data with a background-only reference (a form of weak supervision). The sensitivity of previous strategies degrades significantly when the signal is too rare or there are many unhelpful features. Our prior-assisted weak supervision (PAWS) method incorporates information from a class of signal models to significantly enhance the search sensitivity of weakly supervised approaches. As long as the true signal is in the prespecified class, PAWS matches the sensitivity of a dedicated, fully supervised method without specifying the exact parameters ahead of time. On the benchmark LHC Olympics anomaly detection dataset, our mix of semisupervised and weakly supervised learning is able to extend the sensitivity over previous methods by a factor of 10 in cross section. Furthermore, if we add irrelevant (noise) dimensions to the inputs, classical methods degrade by another factor of 10 in cross section while PAWS remains insensitive to noise. This new approach could be applied in a number of scenarios and pushes the frontier of sensitivity between completely model-agnostic approaches and fully model-specific searches.

Chi Lung Cheng

Gup Singh

Incorporating Physical Priors into Weakly Supervised Anomaly Detection

Hadronization models used in event generators are physics-inspired functions with many tunable parameters. Since we do not understand hadronization from first principles, there have been multiple proposals to improve the accuracy of hadronization models by utilizing more flexible parametrizations based on neural networks. These recent proposals have focused on the kinematic properties of hadrons, but a full model must also include particle flavor. In this paper, we show how to build a deep learning-based hadronization model that includes both kinematic (continuous) and flavor (discrete) degrees of freedom. Our approach is based on generative adversarial networks and we show the performance within the context of the cluster hadronization model within the herwig event generator.

Jay Chan

Xiangyang Ju

Poland

Adam Kania

Vishnu Sangli

Andrzej Siodmok

Integrating particle flavor into deep learning models for hadronization

There have been many applications of deep neural networks to detector calibrations and a growing number of studies that propose deep generative models as automated fast detector simulators. We show that these two tasks can be unified by using maximum likelihood estimation (MLE) from conditional generative models for energy regression. Unlike direct regression techniques, the MLE approach is prior independent and non-Gaussian resolutions can be determined from the shape of the likelihood near the maximum. Using an ATLAS-like calorimeter simulation, we demonstrate this concept in the context of calorimeter energy calibration.

Haoxing Du

Austria

Claudius Krause

Ian Pang

David Shih

Unifying simulation and inference with normalizing flows

Foundation models are neural networks that are capable of simultaneously solving many problems. Large language foundation models like ChatGPT have revolutionized many aspects of daily life, but their impact for science is not yet clear. In this paper, we use a new foundation model for hadronic jets to solve three key challenges in collider physics. In particular, we show how experiments can (1) save significant computing power when developing reconstruction algorithms, (2) perform a complete uncertainty quantification for high-dimensional measurements, and (3) search for new physics with model agnostic methods using low-level inputs. In each case, there are significant computational or methodological challenges with current methods that limit the science potential of deep learning algorithms. By solving each problem, we take jet foundation models beyond proof-of-principle studies and into the toolkit of practitioners.

Solving key challenges in collider physics with foundation models

Machine learning has become an essential tool in jet physics. Due to their complex, high-dimensional nature, jets can be explored holistically by neural networks in ways that are not possible manually. However, innovations in all areas of jet physics are proceeding in parallel. We show that specially constructed machine learning models trained for a specific jet classification task can improve the accuracy, precision, or speed of all other jet physics tasks. This is demonstrated by training on a particular multiclass generation and classification task and then using the learned representation for different generation and classification tasks, for datasets with a different (full) detector simulation, for jets from a different collision system (<math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>p</mi><mi>p</mi></math> versus <math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><mi>e</mi><mi>p</mi></math>), for generative models, for likelihood ratio estimation, and for anomaly detection. We consider our omnilearn approach thus as a jet-physics foundation model. It is made publicly available for use in any area where state-of-the-art precision is required for analyses involving jets and their substructure.

Method to simultaneously facilitate all jet physics tasks