journal article
LitStream Collection
Home
Itzhak, Nevo; Jaroszewicz, Szymon; Moskovitch, Robert
doi: 10.1007/s10994-025-06756-7pmid: N/A
In real-life data of various domains, such as traffic, meteorology, or healthcare data, events may have varying durations. Moreover, heterogeneous multivariate temporal data may consist of varying samplings, including regular sampling in different frequencies or irregular, as well as events data of different types, having fixed or varying duration. We propose to uniformly represent heterogeneous multivariate temporal data using symbolic time-intervals, from which a model that predicts an occurrence of events early can be learned. We introduce a novel use of time-interval-related patterns (TIRPs), in which patterns that end with an event of interest can be used to continuously estimate the event’s occurrence probability in real-time. Recently, we introduced a model that allows continuous prediction of the completion of a pattern, which is extended in this work, to also predict the expected completion time. This work focuses on predicting the probability and time occurrence of an event based on multiple different instances of patterns that end with the event, for which we propose and evaluate aggregation functions. A rigorous evaluation was conducted on four real-life datasets to assess the effectiveness of the proposed model and the aggregation functions. The proposed model performed better than the baseline models (ResNet, LSTM-FCN, ROCKET, and XGBoost) for all datasets.
He, Hao-Yuan; Liu, Yu; Liu, Ren-Biao; Xie, Zheng; Li, Ming
doi: 10.1007/s10994-024-06668-ypmid: N/A
Label refinement methods are designed to improve the quality of training labels by incorporating model predictions into the original training labels. By adjusting the combination coefficient of the noisy label, the impact of noise is reduced, which in turn makes the training process more robust. However, previous label refinement methods are unable to model instance-dependent noise, which is the most realistic type of noise. To address this limitation, we propose a simple approach, probabilistic instance-dependent label refinement (referred to as π\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\pi$$\end{document}-LR). Inspired by the fact that humans are more likely to make mistakes when annotating confusing instances, we propose to estimate the probability of whether a sample is confusing, which can be useful for modeling noise generation. Our approach exploits this concept by assigning a confusing probability ηi\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\eta _i$$\end{document} to each instance xi\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\varvec{x}_i$$\end{document} from a probabilistic perspective. This provides a clear understanding of how instance-dependent noise affects true labels. Empirical evaluations show that π\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\pi$$\end{document}-LR improves the robustness of the model in the presence of label noise and outperforms all compared methods on both realistic and synthetic label noise, while maintaining high efficiency in time and space.
El Hamri, Mourad; Bennani, Younès; Falih, Issam
doi: 10.1007/s10994-025-06749-6pmid: N/A
Domain adaptation arises as an important problem in statistical learning theory, arising when the data-generating processes differ between the training and test samples, respectively called source and target domains. Recent theoretical advances have demonstrated that the success of domain adaptation algorithms heavily relies on their ability to minimize the divergence between the probability distributions of the source and target domains. However, minimizing this divergence cannot be achieved independently of other key ingredients, such as the source risk or the combined error of the ideal joint hypothesis. The trade-off between these terms is often ensured through algorithmic solutions that remain implicit and are not directly reflected by the theoretical guarantees. To get to the bottom of this issue, we propose in this paper a new theoretical framework for domain adaptation through hierarchical optimal transport. This framework provides more explicit generalization bounds and enables us to consider the natural hierarchical organization of samples in both domains into structures, i.e. classes or clusters. Additionally, we provide a new divergence measure between the source and target domains called Hierarchical Wasserstein distance that indicates under mild assumptions, which structures need to be aligned to achieve successful adaptation.
doi: 10.1007/s10994-025-06753-wpmid: N/A
Multi-relational databases are the basis of most consolidated data collections in science and industry today. Most learning and mining algorithms, however, require data to be represented in a propositional form. While there is a variety of specialized machine learning algorithms that can operate directly on multi-relational data sets, propositionalization algorithms transform multi-relational databases into propositional data sets, thereby allowing the application of traditional machine learning and data mining algorithms without their modification. One prominent propositionalization algorithm is RELAGGS by Krogel and Wrobel, which transforms the data by nested aggregations. We propose a new neural network based algorithm in the spirit of RELAGGS that employs trainable composite aggregate functions instead of the static aggregate functions used in the original approach. In this way, we can jointly train the propositionalization with the prediction model, or, alternatively, use the learned aggegrations as embeddings in other algorithms. We demonstrate the increased predictive performance by comparing N-RELAGGS with RELAGGS and multiple other state-of-the-art algorithms.
doi: 10.1007/s10994-025-06755-8pmid: N/A
Most machine learning algorithms tend to bias towards the majority class when a dataset exhibits a skewed distribution in the class variable. This is called the class imbalance problem and is frequently encountered in real-life applications. One of the most prevalent methods for addressing class imbalance is data resampling, which generates or removes samples to balance the dataset. A well-known issue with oversampling is noise generation. Noise removal or hybrid resampling is used to deal with noise. However, these methods cause imbalance to re-emerge. In this study, a data relocation approach named DatRel is proposed to address the noise generation problem of oversampling without causing imbalance. The proposed approach utilizes pure and proper class cover catch digraphs (P-CCCD) to determine dominant points and cover areas for minority class. Then, new samples from oversampling are drawn to the dominant points until they are covered. This process ensures that newly generated samples never overlap with a negative sample. Imbalance is not affected since no sample is removed by undersampling. The proposed DatRel approach is applied to commonly used oversampling methods, namely SMOTE, ADASYN, and BLSMOTE. Moreover, the performance of the DatRel approach is compared to noise filtering methods such as Tomeklink, ENN, NEATER, and NearMiss after SMOTE. Several baseline classification algorithms are employed, and comparisons are made using various metrics. Results using 49 imbalanced datasets show that DatRel improves classifier performance in oversampling methods and demonstrates its value in comparison to other noise removal techniques according to AUC, BACC, F1, GMEAN, and MCC.
Liu, Bingyan; Wu, Li; Wang, Xiaoying; Huang, Jianqiang; Zhang, Guojing
doi: 10.1007/s10994-025-06758-5pmid: N/A
Time series forecasting utilizes historical data to forecast future information over a specific period. It aims to predict forthcoming developmental trends through meticulous statistical analysis and modeling of historical data, addressing real-life challenges like power load prediction, traffic condition prognostication, and extreme weather warnings. Currently, Transformer-based models for time series prediction normally segment the original time series into multiple patches. While this modeling methodology has demonstrated superiority in improving performance, the approach of patch partition based on a fixed length constrains the model’s predictive accuracy when dealing with time series forecasting tasks of varying lengths. To overcome the limitation, this article proposes an innovative Cross-Patch Aggregated Transformer (CPAT), which introduces the Patch Reconstruction module to restructure patches between encoder layers, facilitating cross-patch connections and information interaction. This empowers the model to focus on the correlation among adjacent patches, acquiring effective representations of both global and local features. Consequently, the modeling of time dependency becomes more precise. Extensive experiments conducted on eight publicly available benchmark datasets in real-world scenarios showcase that the proposed CPAT model attains state-of-the-art (SOTA) accuracy overall compared to existing baseline models. Notably, it achieves relative improvement rates of 5.46% and 2.56% for Mean Square Error (MSE) and Mean Absolute Error (MAE), respectively, augmenting the predictive capabilities of Transformer family models in time series tasks.
Alvarado-Pérez, Juan Carlos; Garcia, Miguel Angel; Puig, Domènec
doi: 10.1007/s10994-024-06715-8pmid: N/A
Analyzing large volumes of high-dimensional data poses significant challenges. Dimensionality reduction aims to reveal the most prominent properties of data by embedding them into a low-dimensional representation. Spectral dimensionality reduction methods using kernel matrices have been proven to yield optimal results. Online versions of those methods are desirable to incrementally project new data without recomputing the whole embedding from the complete dataset. In addition, integrating different spectral methods may have a synergistic effect. This paper presents an online dimensionality reduction method based on deep neural networks that integrates embeddings optimized by statistical approximation of neighborhoods and induced by different spectral methods through stacking ensemble learning. In particular, the proposed method first applies a self-supervised stage in order to train a set of deep encoders based on the embeddings induced by different spectral methods applied to a given input dataset. Those basis encoders are optimized and then integrated through a metamodel constituted by a fully connected network. A supervised and an unsupervised approach have been designed depending on whether the final aim is to enforce topological preservation or cluster induction. The proposed method has been experimentally validated on well-known image datasets and compared to some of the most relevant dimensionality reduction techniques by using widely-used quality measures.
Seyedsalehi, Shirin; Salamat, Sara; Arabzadeh, Negar; Ebrahimi, Sajad; Zihayat, Morteza; Bagheri, Ebrahim
doi: 10.1007/s10994-024-06664-2pmid: N/A
Recent studies have demonstrated that while neural ranking methods excel in retrieval effectiveness, they also tend to amplify stereotypical biases, especially those related to gender. Current mitigation strategies often focus on adjusting training methods, like adversarial techniques or data balancing, but typically overlook explicit consideration of gender as an attribute. In this paper, we introduce a systematic approach that treats gender as a distinct component within neural ranker representations. Our neural disentanglement method separates content semantics from gender information, enabling the neural ranker to evaluate document relevance based on content alone, without the interference of gender-related information during retrieval. Our extensive experiments demonstrate that: (1) our disentanglement approach matches the effectiveness of baseline models and offers more consistent performance across queries of different gender affiliations; (2) isolating gender within the representations allows the neural ranker to produce an unbiased list of documents, not favoring any specific gender; and (3) the disentangled gender component effectively and concisely captures gender information independently from the semantic content.
Showing 1 to 10 of 14 Articles