LimiX: Unleashing Structured-Data Modeling Capability for Generalist Intelligence

Pretraining Approach: LimiX utilizes a novel context-conditional masked modeling (CCMM) objective for pretraining. This method treats structured data as a joint distribution over variables and missingness, allowing it to predict masked entries by leveraging dataset-specific contexts. The pretraining corpus is synthetically generated using hierarchical Structural Causal Models (SCMs), ensuring diverse and controllable causal dependencies. This episodic, context-conditional approach enables rapid, training-free adaptation at inference time, a significant departure from traditional per-dataset retraining, making LimiX highly efficient for enterprise deployment.

Performance Metrics: LimiX's performance is rigorously evaluated across 10 large structured-data benchmarks covering diverse regimes of sample size, feature dimensionality, number of classes, categorical-to-numerical ratios, and missingness. Key metrics include ROC AUC, Accuracy, F1-score for classification, and Normalized RMSE, R² for regression. The evaluation demonstrates that LimiX consistently surpasses strong baselines, including gradient-boosting trees, deep tabular networks, recent tabular foundation models, and automated ensembles, often by substantial margins, solidifying its state-of-the-art status.

Model Architecture: LimiX employs a lightweight, scalable transformer-based architecture. It represents structured data as sample-feature embeddings and learns dependencies across both features (columns) and samples (rows). A key innovation is the low-rank discriminative feature encoding (DFE), which explicitly encodes column identities to enhance discriminability and share statistical strength across features, avoiding the parameter inflation typical of column-aware attention. This asymmetric design, with two feature-level attention passes and one sample-level pass, optimizes modeling capacity for heterogeneous tabular schemas.