The vast majority of business data is tabular—it lives in data warehouses, CRMs, and financial ledgers—but creating a reliable model from it still means building a new one from scratch for each database, then maintaining hyperparameter tuning loops, feature engineering, and reengineering pipelines to deal with data drift. Google Research suggests a way around this: A new foundational model called TabFM Instead, it treats table prediction as a learning problem in context.
It can generate predictions for a new, unprecedented schedule in a single forward pass. For enterprise developers and AI engineers, this reduces the time to production from several weeks of pipeline engineering to one API call.
The problem with traditional ML
To extract reliable predictions from a gradient-boosted tree, data scientists must build and maintain complex data pipelines. They must clean up confounding entries, impute missing values, code categorical variables into numerical formats, and plot special feature intercepts.
Once the data is available, they must run iterative hyperparameter optimization loops, searching through learning rates, tree depths, subsampling coefficients, and regularization networks to find the best configuration.
Once these traditional models are in place "incur ongoing operational debt through monitoring data drift and reengineering pipelines to remain accurate," Google Research researcher Weihao Kong told VentureBeat about it.
Meanwhile, the rest of the AI industry has moved on. Generative AI models for text and computer vision have seamlessly transitioned to zero-shot inference, where a model can perform an entirely new task simply by being asked the context.
Large language models (LLM) are already superior learning in contextso why can’t we feed the tables into ready LLM?
Because LLMs are taught natural language rather than structured data, they have difficulty processing tables directly. First, their context constraints are quickly exhausted by medium-sized tables with only a few thousand rows and hundreds of columns. Second, LLMs suffer from tokenization inefficiencies, clumsily splitting numerical values and destroying mathematical precision. Finally, they suffer from structural blindness. When a 2D table is serialized as a 1D text string, LLMs don’t keep track of which value belongs to which row and column as the table grows.
"Therefore, today it is more efficient to use LLM to write code that does feature engineering and calls XGBoost rather than asking LLM to read the table itself." Kong said.
What is TabFM?
With TabFM, you do not update any model weights to produce results. Instead, you take your historical samples (training rows with known labels) and target rows (the new data you want to predict) and feed them to the model as a single, unified instruction. The model learns to interpret the relationships between columns and rows directly from this context at runtime.
For example, consider a business analyst trying to predict customer churn. Instead of building a custom data pipeline and training an XGBoost model, they can simply pass a sample of historical user session data alongside a new, active session to TabFM. In one forward pass, the model returns the instantaneous discharge probability.
TabFM overcomes the limitations of LLMs by treating data as a grid, maintaining structural integrity without forcing them into a one-dimensional text string.
TabFM synthesizes the strengths of previous experimental architectures, TabPFN and TabICL, to efficiently process various table structures while providing scalable zero-shoot prediction. TabPFNDeveloped by Prior Labs, it first proved that the transformer architecture could perform zero-shot classification on small tables, although it struggled to scale computationally to larger datasets.
Later on, TabICLDeveloped by France’s National Research Institute for Digital Science and Technology, it solved this bottleneck by applying row compression, which allows in-context learning to efficiently process larger tables.
TabFM combines the deep feature contextualization of TabPFN with the efficient compression of TabICL into a new hybrid design built on three key mechanisms:
1. Alternate row and column focus: The raw table is first processed through a multi-layer focus module that switches between both columns (features) and rows (patterns). By consistently involving these two dimensions, the model captures complex trait interactions. This deep contextualization does the heavy lifting that typically requires tedious manual feature development by data scientists.
2. Row compression: After this contextualization, the cross-participation information for each row is compressed into a single, dense vector representation. TabICL pioneered this by using CLS tokens to compress the array’s rich information into a vector, "Unlike TabPFN v2, v2.5 and v2.6, which participates over a full cellular network in the network," Kong explained. This drastically reduces the computational footprint.
3. Learning in context (ICL): A causal transformer then operates on this sequence of compressed placements. This Transformer model uses TabICL’s attention mechanism to attend to these dense row vectors, dramatically reducing computational costs and allowing the model to efficiently process large datasets.
TabFM’s main selling point is its pre-prep recipe. The model was fully trained on hundreds of millions of synthetic datasets. These datasets are dynamically generated using structural causal models (SCMs) incorporating various random functions. Trained only on synthetic SCMs, TabFM learned the underlying mathematical priors of tabular feature interactions without accepting real-world, cryptic CSV files.
TabFM is in action
To test the model’s capabilities, Google researchers benchmarked TabFM against TabArena, a comprehensive evaluation suite covering 51 different tabular data sets across 38 classification and 13 regression tasks.
In these public benchmarks, TabFM’s zero-shoot predictions match or match the already heavily regulated controlled baselines. Google cautions, however, that this doesn’t automatically mean TabFM will dethrone bespoke, hyper-optimized production models in every enterprise workload.
"Rather than replacing hyper-optimized production models, the real practical business value it unlocks for lean engineering teams is speed," Kong said. "This allows data analysts and back-end engineers to instantly create high-quality base models without a dedicated data science team managing a complex lifecycle."
Also a research group for advanced practitioners who want to achieve maximum accuracy "TabFM-Ensemble" configuration. By running the model through 32 different variations and mixing the results, TabFM further improves performance.
Startups, exchanges, and the future of the cloud
The shift to in-context learning for spreadsheets presents a new economic trade-off that engineering teams must consider.
Training with traditional algorithms is slow and expensive, but inference is lightning fast and cheap. TabFM reverses this dynamic. When the training time drops to zero, the result is significantly worse. Because the model must process the entire historical data set as context during each prediction, it requires more computation and memory at runtime.
In this new paradigm, "traditional machine learning training becomes a “preloading” phase (KV caching) in the context window," Kong said. Although this prefetching cost is high, it is paid only once per table and the cache is reused in subsequent queries. "Catch is the prediction delay that no amount of caching can remove," Kong added. Each new prediction requires going through a huge transformer. "Any production API that requires single-digit millisecond response times cannot tolerate TabFM’s forward link overhead."
Today, the barrier to entry is low for developers who want to evaluate the model. Google designed TabFM as a drop-down replacement for traditional ML workflows, and scikit-learn offers a compatible API (TabFMClassifier and TabFMRegressor). It natively handles mixed numeric and categorical columns, works directly with pandas’ DataFrames, and requires no manual row encoders or numeric scalers. The library supports both JAX and PyTorch backends.
However, enterprise teams should be aware of existing limitations and licensing restrictions. The model architecture has a hard limit of 10 output classes for classification tasks, and it is optimized for tables with up to 500 features. More importantly, when removing Google main codebase pre-trained model weights are published under the permissive Apache 2.0 license Hugging Face under the strict tabfm-non-commercial-v1.0 license. Developers can evaluate the model internally, but it cannot yet be implemented in commercial products.
Forward-thinking Google is solving commercial deployment friction through its cloud ecosystem. TabFM integrates directly with Google BigQuery, allowing analysts to run zero-shot predictions via the “AI.PREDICT” command. By putting the underlying model output next to the data warehouse, TabFM can soon make complex table machine learning accessible as a basic database query.
In practice, TabFM shines in rapid prototyping, high data throughput environments, and small to medium sized databases below 100,000 rows. Conversely, teams should stick with traditional models for serious, ultra-low-latency APIs or massive tables with over a million rows that require aggressive row sampling that currently undermines the competitive advantage of the underlying model.
