Toward generalizable foundation models for 3D BIM geometry using a joint embedding predictive architecture

Automation in Construction

Jack Wei Lun Shi1, Wawan Solihin1,2, Yufeng Weng1, Houhao Liang3, Yimin Zhao1,
Leong Hien Poh1,
Justin Ker-Wei Yeoh1

1Department of Civil and Environmental Engineering, National University of Singapore
2Research and Innovation, NovaCITYNETS Pte.Ltd. 3CNRS@CREATE, 1 Create Way, CREATE Tower, Singapore
More details, including all training code and model weights, will be released soon. Thank you for your interest!

Abstract

The adoption of artificial intelligence in the Architecture, Engineering, and Construction (AEC) domain is hindered by a reliance on task-specific models that fail to generalize across diverse geometric applications. To address this limitation, this paper introduces a point cloud-based foundation model for 3D Building Information Modeling (BIM) geometry, pre-trained via a Latent-Euclidean Joint Embedding Predictive Architecture on individual BIM objects. By enforcing alignment across multi-view BIM object representations within a regularized latent space, the proposed model extracts robust geometric features while suppressing low-level geometric noise. Extensive evaluations demonstrate the generalizability of the learned representations across multiple downstream tasks, achieving competitive performance in standard and fine-grained object classification, semantic segmentation via transfer learning, in- and out-of-distribution part segmentation of BIM and computer-aided design objects respectively, and zero-shot tasks including shape retrieval and anomaly detection. These results advance toward a foundation model that facilitates diverse applications for 3D BIM geometry.

How the model is pre-trained

The proposed model adapts the Latent-Euclidean Joint Embedding Predictive Architecture (LeJEPA) to 3D BIM point clouds, learning geometric features by enforcing predictive consistency across multiple latent views of the same object. From a single BIM point cloud, a hierarchical multi-view generation stage produces 2 global views (i.e., 75% of the object, capturing overall topology) and 8 local views (i.e., 12.5%, capturing fine-grained detail) via Farthest Point Sampling and k-Nearest Neighbor cropping, followed by an augmentation pipeline that mimics real-world geometric noise. Each view is tokenized into 64 patch embeddings by a shared Mini-PointNet and encoded by a transformer backbone. The training objective then pulls every view embedding toward the mean of the global view embeddings, while Sketched Isotropic Gaussian Regularization (SIGReg) shapes the joint distribution of the embeddings into an isotropic Gaussian. This explicit regularization prevents representation collapse, removing the need for training heuristics such as stop-gradients or momentum-based teacher networks.

Pre-training framework of the proposed foundation model
Fig. 1. Pre-training framework of the proposed foundation model.

One model, many downstream tasks

Because the pre-trained encoder already extracts robust and generalizable geometric features, downstream architectures remain lightweight. By appending simple predictive heads to the frozen or fine-tuned backbone, a single foundation model serves object classification, part segmentation, and scene-level semantic segmentation, without constructing complex task-specific models. These heads are intentionally kept simple in order to isolate the quality of the learned representations, and further performance gains can be anticipated with more advanced decoding architectures.

Overview of the proposed model adapted for downstream task evaluations
Fig. 7. Overview of the proposed model adapted for downstream task evaluations.

Object classification

On IFCNetCore, the proposed model achieves the highest performance across every reported metric, with an overall accuracy of 89.80% and a mean class accuracy of 89.17%. Compared to the strongest baseline (i.e., MVCNN), this corresponds to gains of +2.83% on overall accuracy and +3.63% on mean class accuracy. The larger improvement on mean class accuracy indicates that the model is effective on under-represented building components, narrowing the performance gap between majority and minority classes that typically penalizes models trained on imbalanced data.

Table 2. Accuracy metrics on IFCNetCore.
Model Overall Acc. (%) Mean Class Acc. (%) Precision Recall F1
MVCNN86.9785.5487.4886.9786.93
MeshNet85.7583.3286.4585.7585.72
DGCNN82.3079.1183.2682.3082.15
SpaRSE-BIM81.5983.0282.7881.5981.80
BIM-JEPA89.80 ± 0.1789.17 ± 0.3190.13 ± 0.3489.80 ± 0.1789.48 ± 0.28

On the imbalanced BIMGEOM dataset, the model achieves an overall accuracy of 93.40% and a mean class accuracy of 91.14%. Its macro-averaged F1 score, a metric that treats minority and majority classes equally, reaches 90.87%, outperforming supervised baselines such as SpaRSE-BIM. Crucially, this minority-class performance is achieved without relying on any explicit class-balancing augmentations during the pre-training and fine-tuning stages, validating that the LeJEPA formulation successfully prevents representation collapse and forces the encoder to extract discriminative, view-invariant features even for under-represented geometric classes.

Table 3. Accuracy metrics on BIMGEOM.
Model Overall Acc. (%) Mean Class Acc. (%) Macro Avg. (P / F1) Weighted Avg. (P / F1)
Mesh to Graph8576
5-NN Graph8378
SpaRSE-BIM90.4787.6684.70 / 85.7891.43 / 90.77
BIM-JEPA93.40 ± 0.1391.14 ± 0.3590.76 ± 0.12 / 90.87 ± 0.2393.49 ± 0.12 / 93.39 ± 0.13

Scene segmentation

Semantic segmentation on BIMNet examines how well representations learned from isolated BIM objects transfer to multi-object indoor scenes. In contrast to the comparison baselines, the model operates on geometry alone (i.e., XYZ only, without color) and has not been exposed to multi-object scene contexts during pre-training. Despite these constraints, it reaches a competitive overall mIoU of 40.22% and an overall mAcc of 49.52%, matching close to the performance of task-specific architectures that are pre-trained on scenes. The model delineates prominent building and non-building elements, and accurately identifies core components such as slabs, walls, windows, and furniture. It also achieves the highest IoU across all evaluated models on stairs (i.e., 71.21%) and railings (i.e., 52.99%), which are relatively more topologically complex than standard planar surfaces such as walls or slabs.

Semantic segmentation of various scenes
Fig. 10. Semantic segmentation of various scenes.

Part segmentation

Beyond assigning a single label to a whole object, the learned features are dense enough to decompose an object into its constituent parts, which is directly relevant to tasks such as automated code compliance checking. On a curated dataset of 200 BIM objects spanning staircases, ramps, railings, windows, and doors with 17 part labels in total, the model reaches a mean mIoU_C of 78.86%, an improvement of +18.28 percentage points over a k-NN baseline that segments using positional information alone. Performance is strongest on ramps, doors, windows, and railings, and the model accurately delineates large components such as ramp flights, landings, door leaves, window frames, and handrails. It degrades on parts that are thin or finely repeated, such as risers and treads: because the training objective operates in embedding space rather than reconstructing the input geometry, the learned representations tend to emphasize globally discriminative structure over fine-grained local detail. This is visible in the staircase, where the model converges to a shortcut solution that approximately matches the aggregate proportions of tread and riser without recovering their per-step alternation, while the larger and more globally distinctive stringer and landing parts remain accurately segmented.

Part segmentation results for BIM objects
Fig. 8. Part segmentation results for BIM objects. Top row: ground truth; bottom row: prediction.

Zero-shot shape retrieval

Shape retrieval tests the inherent capacity of the model to cluster semantically and topologically similar geometries without any task-specific fine-tuning. Input point clouds are simply passed through the projection head to obtain a compact 256-dimensional embedding per object, and the top-6 nearest neighbors are retrieved by Euclidean distance in this latent space. The approach achieves a macro-averaged precision of 86.2% across the 20 tested IFC classes. Structural elements such as IfcStair, IfcRailing, and IfcDoor demonstrate great retrieval precision, ranging from 96.2% to 100%, confirming that the model distinguishes their global topological signatures with high confidence. It also handles intra-class variation robustly, easily disambiguating features such as single versus double door panel configurations. For classes with higher variability, such as IfcPipeFitting and IfcFurniture, retrieval precision remains high at 93.2% and 91.3% respectively, with the model successfully matching a query chair to other chairs despite never being explicitly pre-trained on fine-grained furniture sub-classes.

Retrieval performance across representative IFC classes
Fig. 11. Qualitative evaluation of retrieval performance across representative IFC classes.

And that's the overview! If you would like to dig into the methodology, the full set of experiments, and everything in between, do take a look at the full paper.

Citation

@article{shi2026toward,
  title={Toward generalizable foundation models for 3D BIM geometry using a joint embedding predictive architecture},
  author={Shi, Jack Wei Lun and Solihin, Wawan and Weng, Yufeng and Liang, Houhao and Zhao, Yimin and Poh, Leong Hien and Yeoh, Justin K.W.},
  journal={Automation in Construction},
  volume={191},
  pages={107169},
  year={2026},
  publisher={Elsevier}
}