PM4Splat3DScene: A Scriptable Parametric Modeling Interface for 3D Gaussian Splatting Scene Synthesis Using PM4VR

The quest for photorealistic, efficient, and flexible 3D scene representation has long been a central pursuit in computer graphics and vision. The recent dominance of Neural Radiance Fields (NeRFs) [5, 17, 26] demonstrated the power of coordinate-based neural networks to model continuous volumetric radiance and density, achieving stunning view synthesis quality. However, NeRFs are fundamentally implicit and slow; rendering requires querying a deep network millions of times per frame, precluding real-time interaction without significant approximation.

In this context, 3D Gaussian Splatting (3DGS) [6], introduced in 2023, emerged as a paradigm-shifting alternative. Its core insight was: represent a scene not with an implicit neural network, but with an explicit set of discrete, learnable primitives—anisotropic 3D Gaussians. Each Gaussian is a compact, differentiable entity with spatial extent (covariance) and view-dependent appearance (spherical harmonics) [4, 22]. The rendering process, differentiable splatting, projects these 3D primitives to 2D and blends them efficiently via tile-based rasterization on GPU [19, 25]. This combination of an explicit, structured representation with a traditional rasterization pipeline yielded a revolutionary outcome: real-time rendering of photorealistic novel views at high resolution, coupled with fast, stochastic optimization [7] from multi-view images. The field of 3D scene representation has undergone a seismic shift with the introduction of 3DGS. Moving beyond the implicit volumetric paradigms of NeRFs, 3DGS offers an explicit, structured, and inherently real-time capable representation [20, 24] and de-blurred effects [3, 8]. For sparse-view and single-view 3DGS reconstructions, there are methods like GaussianPro [2] and SparseGS [23] that integrate diffusion priors (e.g., from Stable Diffusion [21]) or monocular depth estimators to constrain optimization, mitigating the artifacts that plague standard 3DGS with few inputs. The algorithm of anisotropic 3D Gaussians with spherical harmonics for view-dependent appearance, and its optimization pipeline combines differentiable rasterization with adaptive density control.

Meanwhile, parametric modeling offers a solution by defining models through mathematical equations and parameters, enabling intuitive, flexible design. When combined with Virtual Reality (VR), PM4VR platform [9] provide immersive, interactive spatial interfaces for manipulating parametric designs through VR controllers [10, 11, 12, 13, 14, 15]. As 3DGS has emerged as a leading technique for reconstructing and rendering complex scenes from images, 3DGS is poised to impact fields of parametric modeling. However, there is no existing parametric modeling interface for direct integration of 3D Gaussian scenes with mesh geometry. For example, SideFX Houdini [1] does not include built‑in 3DGS support. Given the above observations, this paper presents PM4Splat3DScene (as shown in Fig. 1), a novel system that synthesizes these paradigms. Its core contribution is a scriptable parametric modeling interface built within VR, designed explicitly for 3D Gaussian Splatting scene creation. This work extends the PM4VR framework to treat "Splat 3D Scenes" as parametric entities controllable via VR controllers.

Overview. Fig. 2 illustrates the overview of the PM4Splat3DScene system, which bridges procedural content creation with high-fidelity, explicit neural scene representations. This system streamlines the creation of 3DGS assets through easy-to-write Java^♭ code. The process begins with (a) a single image input, which serves as the initial visual source for reconstruction. Next, (b) users write Java^♭ code to define and load corresponding 3DGS assets with PM4VR's coding interface. After preparing the script, (c) it is executed within PM4Splat3DScene to trigger automated 3DGS asset creation. (d) This system employs Apple's SHARP model [16] to produce photo-realistic 3DGS assets through a well-trained deep neural network. Finally, (e) users can synthesize and customize parametric designs directly inside the 3DGS scene via PM4Splat3DScene, enabling immersive spatial composition.

Adding Splat3D Scene. PM4Splat3DScene's addSplat3D method establishes a streamlined, on‑demand pipeline for creating and loading a 3D Gaussian Splatting (3DGS) asset in Unity Editor by chaining several steps together so that the entire workflow can be triggered with execution of Java^♭ script. When addSplat3D("assetName") method is invoked, the logic implemented in PM4Splat3DScene first checks whether a prebuilt 3DGS asset already exists at "Assets/GaussianAssets/assetName.asset"; if it does, the system immediately proceeds to load that asset into the target GameObject in real-time, minimizing unnecessary computation to regenerate the 3DGS assets. If the asset is missing, the pipeline automatically pivots to a generation path: it begins by predicting or generating the underlying 3D point cloud from a single input image by calling the above Python script implemented by PM4Splat3DScene, which reconstructs the scene and outputs an intermediate.ply file. With that.ply file in place, the code converts it into a Unity‑native GaussianSplatAsset using a rendering package for Unity Editor [18]—storing the result back under "Assets/GaussianAssets" using the provided assetName. To keep the project tidy and avoid polluting the repository with build byproducts, the pipeline then deletes the temporary files that were required only for asset creation. Finally, regardless of whether the asset was found or created, the code loads resulting *.asset into specified GameObject, instantiating the 3DGS content in the scene.

In order to evaluate the applicability and functionality of the proposed PM4Splat3DScene framework, a series of numerical experiments was designed and executed within the Unity Editor version 2022. As a reference for these experiments, Figure 3 presents six distinct Java^♭ scripts that were authored for parametric modeling of conceptual outdoor installation art design and interactively load the underlying 3DGS scene structures. Here is a detailed description of the settings for PM4Splat3D system when converting a Gaussian Splat file into an optimized Unity-ready asset within 30 (secs): The input contains a 267.3 MB point cloud with about 1.13 million splats, and here it doesn't import the associated training cameras for accurate scene alignment. To speed up the output and save memory and storage, the system selects a global quality of Very Low, which applies aggressive compression across the splat attributes to drastically reduce file size. Under this preset, Position uses an 11-bit normalized format, Scale uses a 6-bit normalized format, Color is encoded using the BC7 GPU texture format, and SH (Spherical Harmonics lighting) is compressed using the Cluster 4k method, each option balancing fidelity and memory efficiency. The interface displays estimated memory usage for each attribute and shows that the resulting asset will shrink to 14.7 MB, roughly 18 times smaller than the original input. Which then automatically generates an optimized splat asset that loads quickly, uses far less GPU memory, and is placed directly into a scene.

The subsequent, visually compelling execution results of these scripts are correspondingly and respectively detailed in Figure 5, which collectively serve as a visual testament to the framework's generative capabilities. These experimental outcomes illustrate the deployment of six diverse parametric surface scripts via the PM4Splat3DScene pipeline to synthesize fully-immersive 3DGS scenes, each scene being fundamentally architected upon sophisticated mathematical parametric designs; the resultant scenes encompass a varied and illustrative set of forms, including a (1) lake, (2) seashore, (3) center, (4) tower, (5) campus, and (6) forest. For the original image inputs, please refer to Fig. 4; The mathematical underpinnings of these visualized scenes are derived from six remarkably different and mathematically significant parametric surfaces, each selected for its unique topological and geometric properties: namely, (1) the Ring Dupin Cyclide, a canal surface with circular lines of curvature; (2) the non-orientable Boy's surface, a model of the projective plane; (3) the famed Klein's bottle, a non-orientable surface with no distinct interior or exterior; (4) Dini's Surface, a coiled helicoid-like shape with constant negative curvature; (5) the Sievert-Enneper Surface, known for its constant mean curvature; and (6) the algebraic Clebsch Diagonal Surface, a cubic surface renowned for its symmetry and 27 real lines. The successful compilation of these abstract mathematical constructs into visually rich 3DGS environments through the provided Java^♭ scripts in Fig. 3 demonstrates the PM4Splat3DScene framework's bridging of parametric designs within photorealistic 3D scenes.

User Study. Building directly upon these numerical experiments that validated the technical capability of the PM4Splat3DScene framework for generating parametric designs, the subsequent investigation into the human-centric utility and practical interactive potential of this novel framework is empirically explored through a preliminary user study. The visual outcomes of this study are captured in Fig. 6, wherein a participant, equipped with an Oculus Quest 2 headset, was situated within a fully immersive virtual environment rendered in real-time by the 3DGS Unity pipeline. Within this photorealistic radiance field setting, the user was tasked with directly manipulating the design parameters of a complex 3D installation art piece, specifically a series of conceptual sculptures whose underlying forms were defined by the same mathematical surface as shown in Fig 1. By leveraging the PM4Splat3DScene framework, the user can refine the parametric designs in real-time; a process made tangibly immediate as the visual topology, scale, and intricate geometry of the sculptural forms morphed and evolved instantaneously within the 3DGS scene in direct response to the act of scrolling through parameter sliders using the VR controller. This seamless loop between manual input and visual transformation, where the immersive 3DGS environment itself becomes an artistic medium, mirrors the foundational technical experiments but transposes them into a user-centric context. Consequently, this immersive creative workflow provides compelling initial evidence of the framework's potential to enable intuitive artistic customization, rapid prototyping, and fluid design iteration directly within high-fidelity, photorealistic 3DGS scenes, thereby bridging a critical gap between abstract algorithmic generativity and embodied creative practice. For a complete video recording that captures the full nuance of the participant's exploratory session, are provided for review at the following link: https://youtu.be/pCCM-Qbre8k.

This paper presents PM4Splat3DScene: a scriptable parametric modeling interface for 3D Gaussian splatting scene synthesis using PM4VR. PM4Splat3DScene introduces a groundbreaking approach to content creation for parametric modeling within 3D Gaussian Splatting (3DGS) scenes. By fusing the flexibility of parametric modeling with the immersion of VR and the real-time fidelity of 3DGS, it opens a new avenue for artists, designers, and researchers to intuitively adjust or refine 3D parametric models through an immersive design experience within photorealistic radiance field scenes. This work lays the foundation for the future of interactive parametric design within real-time photorealistic display devices. Future work will explore advanced bidirectional interactions between parametric models and radiance fields, enabling dynamic constraints and semantic-level editing that adapt in real time. Additional extensions may integrate physics-aware modeling, multi-user collaboration, and scalable cloud-based scene synthesis workflows. Furthermore, expanding the system to support heterogeneous radiance representations and optimized on-device processing will broaden its applicability across emerging VR/AR hardware platforms.