Ancestral Craft and Emerging Technologies: Designing Futures from Place

gabriela bila advincula, Media Lab, Massachusetts Institute of Technology (MIT), Cambridge, USA, gba@mit.edu

Jonny Cohen, Media Lab, Massachusetts Institute of Technology (MIT), Cambridge, USA, jonny@mit.edu

DOI: https://doi.org/10.1145/3799822.3812473
SIGGRAPH Spatial Storytelling '26: Special Interest Group on Computer Graphics and Interactive Techniques Conference Spatial Storytelling, Los Angeles, CA, USA, July 2026

Immersive media such as extended reality (XR) often rely on generic interaction models and aesthetic languages detached from the places they represent. This paper presents a situated design methodology for spatial storytelling developed through a free-roaming XR installation in Amazonia that combines embodied interaction, craft-based fabrication, and material robotics. Two interconnected systems structure the work. The first is a body-scale interface derived from regional basketry and performative costume traditions, translating full-body movement into navigation through a infrared tracking pipeline and real-time virtual environment. The second consists of underactuated robotic sculptures constructed from miriti, a lightweight riverine material, whose movement emerges through compliant structures and minimal actuation. Both systems are synchronized, creating continuous feedback between participant, artifact, and spatial narrative. The project proposes cultural specificity not as representational content alone, but as a generator of interaction logic, fabrication processes, and computational behavior within immersive media systems.

CCS Concepts: • Human-centered computing → Virtual reality; Empirical studies in interaction design; • Applied computing~Media arts;

Keywords: spatial storytelling, embodied interaction, material robotics, situated media

ACM Reference Format:
gabriela bila advincula and Jonny Cohen. 2026. Ancestral Craft and Emerging Technologies: Designing Futures from Place. In Special Interest Group on Computer Graphics and Interactive Techniques Conference Spatial Storytelling (SIGGRAPH Spatial Storytelling '26), July 19--23, 2026, Los Angeles, CA, USA. ACM, New York, NY, USA 3 Pages. https://doi.org/10.1145/3799822.3812473

Figure 1: Left: XR basketry interface in use during exhibition. Right: kinetic miriti sculptures in the installation space. Photographs by Bruno Carachesti.

1 Introduction

Spatial computing has expanded how environments are represented and inhabited, yet its interaction grammar often remains universalized: standardized controllers, generic locomotion systems, and interface conventions detached from the places they represent [Jerald 2015]. In contrast, vernacular fabrication practices operate as knowledge systems that structure relationships between bodies, materials, and environments.

This paper presents a situated framework for spatial storytelling developed through a free-roaming XR installation in Amazonia. Combining embodied interaction, craft fabrication, and material robotics, the project investigates how culturally specific practices can shape not only representation, but the operational logic of immersive systems [Escobar 2018]. Two interconnected systems structure the work: a body-scale XR basketry interface and a set of underactuated kinetic sculptures fabricated from miriti, a lightweight riverine material.

Figure 2: Left: fabrication of the woven basketry interface. Right: Infrared tracking view used to capture participant movement and spatial orientation. Photos by the authors.

2 XR Basketry Interface

The XR interface adapts regional basketry and festive costume traditions into a body-scale input device. Suspended elastically from the ceiling, the woven structure mediates between participant and computational system through resistance, oscillation, and delayed motion. Navigation emerges not through discrete commands, but through continuous negotiation with the artifact.

Movement is captured using a top-down infrared camera tracking reflective markers mounted on the structure. Through thresholding and blob detection, the system extracts position, orientation, velocity, and a pseudo-depth dimension derived from marker distance. Motion data streams via OSC into Unreal Engine and TouchDesigner, where bodily movement modulates camera navigation, sound, lighting, and environmental behaviors.

Beyond the interface itself, the virtual environment was also developed through collaborative and place-based processes. Combining cinematic footage, real-time environments, and speculative worldbuilding, the project imagines seasonal urban futures shaped by tides, flooding cycles, and shifting ecological conditions in Amazonia. Rather than functioning as representational decoration, the basketry structure shapes the interaction model itself, translating embodied and material forms of knowledge into computational behavior [Ishii and Ullmer 1997].

Figure 3: Details of the motor system and underactuated kinetic miriti wood structure. Photos by the authors.

3 Kinetic Miriti Sculptures

The second system consists of kinetic sculptures fabricated from miriti, a lightweight plant-based material traditionally used in Amazonian craft practices. Rather than treating miriti as a flexible substrate, the sculptures are constructed from articulated rigid segments connected through textile joints and embedded elastic structures.

Motion emerges through an underactuated tendon-driven system in which a single motor tensions the segmented body. Elastic return forces and slight variations in carving, joint tightness, and assembly produce oscillatory behaviors that exceed the simplicity of the control input. Instead of prioritizing precise control, the system leverages compliance and material variability as expressive components of motion.

Integrated with synchronized lighting and XR environments through a shared real-time pipeline, the sculptures operate as distributed spatial actors within the installation, linking robotic behavior, atmosphere, and narrative space.

4 Distributed Workflow and Reflection

The project was developed through a distributed workflow connecting designers, engineers, and craft practitioners across Brazil and the United States. Basketry and miriti components were fabricated locally while sensing, real-time systems, and electromechanical components evolved in parallel through remote iteration. Modular connections and adaptive calibration strategies allowed final on-site assembly to absorb material variability rather than eliminate it.

Together, the two systems propose an alternative approach to immersive media in which cultural specificity operates not only as representational content, but as a generator of interaction logic, fabrication processes, and computational behavior. In this framework, computation becomes a mediator between local knowledge systems, material ecologies, and speculative futures [Haraway 2016].

Acknowledgments

Imaginary Atlas was led by Gabriela Bila, with technology direction by Diogo Costa Pinto. The project was developed through collaborations between MIT Media Lab City Science, Quanta – Novas Fronteiras, artists, engineers, and craft practitioners across Brazil and the United States. The authors would like to thank Marcelo Vaz (DIMIRITI) for the miriti and basketry pieces; Jonathan Cohen and Kye Shimizu for electromechanical system development; Dimitre Lima for tracking and LED lighting systems; and Luan Rodrigues and Victoria Machado for local research assistance.

This work was supported by Rumos Itaú Cultural, Fundo de Apoio à Cultura, and MIT CAST – Center for Art, Science and Technology. The authors also thank the Vila Sapo Community in Mosqueiro, Pará, Brazil, and the Vale da Benção Community in Caxiuanã National Forest, Pará, Brazil, as well as institutional partners including FAU-UFPA, Ferreira Penna Scientific Station, Fórum Landi, and Museu Paraense Emílio Goeldi.

References

Arturo Escobar. 2018. Designs for the Pluriverse: Radical Interdependence, Autonomy, and the Making of Worlds. Duke University Press, Durham, NC.
Donna Haraway. 2016. Staying with the Trouble: Making Kin in the Chthulucene. Duke University Press, Durham, NC.
Hiroshi Ishii and Brygg Ullmer. 1997. Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’97). ACM, New York, NY, 234–241. https://doi.org/10.1145/258549.258715
Jason Jerald. 2015. The VR Book: Human-Centered Design for Virtual Reality. Association for Computing Machinery and Morgan & Claypool.

Footnote

^⁎Authors contributed equally to the paper

CC-BY license image
This work is licensed under a Creative Commons Attribution 4.0 International License.

SIGGRAPH Spatial Storytelling '26, Los Angeles, CA, USA