Robotecture: A Modular Shape-changing Interface Using Actuated Support Beams

3.1.1 Self-lifting Surface. As mentioned in Section 2, previous shape-changing displays such as pin-based actuation approach require extra supporting structures beneath the surface, consequently restricting the use of internal space. In contrast, Robotecture's innovative structure introduces a self-lifting surface by utilizing actuated beams to lift the unit without requiring extra support under itself. This approach makes it possible to leverage both sides of the surface, thereby opening up new interaction possibilities, such as enclosure simulation for architecture, furniture, and intelligent storage systems.

3.1.2 Sparse Grid Design. Simulating large-scale surfaces, such as architecture, typically doesn't require high-resolution shape rendering. Therefore, we propose a sparse grid design that reduces the number of robotic units needed to form the surface while still displaying shapes with reasonable detail. This approach makes Robotecture a cost-effective solution for large-scale shape-changing displays suitable for furniture and architecture simulation. Additionally, the shape-rendering resolution can be further adjusted by varying the length of the supporting beams: longer beams result in fewer robotic units and lower resolution, while shorter beams provide higher resolution with more robotic units.

3.1.3 Modular Robotic Unit. Our modular robotic unit design allows Robotecture to adapt to different user activities, making it easy to assemble surfaces of varying sizes and configurations. Much like assembling Lego blocks, the robotic units can be easily connected by stacking the beams together. This design choice enables new modules to be easily added to the existing grid by inserting motors into the unit chassis and connecting motors to support beams. With their self-lifting capability, the units can be arranged at various angles to create surfaces that go beyond horizontal and vertical planes. This design strategy promotes adaptability and customization, making Robotecture highly versatile for responsive physical environments.

3.1.4 Smooth and Weight-tolerance Interface. To adapt to use cases such as smart furniture, it is important that the shape-changing surface can accommodate human interaction comfortably and support substantial weight. To achieve this, Robotecture is equipped with a fabric layer that covers its robotic grid, providing a smooth, soft, and user-friendly interface for human interactions.

As the units move, the surface enclosed by four adjacent beams will be reshaped continuously with a change in its surface area. Therefore, using elastic materials or deployable structures to cover the surface is a fundamental design choice for Robotecture. For use cases that require direct body contact, such as smart beds and chairs, soft materials such as elastic fabric blankets or sheets can be used. For applications requiring rigid connections, such as architectural floors or walls, rigid materials with deployable origami structures that can expand and contract to cover the surface are suitable.

Additionally, the incorporation of beam support enhances the surface's weight tolerance, reducing deformation under applied force and ensuring the system's overall robustness.

4.1.1 Modular Actuator. Each actuated unit, modularized in a chassis (with the dimension of 15cm x 10cm x 10cm, see Figure 4(h)), is equipped with two servo motors, with each motor's shaft linked to the joint of two support beams connected at one end. The motor transmits the rotational force to the two support beams, then to adjacent components, and finally to the support frame. As a result, rotation causes the unit to raise and lower.

The model of the servo motors used in the prototypes of Robotecture is MG996r (Figure 4(a)), which has a maximum torque of 1.47N · m at the voltage of 6V. In the Robotecture prototype with the layout of 3x3, we use two 16-channel PWM servo drivers (PCA9685) and a 12V 50A power supply for powering all 18 motors via two 6V 25A power adapters and two power distribution boards. The system is controlled by an Arduino UNO microcontroller. The software control manages the rotation (thus, manages the height) of each unit through serial communication.

4.1.2 Telescopic Beams. For each unit, every support beam consists of 2 rigid poles with the internal pole sliding inside the external tube, allowing the beam to collapse or expand telescopically. In our prototype, the internal pole is a wooden square dowel rod (Figure 4(e)) measuring 0.95cm on each side and 16cm in length; and the external tube is a 3d-printed tube (Figure 4(c)) that allows the wooden rod to slide smoothly in or out of it. With the telescopic beams connected to both of its unit's motors and adjacent units’ motors, the telescopic beams will collapse or expand in response to the rotations of its motor.

4.1.3 Rotational Motion to Height Adjustment. The angle between each motor's two support beams can be controlled through the motor's rotation. As a result of this rotation, the unit's height changes due to the nature of the mechanical structure. As depicted in Figure 5, when the angle α is 180 degrees, the unit is at its neural position (the two beams are aligned in a straight line); when the angle α(α₁ + α₂) is smaller than 180 degrees, the unit lifts itself up; when the angle α is larger than 180 degrees, the unit lowers itself down.

In order to control the height adjustment, we need to calculate the rotational angle required for each motor. For each pair of support beams, α represents the angle required to determine the rotation needed from the motor. We can derive angle α by calculating α₁ and α₂ with Equation 1 and 2. The method calculates tan(α₁) and tan(α₂) from trigonometric ratios in Equation 1, where l is the horizontal distance from the unit to the adjacent unit (also the cell size of the grid) and (b − a), (b − c) indicate the corresponding height difference between units. Then, we can derive tan(α), and subsequently, arctan can help calculate the rotation angle α. In this way, we create a mapping between rotation and height, which allows the system to control height as a function of rotation.

Finally, we use arctan to get the rotation angle α. It is worth mentioning that the tan function exhibits periodicity. Therefore, to ensure injectivity in obtaining the angle, we need to restrict the domain by comparing the heights between the three adjacent units and checking whether the angle is greater or smaller than 180 degrees. Generally speaking, a smaller angle α results in a greater height and a larger angle α results in a lower height for the unit. However, the actual angle is limited to a range of 90 to 270 degrees due to the mechanical structure. Freedom of movement is bounded by the length of the support beams and the chassis which provides space for the support beams to move without colliding with them.

4.1.4 Support Frame. The outermost rigid support frames of our prototypes are assembled with 3D-printed joints connecting the square wooden rods (0.95cm on each side with two lengths of 20 cm and 8.5cm). The dimensions of the frame depend on the layout of the target setup. For each grid cell, the size (step width) is 20cm; For our prototypes, the 2x2 layout (see Figure 2) is 64cm x 64cm and the 3x3 layout (see Figure 7(a)) is 85cm x 85cm.

4.1.5 Fabric Display. As Figure 8 shows, an elastic fabric layer can attach to the top of Robotecture to render the approximate shape of a smooth surface. In our prototype, this layer of fabric uses hoop and loop tape to stick onto each unit and to the frame edge, so that it can be exchanged for other materials. Our applications use different types of elastic fabric materials, transparent or opaque, to serve various interaction purposes.

4.1.6 Software and Electronic Control. In our 3x3 implementation, one Arduino UNO board is used to process the model and send out PWM (pulse-width modulation) signals to control the motors to rotate to the desired angle. two Adafruit PCA 9685 servo driver boards are used to distribute the Arduino signal to 18 servo motors used in the 9 Robotecture units. For the electrical power supply, a 12V regulated power supply was transited to a 6V power with a power transformer. The power was connected to the motors with wires through a power distribution board. The prototype communicated with a laptop computer (Windows operating system, i7-1365U CPU, 32GB memory, integrated Intel GPU) through serial port. We implemented our applications with Unity and WebXR.

6.1.1 Smart Architecture. Architecture serves as an exemplary use case for large-scale enclosures facilitated by Robotecture. By employing Robotecture units as dynamic, shape-changing roofing structures, we unlock an array of possibilities that traditional, rigid architectural elements cannot achieve. This not only enhances user convenience but also improves the building's adaptability to environmental conditions. We used our prototype as a miniature architecture model to illustrate several scenarios using a 3x3 layout:

1. Intelligent Opera House: In Figure 10(a), we simulated a smart opera house in which the ceiling dynamically reshapes into a semi-spherical structure to optimally reflect the performer's voice toward the audience.
   
2. Solar-Powered Roof: With solar panels affixed to the rooftop, in Figure 10(c), the structure can detect the sun's direction and automatically tilt the roof to maximize sunlight exposure, thereby optimizing solar energy collection, contributing to eco-friendly home design.
   
3. Rainwater Collection: When it rains, the roof can reconfigure itself to create a central dip, thus establishing an efficient system for rainwater collection. In Figure 10(d), we simulated a rainfall scenario using beads, and demonstrated how the roof surface would depress upon sensing rainfall. This feature also potentially enables eco-friendly architectural designs by improving the utilization of natural resources.
   
4. Light Bulb Replacement: In Figure 10(b), if a fault light bulb hanging from the ceiling is out of the user's reach, the ceiling node to which the light is attached can lower itself, allowing for easy replacement. This principle can extend to similar situations, such as hanging paintings or accessing items stored overhead, providing greater user convenience and increasing object accessibility.
   

6.1.2 Dynamic Space Division. For room-scale interactions, Robotecture can be used to dynamically partition a room. In Figure 10(e), we demonstrate this concept with a 1x2 display, which bisects a miniature model of a house into two distinct rooms. The available space in each room can be adjusted dynamically based on occupancy. When an individual occupies one room, the Robotecture wall recedes, thereby providing more space for the occupied room while proportionally reducing the size of the unoccupied room. This novel functionality helps users optimally leverage limited spatial resources.

6.2.1 Smart Furniture Simulation. The user-friendly interface of Robotecture makes it a prime candidate for applications in smart furniture simulation. Its beam-supporting structure also adds extra support for the surface, providing more stability and safety for interactions such as sitting or lying on top of the surface. Leveraging our 3x3 prototype, we developed a shape-shifting bed capable of transforming into a chair, aimed at facilitating user wakefulness when an alarm activates. In Figure 10(f), this structure transitions from a planar surface (acting as a bed) into a chair-like form, effectively aiding in physically rousing the user from sleep. Integrating a pressure sensor and using a larger grid would provide the potential for detecting the user's body contours and subtly adjusting the surface shape in response, to optimize for comfort and ergonomic fit. This paves the way for adaptive living environments, continually adjusting to user needs and preferences. By dynamically altering physical affordances, the system can render daily tasks more conveniently and efficiently. The potential for novel forms of user interaction encompassing both tangible and spatial experiences enriches the dialogue between humans and their environment.

6.2.2 Haptic display for MR. Robotecture serves as an adept medium for embodying virtual models in physical reality, offering a tangible display with projection capabilities or tactile feedback within VR scenarios. In Figure 10(h), we constructed a terrain simulation of a vehicle driving along a mountain road. We incorporated projection mapping in tandem with the physical re-creation of variably tall trees using Robotecture, thereby curating a physical display experience. Furthermore, in Figure 1(d) we rendered a rocky landscape within our VR setting and simulated that model on the Robotecture surface. Users can engage physically with the surface, receiving haptic feedback that aligns with the VR environment. As we merge more surfaces at differing layers and angles, we envision this system forming increasingly intricate and immersive spaces, like the comprehensive environment within a cave, inclusive of floor, walls, and ceiling. Such a scenario might pose significant challenges to alternative approaches. The surface's smoothness ensures a comfortable interaction, while the easily interchangeable fabric material permits rapid adoption of diverse textures.

6.3.1 Hiding and Revealing Objects. In Figure 10(g), we showcase Robotecture's ability to conceal and reveal objects using a ball approximately 7cm in diameter. A small slit was made in the elastic fabric covering the surface to create a passageway for the ball, which initially lies on a base below the surface. When the beam structure lowers, the fabric is stretched both by the beam and the object, which causes the slit to open. This mechanism allows the object to pass up through the surface. Once the object is above the surface and the stretching force is reduced, the fabric returns to its original form, allowing the object to rest stably above the now-closed surface. To take the ball back down to its underneath storage, the user can push the ball through the slit. Robotecture fabric will remain its original form, effectively concealing the object. This dynamic process illustrates the potential of Robotecture as an interactive storage system that responds to users’ needs.

6.3.2 Object Actuation and Tangible Telepresence. Robotecture also has the capacity to move objects on its surface by creating an inclined plane, thus providing a sliding surface. As shown in Figure 1(b), once the ball is brought to the surface by Robotecture, it can then be slid towards the user's hand based on the hand position. In Figure 10(i), when integrated with a network system, Robotecture can be remotely controlled to manipulate physical objects from a distance, thereby implementing tangible telepresence.

7.1.1 Height Range. As we mentioned in Section 4.1.3 and Section 5, the height range is bound by the grid size, and the architecture can't form right angles between adjacent units. Therefore, for the mechanical design to reach a vertical surface from a horizontal plane, an origami-inspired SCUI (similar to Mori [2]) would be a reasonable approach, which would be one of our future directions.

7.1.2 Self-lifting and Weight Bearing. There is a side effect discussed in the previous sections that self-lifting causes loading capacity bound by its position in the layout. Units in the center usually have lower capacity than the units closer to the outer frame. One way to overcome this issue while maintaining the expandability of Robotecture is by connecting smaller grids with rigid frames to form a greater surface, as depicted in figure 9. This way the capacity limitation will be only restricting the inner part of each section, while maintaining robustness for the greater surface. However, this design will further limit the height range of the overall surface and greatly reduce the potential of shapes to form in the scale of the greater surface.

Another side effect of the self-lifting design is that the cumulative free play of the mechanical structures (e.g. the small gaps between inner and outer components of each support beam) and motors becomes noticeable due to gravity as the number of units increases. In the 3x3 layout prototype, we observed a slight sagging effect in the center unit when the system is placed horizontally. The sagging effect could potentially reduce the precision of the height adjustment as the number of grid units increases.

Another way to solve these issues would be using higher precision components and higher quality materials for the support beams rather than 3D-printed components. Additionally, using high-quality motors with better power-to-weight ratios and minimal free play would also contribute to increasing load capacity and reducing sag. For even greater capability, motors like the RDS5180 150kg combined with aluminum alloy or carbon fiber beams would enhance both strength and durability. According to our calculations, this setup could support up to 1.2 kg at the center of a 10x10 grid structure, with an estimated cost of $80 per unit.

Additionally, according to our tests, placing the prototypes vertically can significantly recover the load capacity of the outer units and thereby reduce sagging effects. This is because this orientation causes the weight-bearing to be distributed more on the support beams rather than relying on the motors to compensate for the weight.

7.1.3 Size and Resolution. The Robotecture's sparse grid structure presents some trade-offs on the size and resolution of the shape-changing display. In our prototypes, the individual units are generally larger when compared to pin-based SCUIs. The primary cause of this limitation is the size of the MG996R motor, which influences both the chassis dimensions and the length of beams at certain levels. Therefore, one trade-off of the sparse grid structure is that the resolution is lower than most pin-based SCUIs. We could use smaller motors (e.g. SG90) and increase the number of motors to create a denser and higher-resolution version. However, this would cause a reduced load capacity, as smaller motors typically generate lower power. Nevertheless, a reduced load capacity for a denser layout should be a concern because the intended focus of Robotecture is primarily designed for larger scales, at least tabletop and room scale. The smaller prototypes showcased in previous sections are only intended for validating the design and mechanisms. The smaller scale simplifies engineering and reduces cost.

7.1.4 Lack of Physical Activity Sensing. Robotecture primarily focuses on leveraging its shape-changing attributes rather than sensing physical activities. Thus, to simplify the design and engineering process, the mechanical structure does not have an embedded force sensor to directly capture users’ input. In future work, we plan to integrate force sensing into our prototypes, such as bonding torque sensors to the shaft of the motors to enable more versatile interactive experiences. For example, Robotecture prototypes at tabletop size could help scan and replicate objects’ physical properties (similar to the functionality of inFORCE [24]).

7.2.1 Adaptive Floors. One possible scenario is that floors constructed using Robotecture architecture in a building could dynamically change their shapes, adjust their elevation and curvature, and seamlessly connect with other floors by creating smooth walkable pathways, slopes, and surfaces. As a result, people of all abilities, including those with limited mobility, will be able to move around freely and navigate between different floors without needing to take stairs or elevators. For instance, a user wants to enter their bedroom located in a duplex home. The Robotecture-based first floor dynamically reconfigures to lift the user, while simultaneously the second floor lowers to meet the user. Additionally, by incorporating Robotecture's object handling feature, it becomes efficient and convenient for users to transport heavy objects between floors.

In order to achieve this, Robotecture-based floors will require powerful motors, stronger frames and support beams to lift the structure and to support the weight of humans. Industrial-level motors (e.g. Moog Industrial Motors) with torque over 981 N · m (capable of lifting 100 kg with 1-meter linkage) would have the potential to be selected for the application. Our calculation shows that while using Moog motors in a 3x3 grid with beam length to be 1meter, the central unit can stand the weight of an adult. Using resilient materials capable of bearing human weight such as metallic (e.g. aluminum alloy, steel) facets or woven polypropylene, individuals will be able to freely walk across the entire surface, including areas covered by these resilient materials, without being restricted to only walking on the support beams. This scenario opens up research opportunities to develop functionalities that can be enhanced by eliminating the need for stairs or elevators, as well as exploring the possibilities of creating a smart building with improved accessibility.

7.2.2 Reshaping room. Our vision extends to the potential of constructing an entire room, including floors, walls, and ceilings. This could allow the integration of multiple applications that we have showcased, thereby creating an intelligent room that reshapes according to its user's needs. In this vision, the floor could transform into various kinds of smart furniture, such as tables, chairs, beds, and dynamically reshape to accommodate the user's body shape.

Robotecture can also help to create customized public or private spaces based on the requirement of divisions and users’ preferences. Differing from pin-based floors, the self-contained surface design affords storage space beneath the surface, thereby making efficient use of the room's volume. Furthermore, the beam structure, when combined with elastic surface covers, enables a smooth surface while providing sufficient support. This design ensures a comfortable and user-friendly interface for bodily contact.