A document by Neşe ÖZKAN YILMAZ. Click on the document to view its contents.

This work proposes a WebXR-based cross-platform architecture for developing spatial web apps using the A-Frame and Networked-Aframe frameworks with a view to an open and interoperable metaverse, accessible from both the web and extended reality devices. A prototype was implemented and evaluated, supporting the capability of the proposed technology stack to enable immersive experiences across different platforms and devices. Positive feedback on ease of use of the immersive environment further corroborates the proposed approach, underscoring its effectiveness in facilitating engaging and interactive virtual spaces. This research enriches the discourse on the metaverse by describing an architecture based on web technologies and open standards as opposed to proprietary technologies. By adhering to principles of interoperability and inclusivity, it lives up to Tim Berners-LeeâĂŹs vision of the World Wide Web as an open platform that transcends geographical and technical boundaries.

Pan-tilt-zoom (PTZ) cameras, which dynamically adjust their field of view (FOV), are pervasive in large-scale scenes, such as train stations, squares, and airports. In real scenarios, PTZ cameras are required to quickly make decisions informed about where to direct its focus through contextual cues from the surrounding environment. To achieve this goal, some researches project camera videos into three-dimensional (3D) models or panoramas and allow operators to perceive spatial relationships. However, these works face several challenges in terms of real-time processing, localization accuracy, and realistic reference. To address this problem, we propose a visual expansion and real-time calibration for PTZ cameras assisted by panoramic models. We attempt to meet the demand for real-time processing with a motion estimation model for a PTZ camera, to improve calibration performance of PTZ images with only two feature point pairs, and to provide a realistic environmental context through a panoramic model. We verify our methods on both public and our self-built test scene. It can be seen from the experimental results that our method can exhibit impressive accuracy and efficiency.

In this paper, we present an Augmented Reality (AR)-based application combined with a robotic system for percutaneous renal puncture navigation interaction and demonstrate its technical feasibility. Our system provides an intuitive interaction scheme between the surgeon and the robot without the need for traditional external input devices, and applies an image-target-based 3D registration scheme to transform the coordinate system between Hololens2 and the robot without using additional tracking devices. Users can visualise the abdominal puncture phantom and obtain 3D depth information of the lesion site by wearing Hololens2 and control the robot directly using buttons or gestures. To investigate the accuracy and feasibility of the proposed interaction scheme, six subjects were recruited to complete 3D registration alignment accuracy experiments, and puncture positioning accuracy experiments using ultrasound unaided navigation, AR unaided navigation and AR robotic navigation. The results showed that the average alignment error of 3D registration was 3.61±1.05 mm. The average positioning errors of ultrasound freehand navigation, AR freehand navigation and AR robotic navigation were 7.67±2.00 mm, 6.13±1.07 mm and 5.52±0.37 mm, respectively; the average puncture times were 34.86±1.67 s, 22.40±2.07 s, and 29.41±1.37 s.

Stylized animation is often adored for its innovative and daring visual creativity. Due to the strong visual impact and color contrast inherent in woodcut style design, it has been applied in animation and comics. However, traditional woodcuts, hand-drawn, and previous computer-aided methods have yet to address the issues of dwindling design inspiration, lengthy production times, and complex adjustment procedures. To tackle these challenges, we propose a novel network framework, the Woodcut-style Design Assistant Network (WDANet). Notably, our research is the first to utilize diffusion models to streamline the woodcut-style design process. We curate the Woodcut-62 dataset, which features works from 62 renowned historical artists, to train WDANet in absorbing and learning the aesthetic nuances of woodcut prints, offering users a wealth of design references. Our WDANet integrates text and woodcut-style image features based on a denoising network. WDANet allows users to input or slightly modify a text description to quickly generate accurate, high-quality woodcut-style designs, saving time and offering flexibility. As confirmed by user studies, quantitative and qualitative analyses show that WDANet outperforms the current state-of-the-art in generating woodcut-style images and proves its value as a design aid.

The rapid growth of the live streaming industry brought about the VTuber trend, where content creators use character avatars to stream. One of the most accessible way to move a character in real time is by using a vision-based facial motion capture technique. However, previous works still suffer from jittering issues, which hinder the quality of the character’s movements. This work aims to develop a smoothed facial motion capture system that works with a live2D VTuber model. The system combines Mediapipe Face Mesh and OpenCV solutions to capture facial landmarks, which are then used to calculate head pose estimation using the Perspective-n-Point (PnP) function. In addition, the system uses EAR and MAR functions to detect facial features. The motion values obtained from this process are then filtered using a Kalman filter. Finally, the filtered motion data is sent to the Unity engine, which drives the Live2D VTuber model by adjusting the character’s motion parameters. The developed system successfully captures and drives the Live2D VTuber model with smoother motion, overcoming the jitter problem prevalent in previous facial motion capture approaches. The system’s improved motion capture quality makes it a more viable option for a wide range of potential uses.

The work description of virtual reality technology is superior to static reference pictures and videos, which can improve the predictability of decision makers and become a powerful performance tool for urban lighting design. The urban scene is constructed by Unity 3D, and the intelligent lamp pole and Rhododendron landscape lawn lamp are taken as the research objects. On the basis of maximizing the application effect of lamps with the HTC VIVE virtual headset, the city and lamps are digitally mapped 1:1, After applying the model material self-luminous, setting the type of light source and the position of the light source point, setting the transparency mixing, adding glow, and changing the light effect, a highly restored lamp and lamp simulation model are formed. The research results show that this method provides a convenient research method for the three-dimensional reconstruction and visual simulation of urban landscape lamps. It can make decision makers have a clear understanding of the application effect, application quantity and interval of lamps. Immersive observation experience can make the design scheme more honest and reliable. The efficient simulation of lighting lamps will play an important role in the dynamic development of smart cities in the future.