As developments in robotics, autonomous driving, and spatial computing proceed to unfold, a rising variety of Laptop Imaginative and prescient and Machine Studying (CVML) algorithms are incorporating three-dimensional knowledge into their frameworks. Debugging these 3D CVML fashions typically requires going past conventional efficiency analysis strategies, necessitating a deeper understanding of an algorithm’s habits inside its spatio-temporal context. Nonetheless, the dearth of applicable visualization instruments presents a big impediment to successfully exploring 3D knowledge and spatial options in relation to key efficiency indicators (KPIs). To deal with this problem, we discover the appliance of Immersive Analytics (IA) methodologies to reinforce the debugging means of 3D CVML fashions. By in-depth interviews with eight CVML engineers, we establish frequent duties and challenges confronted throughout the growth of spatial algorithms, and set up a set of design ideas for creating instruments tailor-made to spatial mannequin analysis. Constructing on these insights, we suggest a novel immersive analytics system for debugging an indoor localization algorithm. The system is constructed utilizing net applied sciences and integrates WebXR to allow fluid transitions throughout the reality-virtuality continuum. We conduct a qualitative examine with six CVML engineers utilizing our system on Apple Imaginative and prescient Professional, observing their analytical workflow as they debug an indoor localization sequence. We focus on some great benefits of using immersive analytics within the mannequin analysis workflow, emphasizing the position of seamlessly integrating 2D and 3D visualizations throughout various ranges of immersion to facilitate simpler mannequin evaluation. Lastly, we mirror on the implementation trade-offs and focus on the generalizability of our findings for future efforts in immersive 3D CVML mannequin debugging.
- † Harvard College, Cambridge
