Hardik Shah

Developed WildOS, combining complementary strengths of visual and geometric perception for long-range navigation and object search in large-scale off-road and vegetated environments.
Research supported by the Zeno Karl Schindler Foundation Master Thesis Grant.

Replacing traditional keypoint detectors with learned detection and matching methods in the tracking pipeline of Scandit's MatrixScan product, which leverages SLAM for AR-based inventory management on resource-constrained devices. Optimizing training paradigms, and exploring lightweight architectures for efficient on-device inference along with custom evaluation benchmarks.

Developed ForesightNav, an imagination-driven exploration strategy for efficient long-horizon and open-vocabulary navigation using CLIP-grounded semantic and geometric predictions of unseen regions.

Developed POLD2, a deep learning-based pipeline that jointly detects and describes both point and line features in images, optimizing feature extraction for 3D vision tasks like SLAM and pose estimation. By sharing computations between points and lines, POLD2 achieves a significant 9.5x speedup in inference time compared to traditional methods, while maintaining comparable accuracy.

Developed a versatile neural network compression toolbox that optimizes for the model's FLOPs via a novel latency surrogate across a family of compression methods, including pruning and low-rank factorization. Additionally, optimized on-device latency of large vision models used for OCR tasks in Google Lens, and QR-code scanning in GooglePay.

Designed an end to end pipeline for multi-view stereo dense 3D reconstruction from a handheld stereo-camera(Intel RealSense) that outputs stable dense pointclouds. Integration of classical visual SLAM algorithms with U-Net adapted deep learning architectures for dense depth prediction.

Demonstrated use of ensemble learning for the task of activity recognition/video classifi- cation on the Something-Something-v2 dataset. Weak learners were typically used for feature extraction. Explored various methods for combination of features, ultimately used for downstream classification

Worked under under the supervision of Prof. Sandeep Joshi and Prof. Madan Lakshmanan. on classification of a person as fatigued or non-fatigued based on PPG signals of a human subject.