Hardik Shah

Replacing traditional keypoint detectors with learned detection and matching methods in the tracking pipeline of Scandit's MatrixScan product, which leverages SLAM for AR-based inventory management on resource-constrained devices. Optimizing training paradigms, and exploring lightweight architectures for efficient on-device inference along with custom evaluation benchmarks.

Developed a unified CLIP-based representation combining geometry and semantics for Object Goal Navigation in unseen environments. Demonstrated that integrating CLIP feature based representation along with frontier-based exploration outperforms recent segmentation-based methods, highlighting its robust generalization and zero-shot capabilities.

Developed POLD2, a deep learning-based pipeline that jointly detects and describes both point and line features in images, optimizing feature extraction for 3D vision tasks like SLAM and pose estimation. By sharing computations between points and lines, POLD2 achieves a significant 9.5x speedup in inference time compared to traditional methods, while maintaining comparable accuracy.

Developed a versatile neural network compression toolbox that optimizes for the model's FLOPs via a novel latency surrogate across a family of compression methods, including pruning and low-rank factorization. Additionally, optimized on-device latency of large vision models used for OCR tasks in Google Lens, and QR-code scanning in GooglePay.

Designed an end to end pipeline for multi-view stereo dense 3D reconstruction from a handheld stereo-camera(Intel RealSense) that outputs stable dense pointclouds. Integration of classical visual SLAM algorithms with U-Net adapted deep learning architectures for dense depth prediction.

Demonstrated use of ensemble learning for the task of activity recognition/video classifi- cation on the Something-Something-v2 dataset. Weak learners were typically used for feature extraction. Explored various methods for combination of features, ultimately used for downstream classification

Worked under under the supervision of Prof. Sandeep Joshi and Prof. Madan Lakshmanan. on classification of a person as fatigued or non-fatigued based on PPG signals of a human subject.