Semantic-Aware Active Perception

Next-best-view planning for robotic grasping in cluttered environments.

Overview

Robotic grasping in cluttered environments is challenging because target objects are often heavily occluded. This project presents a semantic-aware active perception framework that enables a Franka Panda robot to autonomously explore the scene, localize a target object, and execute a successful grasp without prior knowledge of the object’s location.

The robot actively explores the scene before executing the final grasp.

System Pipeline

Overall pipeline of the semantic-aware active perception framework.

The framework integrates semantic perception, volumetric mapping, active viewpoint planning, and grasp planning in a closed-loop pipeline to autonomously localize and grasp occluded target objects.

Semantic-aware Information Gain

The proposed method combines workspace-level and target-centric geometric information gain to prioritize viewpoints that reveal the target object.

Experimental Results

The proposed framework was evaluated in both simulation and real-world experiments using a Franka Emika Panda robot.

  • Simulation: 84% grasp success under heavy occlusion.
  • Real robot: 10/10 success (fully occluded), 9/10 success (partially visible).

Citation

📄 Paper: Semantic-Aware Active Perception for Next-Best-View Grasp Planning

If you find this work useful in your research, please cite:

Kweon, T. H., & Jeon, S. (2026). Semantic-Aware Active Perception for Next-Best-View Grasp Planning. International Journal of Precision Engineering and Manufacturing.

@article{kweon2026semantic,
  title={Semantic-Aware Active Perception for Next-Best-View Grasp Planning},
  author={Kweon, Tae Hyeon and Jeon, Soo},
  journal={International Journal of Precision Engineering and Manufacturing},
  year={2026}
}