Autonomous Fruit Harvesting Robot

This project is an autonomous fruit harvesting robot developed as a collaborative research project between IIT Kanpur and IIT Palakkad. The system uses a mobile platform with a robotic arm and a vision pipeline based on detection, segmentation, and 3D pose estimation to locate and pick fruits. The stack was implemented in Python and ROS, with simulation in Gazebo and PyBullet.

Fruit Plucking Robot Architecture

Fruit Plucking Robot architecture

Sphere Fitting For Apple Modelling

Point Cloud

Point cloud

Sphere Fitting

Sphere fitting

Deep Learning Detection Results

Sample Images In The Collected Dataset

Sample dataset 1

Sample dataset 2

Mask R-CNN Results On Our Dataset

Raw Image

Mask R-CNN raw image

Tree and Fruit Localization

Tree and fruit localization

Faster R-CNN Results On Our Dataset

Faster R-CNN result 1

Faster R-CNN result 2

YOLO V-5 Results On Our Dataset

YOLO v5 result 1

YOLO v5 result 2

Fruit Plucking Experiment

Point Cloud Simulation of Apple tree for Robot Harvesting

Home Position

Home position

Input Image

Input image

Fruit Localization

Fruit localization

Fruit Plucking

Fruit plucking

Robot Holding Apple

Robot holding apple

Drop Localization

Drop localization

Real Time Results

Test Result Using Captured Image

Test result using captured image

Real Time Result Using ORBBEC Camera

Real time ORBBEC result

2D Detected Point And Its Respective 3D Points In Different Frames

2D detected point and 3D points

False Detection Scenario

False detection

Test Result

Test result