Haofei Ma

Haofei Ma

Harbin Institute of Technology

I am now a master student in the National Key Laboratory of Robot Technology and Systems of Harbin Institute of Technology, mainly engaged in the research of robot autonomous operation technology based on pose estimation and imitation learning.

I graduated from School of Mechanical Engineering, Hebei University of Technology (河北工业大学机械工程学院) with a bachelor’s degree。 Now, I’m pursuing my master’s degree in the School of Mechatronics Engineering, Harbin Institute of Technology (哈尔滨工业大学机电工程学院), advised by Weidong Wang (王伟东).

My research interest includes robot operation, grasping, object pose estimation. I have submitted 5 papers in SCI journals, applied 1 invention patent, authorized 3 utility model patents and 1 software copyright.

📖 Educations

🔍 Projects

Sampling Robots in Complex Environments

2023.03 - Present
sym

Because of the irregularities on the object’s surface and variations in lighting conditions, point cloud images obtained from a single perspective often contain significant gaps and errors at the object’s edges, which can lead to inaccurate grasping pose estimations. To tackle these challenges, this article proposes a practical robot grasping method based on 6D pose estimation and point cloud fusion. First, 6D pose estimation is conducted, utilizing the results from model point cloud and pose estimation to complete the input point cloud through ICP (Iterative Closest Point). Subsequently, the resulting complete object point cloud is used to estimate the 6D grasping pose of the object with the help of the grasping direction estimation network.

Finished Works:

  1. Establish a model-free 6D pose estimation network based on perspective matching. Construct and train the pose estimation network to achieve precise pose estimation for seen objects and a rough estimation method for untrained objects. Create a pose refinement network for untrained objects when sparse point clouds are accessible.

  2. Propose a point cloud fusion and filtering method based on pose estimation. This method addresses the issue of point cloud gaps at the edge of objects when the robot collects samples from a single perspective. The point cloud fusion method is designed to utilize the output of pose estimation to fill in the gaps in the input point cloud and remove noise.

  3. Perform grasp pose estimation using the completed point cloud. Utilizing the grasping angle prediction network and a fast search strategy, the completed point cloud enhances the stability of grasping pose generation, thereby improving the success rate of grasping and its robustness.

  4. Test the system in simulation and the real world using a 6DoF robot, a Realsense camera, and a laptop (ROS/Ubuntu 20.04) as the host computer.

Autonomous Suture Robot System for Endoscopic Surgery

2023.02 - 2023.07
sym

The research objective is to enhance the efficiency and accuracy of autonomous suturing in endoscopic surgery. This involves designing a 3DOF autonomous suturing instrument, developing an efficient laparoscopic surgery robot based on 3D laparoscopy, and establishing a robot system that integrates visual, expert, and navigation systems.

Finished Works:

  1. Perform the recognition and segmentation of surgical instruments and lesion tissues based on Yolov8, and determine the spatial position of the end of the surgical instruments.

  2. Build a vision and robotic arm fusion system, perform hand-eye calibration of cameras and robotic arms, and execute basic robot actions for visual servo.

Rock Core Box Handling Robot

2022.08 - 2023.02
sym

Rock Core samples obtained from drilling before oil extraction are crucial data for assessing mining value, and they are stored in dedicated core boxes. During research and analysis, it’s essential to arrange these rock core boxes neatly on-site in a designated order, one box at a time. However, these boxes are typically heavy, resulting in a high labor intensity that can impact transportation efficiency. The present invention aims to address the labor intensity associated with the current method of transporting core boxes, ultimately improving transportation efficiency.

Finished Works:

  1. Build a sensing system that utilizes infrared sensors to detect surrounding obstacles, as well as fractional laser sensors to detect the current stacking height and alignment.

  2. Implement rock core box instance segmentation based on Mask RCNN, combined with a depth camera to determine its corner space coordinates for visual servo during the robot handling process.

  3. Utilize a monocular camera and Aruco markers to achieve 2D pose adjustment of the robot in place by identifying the offset distance and angle of the Aruco marker.

Medical Pan-Tilt Control System Based on Binocular Vision

2022.01 - 2022.06
sym

In traditional medical processes, traditional Chinese medicine practitioners often need to frequently change positions and adjust their angles to view the surgical area. Additionally, during rehabilitation treatment, the camera’s range can be inadequate to cover the patient’s moving area. Current recording methods commonly used suffer from limitations and lack automation. This article introduces a multi-degree-of-freedom pan-tilt system designed to track the positions of doctors and patients.

Project, Video

Finished Works:

  1. Design the mechanical structure and simulate the platform’s motion, using internal toothed rotary bearings to minimize the structural size. Protective structures have been designed for all sensor components.

  2. Design hardware control algorithms for pan-tilt using STM32 and A4988 drivers, and establish communication between STM32 and the host computer’s ROS system for pan-tilt motion control within ROS.

  3. Propose a target tracking method based on HOG and SIFT feature matching, capable of short-term single target tracking while maintaining robustness to changes in the target object’s appearance.

  4. Construct an experimental Pan-Tilt system and conduct experiments with all proposed algorithms, using 3D printed structural components, a Realsense camera, and stepper motors.

Spherical Environmental Information Collection Robot

2020.11 - 2021.05
sym sym

Monitoring essential information in various hazardous environments, such as petrochemical plants, chemical plants, and disaster sites, is vital for disaster prevention and reduction. In response to this need, our team has designed a spherical environment monitoring robot system. It features a spherical metamorphic structure with multiple telescopic feet that enable the robot to rotate, roll, and achieve all-round motion with high stability. These robots utilize self-organized network communication technology to offer multiple monitoring modes, and the collected data is packaged and transmitted to the upper computer system, enabling real-time dynamic monitoring of complex environmental information over a wide area.

Finished Works:

  1. Design the three-dimensional structure of the spherical robot, including the placement of controllers, sensors inside the sphere, and the spatial arrangement of leg motors.
  2. Implement control of spherical robot leg motors using STM32, calculate motion strategies, and achieve robot motion.
  3. Integrate , accelerometers, temperature and humidity sensors, gas sensors, etc., into the STM32 control system for Bluetooth-controlled robot motion and sensor data reception.

📝 Publications

📃 Papers

Robotics and Autonomous Systems
sym

Reactive Mobile Manipulation Based on Dynamic Dual-Trajectory Tracking

Gongcheng Wang, Haofei Ma, Han Wang, Pengchao Din,g Hua Bai Wenda Xu, Weidong Wang, Zhijiang Du

Paper

  • A real-time generation algorithm for spatio-temporal obstacle avoidance trajectories of EE is proposed, which satisfies both the workspace constraints and obstacle avoidance constraints of the mobile manipulator.
  • A reactive obstacle avoidance strategy is designed for nonholonomic MB obstacle avoidance, incorporating adaptive weights to ensure a smooth and effective obstacle avoidance process for the MB.
Robotics and Autonomous Systems
sym

A study of robotic search strategy for multi-radiation sources in unknown environments

Hua Bai, Wenrui Gao, Haofei Ma, Pengchao Ding, Gongcheng Wang, Wenda Xu, Weidong Wang, Zhijiang Du

Paper

  • A robot search strategy is proposed and evaluated for unknown radiation environments.
  • Combine ADE optimization with the PSPF algorithm to improve estimation efficiency.
  • An exploration scheme is proposed to balance exploration and exploitation.

📚 Patents

🏆 Honors and Awards

🏅 Honors

  • 2023.12, Leading Intelligence · Wang Yanqing Scholarship
  • 2023.10, Excellent Students of Harbin Institute of Technology
  • 2023.09, Top Grade Scholarship in Harbin Institute of Technology
  • 2022.06, Provincial Outstanding Graduates (Top 1%)
  • 2021.05, Provincial Merit Student (Top 1%)
  • 2021.06, Finely Crafted Technology Scholarship
  • 2021.01, Outstanding Student in Hebei University of Technology
  • 2020.12, National Scholarship (Top 0.3%)
  • 2020.12, Excellent Student Cadre in Hebei University of Technology
  • 2020.12, The First Prize Scholarship in Hebei University of Technology

🎏 Competitions

  • 2021.09, “Internet +” Innovation and Entrepreneurship Competition Provincial Silver Award .
  • 2021.08, E-commerce “Innovation, Creativity, and Entrepreneurship” Challenge Provincial Third Prize.
  • 2021.07, Zhou Peiyuan Mechanics Competition Provincial Second Prize and National Excellence Award.
  • 2020.12, Hebei Province College Robot Competition Seconda Prize.
  • 2020.10, Electronic Design Competition Provincial Second Prize.
  • 2020.10, iCAN International Innovation and Entrepreneurship Competition Provincial Second Prize.
  • 2020.09, Mechanical Innovation Design Competition Provincial Second Prize.
  • 2019.12, Mathematics Competition Provincial First Prize.
  • 2019.09, iCAN International Innovation and Entrepreneurship Competition Provincial Third Prize.
  • 2019.06, Mathematics Competition Provincial Second Prize.

💼 Societies

  • 2023.02 - 2023.07, School Office Assistant in Harbin Institute of Technology.
  • 2022.02 - 2022.06, Class Leader in Hebei University of Technology.
  • 2019.09 - 2022.06, Class Study Monitor in Hebei University of Technology.
  • 2020.09 - 2021.08, Director of Haier Key Maker-Lab in Hebei University of Technology.