Research

Perception and Navigation in Agricultural Robotics

In the last years, I addressed several research topics in the field of agricultural robotics, in most cases within the Flourish research project (European Community's Horizon 2020 programme).
Flourish is a project of precision agriculture that aims to reduce the amount of herbicides used to control weeds by means of the use of robotics and artificial intelligence technologies. A flying robot (UAV) and an autonomous ground vehicle (UGV) cooperate in an almost fully automated way to monitor the crop and to precisely remove the weeds. The UAV is in charge to fly over and inspect the crops from the sky, the UGV uses the information provided by the UAV in order to reach and remove the detected weeds.






In this context, at Sapienza, University of Rome we developed novel solutions to tackle effectively different perception and navigation tasks, among others, crop and weed classification with convolutional neural networks and synthetic datasets, robust global localization with multi-sensor data fusion, and UAV-UGV collaborative 3D environment reconstruction.




We also made available the datasets we collected, and some software packages we developed at Sapienza within the Flourish project:


Related Papers:

A. Pretto, S. Aravecchia, W. Burgard, N. Chebrolu, C. Dornhege, T. Falck, F. Fleckenstein, A. Fontenla, M. Imperoli, R. Khanna, F. Liebisch, P. Lottes, A. Milioto, D. Nardi, S. Nardi, J. Pfeifer, M. Popović, C. Potena, C. Pradalier, E. Rothacker-Feder, I. Sa, A. Schaefer, R. Siegwart, C. Stachniss, A. Walter, W. Winterhalter, X. Wu and J. Nieto Building an Aerial-Ground Robotics System for Precision Farming: An Adaptable Solution In: IEEE Robotics & Automation Magazine, Volume: 28, Issue: 3, 2021
(PDF, BibTeX) 

@article{prettoRAM2021,
    author={Alberto Pretto and St{\'e}phanie Aravecchia and 
            Wolfram Burgard and Nived Chebrolu and
            Christian Dornhege and Tillmann Falck and
            Freya Fleckenstein and Alessandra Fontenla and 
            Marco Imperoli and Raghav Khanna and Frank Liebisch and
            Philipp Lottes and Andres Milioto and Daniele Nardi and
            Sandro Nardi and Johannes Pfeifer and 
            Marija Popovi{\'{c}} and Ciro Potena and 
            C{\'e}dric Pradalier and Elisa Rothacker-Feder and
            Inkyu Sa and Alexander Schaefer and Roland Siegwart and 
            Cyrill Stachniss and Achim Walter and 
            Wera Winterhalter and Xiaolong Wu and Juan Nieto},
  journal={IEEE Robotics \& Automation Magazine}, 
  title={Building an Aerial-Ground Robotics System for 
         Precision Farming: An Adaptable Solution}, 
  year={2021},
  volume={28},
  number={3},
  pages={29--49},
  doi={10.1109/MRA.2020.3012492}}


M. Fawakherji, C. Potena, A. Pretto, D.D. Bloisi, and D. Nardi Multi-Spectral Image Synthesis for Crop/Weed Segmentation in Precision Farming In: Robotics and Autonomous Systems, Volume 146, December 2021
(PDF, code, BibTeX) 

@article{fppbnRAS2021,
title = {Multi-Spectral Image Synthesis for Crop/Weed
Segmentation in Precision Farming},
author = {Mulham Fawakherji and Ciro Potena and 
Alberto Pretto and Domenico D. Bloisi and Daniele Nardi},
journal = {Robotics and Autonomous Systems},
volume = {146},
pages = {103861},
year = {2021},
issn = {0921-8890},
doi = {https://doi.org/10.1016/j.robot.2021.103861}}


M. Fawakherji, A. Youssef, D. D. Bloisi, A. Pretto, and D. Nardi Crop and Weed Classification Using Pixel-wise Segmentation on Ground and Aerial Images In: International Journal of Robotic Computing, Vol: 2, Issue: 1, April 2020, pages 39-57
(PDF, BibTeX) 
@article{fybpn_IJRC2020,
  title={Crop and Weed Classification Using Pixel-wise 
         Segmentation on Ground and Aerial Images},
  author={Fawakherji, Mulham and Youssef,Ali and Bloisi, 
          Domenico D. and Pretto, Alberto and Nardi, Daniele},
  journal={International Journal of Robotic Computing},
  volume    = {2},
  number    = {1},
  year      = {2020},
  pages     = {39--57},
  doi={10.35708/RC1869-126258}
}


C. Potena, R. Khanna, J. Nieto, R. Siegwart, D. Nardi, and A. Pretto AgriColMap: Aerial-Ground Collaborative 3D Mapping for Precision Farming In: IEEE Robotics and Automation Letters, Vol: 4, Issue: 2, April 2019, pages 1085-1092
(PDF, video, code, datasets, BibTeX) 
@article{pknsnp_RA-L2019,
  title={{A}gri{C}ol{M}ap: {A}erial-Ground Collaborative {3D} Mapping
         for Precision Farming},
  author={Potena, Ciro and Khanna, Raghav and Nieto, Juan and
          Siegwart, Roland and Nardi, Daniele and Pretto, Alberto},
  journal={IEEE Robotics and Automation Letters},
  volume    = {4},
  number    = {2},
  year      = {2019},
  pages     = {1085--1092},
  doi={10.1109/LRA.2019.2894468}
}


M. Fawakherji, A. Youssef, D.D. Bloisi, A. Pretto, and D. Nardi Crop and Weeds Classification for Precision Agriculture using Context-Independent Pixel-Wise Segmentation In: Proc. of the IEEE International Conference on Robotic Computing (IRC), 2019
(PDF, BibTeX) 
@inproceedings{fybpn_IRC_2019,
  author={Fawakherji, Mulham and Youssef,Ali and Bloisi, Domenico D. 
	  and Pretto, Alberto and Nardi, Daniele},
  title     = {Crop and Weeds Classification for
               Precision Agriculture using Context-Independent 
	       Pixel-Wise Segmentation},
  booktitle = {Proc. of the {IEEE} International Conference on Robotic 
               Computing ({IRC})},
  year      = {2019},
  doi={10.1109/IRC.2019.00029}
}


M. Imperoli, C. Potena, D. Nardi, G. Grisetti and A. Pretto An Effective Multi-Cue Positioning System for Agricultural Robotics In: IEEE Robotics and Automation Letters, Vol: 3, Issue: 4, October 2018, pages 3685-3692
(PDF,video, code, datasets, BibTeX) 
@article{ipngp_RA-L2018,
  title={An Effective Multi-Cue Positioning System for 
         Agricultural Robotics},
  author={Imperoli, Marco and Potena, Ciro and Nardi, Daniele and 
          Grisetti, Giorgio and Pretto, Alberto},
  journal={IEEE Robotics and Automation Letters},
  volume    = {3},
  number    = {4},
  year      = {2018},
  pages     = {3685--3692},
  doi={10.1109/LRA.2018.2855052}
}


M. Di Cicco, C. Potena, G. Grisetti and A. Pretto Automatic Model Based Dataset Generation for Fast and Accurate Crop and Weeds Detection Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017
(PDF, video, datasets, BibTeX) 
@inproceedings{dpgp_IROS2017,
  author    = {Di Cicco, Maurilio and Potena, Ciro and
               Grisetti, Giorgio and Pretto, Alberto},
  title     = {Automatic Model Based Dataset Generation for Fast and 
               Accurate Crop and Weeds Detection},
  booktitle = {Proc. of the {IEEE/RSJ} International Conference on
               Intelligent Robots and Systems ({IROS})},
  year      = {2017}
}


C. Potena, D. Nardi and A. Pretto Fast and Accurate Crop and Weed Identification with Summarized Train Sets for Precision Agriculture In: Proceedings of the 14th International Conference on Intelligent Autonomous Systems (IAS-14), July 3-7 , 2016 Shanghai, China - BEST STUDENT PAPER AWARD - FINALIST
(PDF, BibTeX) 
@inproceedings{pnp_ias2016,
  author={Potena, C. and Nardi, D. and Pretto, A.},
  title={Fast and Accurate Crop and Weed Identification
         with Summarized Train Sets for Precision Agriculture},
  booktitle={Proc. of the 14th International Conference on 
             Intelligent Autonomous Systems (IAS-14)},
  year={2016}
}


Sensors Calibration

The calibration of sensors and sensor ensembles is an active task in my research activity. Among others, we have developed a novel protocol to calibrate Inertial measurement units (IMUs) based on the multi-position scheme, and a novel calibration framework to calibrate both the intrinsic and extrinsic parameters of a general color-depth sensor couple.



Related Papers:

F. Basso, E. Menegatti and A. Pretto Robust Intrinsic and Extrinsic Calibration of RGB-D Cameras In: IEEE Transactions on Robotics, Vol: 34, Issue: 5, Oct. 2018, pages 1315-1332
(PDF,code, BibTeX) 
@article{bmp_T-RO2018,
  title={Robust Intrinsic and Extrinsic Calibration of {RGB-D} Cameras},
  author={Basso, Filippo and Menegatti, Emanuele and Pretto, Alberto},
  journal={IEEE Transactions on Robotics}, 
  title={Robust Intrinsic and Extrinsic Calibration of RGB-D Cameras}, 
  year={2018}, 
  volume={34}, 
  number={5}, 
  pages={1315-1332}, 
  doi={10.1109/TRO.2018.2853742}
}


A. Pretto and G. Grisetti Calibration and performance evaluation of low-cost IMUs In: Proceedings of the 20th IMEKO TC4 International Symposium, Sep. 15 - 17, 2014 Benevento, Italy, pages: 429 - 434
(PDF,code, BibTeX) 
@inproceedings{pg_imeko2014,
  title={Calibration and performance evaluation of low-cost IMUs},
  author={Pretto, A. and Grisetti, G.},
  booktitle={Proc. of: 20th IMEKO TC4 International Symposium},
  year={2014},
  pages={429--434}
}


D. Tedaldi, A. Pretto and E. Menegatti A Robust and Easy to Implement Method for IMU Calibration without External Equipments In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2014), May 31 - June 7, 2014 Hong Kong, China, Page(s): 3042 - 3049
(PDF,code, BibTeX) 
@inproceedings{tpm_icra2014,
  title={A Robust and Easy to Implement Method for IMU Calibration 
            without External Equipments},
  author={Tedaldi, A. and Pretto, A. and Menegatti, E.},
  booktitle={Proc. of: IEEE International Conference on Robotics and
                   Automation (ICRA)},
  year={2014},
  pages={3042--3049}
}


F. Basso, A. Pretto and E. Menegatti Unsupervised Intrinsic and Extrinsic Calibration of a Camera-Depth Sensor Couple In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2014), May 31 - June 7, 2014 Hong Kong, China, Page(s): 6244 - 6249
(PDF, code, BibTeX) 
@inproceedings{bassoICRA2014,
  title={Unsupervised Intrinsic and Extrinsic Calibration of a 
           Camera-Depth Sensor Couple},
  author={Basso, F. and Pretto, A. and Menegatti, E.},
  booktitle={Proc. of: IEEE International Conference on Robotics and 
                   Automation (ICRA)},
  year={2014},
  pages={6244--6249}
}


Perception in Industrial Robotics

I'm working in the field of industrial robotics addressing several perception related problems, among others object detection and localization, active perception, sensor design, and object manipulation. In this context, I also coordinated the FlexSight (Flexible and Accurate Recognition and Localization System of Deformable Objects for Pick&Place Robots) research project, funded by the European Community's project ECHORD++. The goal of FlexSight is to design a perception system based on an integrated smart camera (the FlexSight sensor, FSS) that is able to recognize and localize several types of deformable objects that can be commonly found in many industrial and logistic applications.












Related Papers:

D. Evangelista, M. Imperoli, E. Menegatti, and A. Pretto FlexSight - A Flexible and Accurate System for Object Detection and Localization for Industrial Robots In: Proc. of the IEEE International Workshop on Metrology for Industry 4.0 and IoT, 2019
(PDF, BibTeX) 
@inproceedings{eip_METROIND_2019,
  author={Evangelista, Daniele and Imperoli, Marco and
          and Menegatti, Emanuele and Pretto, Alberto},
  title     = {{FlexSight} - {A} Flexible and Accurate System for Object
               Detection and Localization for Industrial Robots},
  booktitle = {Proc. of the {IEEE} International Workshop on Metrology
               for Industry 4.0 and {IoT}},
  year      = {2019},
  doi={10.1109/METROI4.2019.8792902}
}


D. Evangelista, M. Imperoli, E. Menegatti, and A. Pretto Machine Vision for Embedded Devices: from Synthetic Object Detection to Pyramidal Stereo Matching In: Proc. of the Austrian Robotics and OAGM Workshop, 2019
(PDF, BibTeX) 
@inproceedings{eimp_ARW-OAGM2019,
  author={Evangelista, Daniele and Imperoli, Marco and Menegatti, 
          Emanuele and Pretto, Alberto},
  title     = {Machine Vision for Embedded Devices: from Synthetic 
               Object Detection to Pyramidal Stereo Matching},
  booktitle = {Proc. of the ARW & OAGM Workshop},
  year      = {2019},
  doi={10.3217/978-3-85125-663-5-08}
}


D. Evangelista, W. U. Villa, M. Imperoli, A. Vanzo, L. Iocchi, D. Nardi and A. Pretto Grounding Natural Language Instructions in Industrial Robotics Proc. of the IEEE/RSJ IROS Workshop "Human-Robot Interaction in Collaborative Manufacturing Environments (HRI-CME)", 2017
(PDF, BibTeX) 
@inproceedings{evan_HRI-CME2017,
  author    = {Evangelista, Daniele and Villa, Wilson Umberto and 
               Imperoli, Marco and Vanzo, Andrea and Iocchi, Luca and
               Nardi, Daniele and Pretto, Alberto},
  title     = {Grounding Natural Language Instructions in 
               Industrial Robotics},
  booktitle = {Proc. of the IEEE/RSJ IROS Workshop "Human-Robot Interaction
               in Collaborative Manufacturing Environments (HRI-CME)"},
  year      = {2017}
}


M. Imperoli and A. Pretto D2CO: Fast and Robust Registration of 3D Textureless Objects Using the Directional Chamfer Distance In: Proceedings of the 10th International Conference on Computer Vision Systems (ICVS 2015), July 6-9 , 2015 Copenhagen, Denmark, pages: 316-328
(PDF,code, BibTeX) 
@inproceedings{miap_icvs2015,
  author={Imperoli, M. and Pretto, A.},
  title={{D\textsuperscript{2}CO}: Fast and Robust Registration of {3D}
         Textureless Objects Using the {Directional 
         Chamfer Distance}},
  booktitle={Proc. of 10th International Conference on 
             Computer Vision Systems (ICVS 2015)},
  year={2015},
  pages={316--328}
}


M. Imperoli and A. Pretto Active Detection and Localization of Textureless Objects in Cluttered Environments In arXiv preprint arXiv:1603.07022, 2016
(PDF,video,code, BibTeX) 
@article{miap_arxiv2016,
  title={Active Detection and Localization of Textureless Objects 
         in Cluttered Environments},
  author={Imperoli, Marco and Pretto, Alberto},
  journal={arXiv preprint arXiv:1603.07022},
  year={2016}
}


A. Pretto, S. Tonello, E. Menegatti Flexible 3D Localization of Planar Objects for Industrial Bin-Picking with Monocamera Vision System In: IEEE International Conference on Automation Science and Engineering (IEEE CASE 2013), Madison, Wisconsin, (USA), August 17-21, 2013
(PDF, BibTeX) 
@inproceedings{prettoCASE2013,
  title={Flexible 3D Localization of Planar Objects for Industrial 
	 Bin-Picking with Monocamera Vision System},
  author={Pretto, A. and Tonello, S. and Menegatti, E.},
  booktitle={Proc. of: IEEE International Conference on 
           Automation Science and Engineering (CASE)},
  year={2013},
  pages={168 -- 175}
}


UAV perception and control

Recently I'm addressing the problem of vision based navigation of autonomous UAVs. In particular, we have proposed new control solutions based on the Non-linear Model Predictive Controller (NMPC) that aim to enable an UAV to follow a trajectory while constantly facing a target object or landmark, and to reduce the discretization error of the NMPC while keeping bounded the number of discretization points.



Related Papers:

C. Potena, D. Nardi, and A. Pretto Joint Vision-Based Navigation, Control and Obstacle Avoidance for UAVs in Dynamic Environments In: Proc. of the European Conference on Mobile Robots (ECMR), 2019
(PDF, code, BibTeX) 
@inproceedings{pnp_ECMR2019,
  author    = {Potena, Ciro and
               Nardi, Daniele and
               Pretto, Alberto},
  title     = {Joint Vision-Based Navigation, Control and Obstacle
               Avoidance for {UAVs} in Dynamic Environments},
  booktitle = {Proc. of the European Conference on Mobile Robots
               ({ECMR})},
  year      = {2019},
  doi={10.1109/ECMR.2019.8870944}
}


C. Potena, B. Della Corte, D. Nardi, G, Grisetti and A. Pretto Non-Linear Model Predictive Control with Adaptive Time-Mesh Refinement In: IEEE International Conference on Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR), 2018 - BEST STUDENT PAPER AWARD - FINALIST
(PDF, BibTeX) 
@inproceedings{pdcSIMPAR2018,
  author    = {Potena, Ciro and Della Corte, Bartolo and
               Nardi, Daniele and Grisetti, Giorgio and 
               Pretto, Alberto},
  title     = {Non-Linear Model Predictive Control with Adaptive 
               Time-Mesh Refinement},
  booktitle = {{IEEE} {I}nternational {C}onference on {S}imulation,
               {M}odeling and {P}rogramming for {A}utonomous {R}obots 
               ({SIMPAR})},
  year      = {2018}
}


C. Potena, D. Nardi and A. Pretto Effective Target Aware Visual Navigation for UAVs Proc. of the European Conference on Mobile Robots (ECMR), 2017
(PDF, BibTeX) 
@inproceedings{potenaECMR2017,
  author    = {Potena, Ciro and Nardi, Daniele and
               Pretto, Alberto},
  title     = {Effective Target Aware Visual Navigation for UAVs},
  booktitle = {Proc. of the European Conference on Mobile Robots
               ({ECMR})},
  year      = {2017}
}


For an overview of the past research projects, go to the Old Projects section



print
 
Latest News

  • May 2022: Our paper Building an Aerial-Ground Robotics System for Precision Farming: An Adaptable Solution won the 2022 IEEE Robotics & Automation Magazine Best Paper Award!

  • December 2021: I started coordinating the SubEye (Subsea Perception for New Generation Underwater Vehicles) research project

  • August 2021: Our paper Multi-Spectral Image Synthesis for Crop/Weed Segmentation in Precision Farming has been accepted for publication in Robotics and Autonomous Systems!

  • July 2021: Our paper Receding Horizon Task and Motion Planning in Changing Environments has been accepted for publication in Robotics and Autonomous Systems!