.

.

.

.

Fundamental Research (D1)

We aim at making robotic system autonomous and bring them in real world environments and solve real-world problems. Since legislation is one of the key barriers to doing so, we also work on the development of novel teleoperation systems to assist the human operator during complex teleoperation tasks (robotic surgery, nuclear sort and segregation). This makes teleoperation a much easier and cheaper solution allowing more penetration of robotic technologies in the market.
From theoretical point of view, my research focuses on:
  1. Data driven motion planning and control (presented in Sanni 2022; and Mandil 2022 papers; and Tafuro 2022; and Bonvicini 2022 submissions)
  2. Robot learning from demonstrations (Ghalamzan 2015, Ghalamzan 2018 and Osa 2017)
  3. Haptic enabled teleoperation and   (Parsa et al. 2020--Best Paper Award Finalist at IEEE CASE 2020, Parsa et al. 2022, Ghalamzan et al. 2017 and Selvagio et al. 2019);
  4. Grasping and manipulation planning/control (Mavrakis et al. 2016--Best Interactive Paper Award Finalist at IEEE Humanoids 2016, Mavrakis et al. 2017, Ghalamzan et al. 2016, Pardi et al. 2018 and 2021),
  5. Robot Perception (Tafuro ICRA 2022);

Robot Learning from Demonstrations (LfD)

LfD is key to bringing robots to human workspaces at scale. It also can help reduce programming time and cost so that non-experts can easily programme a robot for different tasks. The links between robot and human LfD (Ghalamzan 2015 Incremental and Ghalamzan 2018} demonstrate the available technologies are far away from the corresponding human capabilities of learning from observations. We have been working on developing LfD approaches for a wide range of applications, e.g. Breast Cancer Examination robot (Sanni 2022), Strawberry picking (Alessandra 2022), simple pick-and-place (Ghalamzan 2015 incremental and Ghalamzan 2018) and nuclear sort and segregation (Osa 2017 guiding). We are keen to develop fundamental concepts to make LfD available for non-expert users across several application domains.

Data-driven predictive models (DPM)

: Most manipulation tasks involve interaction with environments. These interactions are always challenging to the model via classical approaches; hence, they are complicated to control. We are interested in developing data-driven predictive forward models that can help a robot to interact with its environment. For instance, Nazari et al. 2021 and Willow et al. 2022 present action conditioned predictive models for tactile sensors. DPM can be used for, e.g., robust object manipulation by slip avoidance controller via model predictive control (MPC). It can be also used for cluster manipulation for picking up an object on the shelf or strawberry picking (Mghames et al. 2020). Predictive models are a core part of a robust interactive robotic task.

Reduced cognitive load teleoperation

Fully teleoperated robotic systems are the sole solutions to many problems, e.g. from robotic surgery to nuclear sort & segregation. These conservative industries do not trust a fully autonomous system even if such a system is available and needs a human operator to be in charge of the task execution. We worked on a new technology of shared control which can bridge the gap between autonomous systems and fully teleoperated systems. We proposed to use haptic force cues as a means to guide human operators. The guidance in teleoperation can provide a human operator with optimal actions in using a surgical robot (Selvaggio et al. 2019) or predicted singularities (Ghalamzan et al. 2017}, and collision (Parsa et al. 2020} and minimum required torques (Rahal et al. 2022). Haptic guidance is important in reducing the cognitive workload (Talha et al. 2016). Motion scaling is also a factor in reducing the cognitive workload (Parsa et al. 2022). We envisage an Intelligent Tele-operation System benefiting from Intelligent Autonomous System via haptic guidance or adaptive motion scaling. That can increase the penetration of robotic technologies in the market and make them cheaper solutions, e.g. it can significantly reduce the operation time of robotic surgery.

Robotic grasping and manipulation

Our work in grasping and manipulation is manifold: (1) task-informed grasping (TIG); (2) Motion planning; (3) Motion Control. The common practice of robotic grasping includes an intelligent system extracting grasp related features from RGB-D images. This alone is not enough for effective and robust robotic grasping as the synthesised grasp may not be good enough for a specific manipulator. Task-informed grasping considers robot specific objectives into consideration in selecting the grasp poses. Hence, the robot can predict the consequences of the grasp actions (e.g. singularity (Ghalamzan et al. 2016), torques (Mavrakis et al. 2016), torques (Mavrakis et al. 2017), collision (Pardi et al. 2018) and affordance (Pardi et al. 2021) during the planned manipulative movements of a follower manipulator. We also work on motion planning for interactive robotic manipulation motion planning and control are needed. For effective motion planning for robotic cutting, one should define the robot movements based on the surface of the object to be cut (Pardi et al. 2020). For robotic breast palpation (to localise the cancerous lumps) motion control and searching policy, such as RL, are required (Giorgo et al. 2022).

Manipulation Perception

Our work in Manipulation Perception is manifold: (1) Strawberry detection-segmentation and picking point localisation; (2) Active Perception; (3) Tactile Perception. We work on Detection, Segmentation, and Localisation for robotic grasping, and fruit picking (Tafuro et al. 2022). We also work on Active Perception which allows a robot to take action to optimise the perception performaence. We also use tactile sensors for the robot to make the robot aware of the interction states.

Related Publications

Published by IML members

                @article{giorgo2022,
                    title={Deep Movement Primitives and control policy for breast cancer examination robot},
                    author={Bonvicini, Giorgio and Sprida, Venkatesh and Zanchettin, Andrea and Ghalamzan, Amir},
                    journal={Robotic Automation Letter (IROS) submitted},
                    year={2022}
                  }

                @inproceedings{Mavrakis2016,
                    doi = {10.1109/humanoids.2016.7803274},
                    url = {https://doi.org/10.1109/humanoids.2016.7803274},
                    year = {2016},
                    month = nov,
                    publisher = {{IEEE}},
                    author = {Nikos Mavrakis and Amir M. Ghalamzan E. and Rustam Stolkin and Luca Baronti and Marek Kopicki and Marco Castellani},
                    title = {Analysis of the inertia and dynamics of grasped objects,  for choosing optimal grasps to enable torque-efficient post-grasp manipulations},
                    booktitle = {2016 {IEEE}-{RAS} 16th International Conference on Humanoid Robots (Humanoids)}
                }

                @inproceedings{Sanni2022,
                    title={Deep robot path planning from demonstrations for breast cancer examination},
                    author=Sanni, Oluwatoin and Bonvicini, Giorgio and Khan, Arshad, MA Khan and Lopez-Custodio, Pablo and Nazari, Kiyanoush and Ghalamzan E., Amir,
                    booktitle=Proceedings of 36th AAAI conference on Artificial Intelligence,
                    year={2022},
                    url={https://arxiv.org/pdf/2202.09265.pdf}
                }
                @inproceedings{Mandil2022,
                    title={Action Conditioned Tactile Prediction: case study on slip prediction},
                    author=Mandill, Willow and Nazari, Kiyanoush and Ghalamzan E., Amir,
                    booktitle=Proceedings of Robotics: Science and Systems (RSS),
                    year={2022},
                    url={https://arxiv.org/pdf/2202.09265.pdf}
                }
                @inproceedings{Parsal2022,
                    title={The Impact of Motion Scaling and Haptic Guidance on Operators’ Workload and Performance in Teleoperation},
                    author=Parsa, Soran and Maior, Horia and Thumwood, Alex and Wilson, Max and Ghalamzan E., Amir,
                    booktitle=CHI Conference on Human Factors in Computing Systems,
                    year={2022},
                    url={https://arxiv.org/pdf/2202.09265.pdf}
                }
                @inproceedings{Tafurol2022,
                    title={Strawberry picking pont, wieght and quality estimation},
                    author=Tafuro, Alessandra and Adewumi, Adeayo and Parsa, Soran and Ghalamzan E., Amir and Debnat, Bappaditya,
                    booktitle=IEEE Conference on Robotics and Automation (ICRA),
                    year={2022},
                    url={https://arxiv.org/pdf/2202.09265.pdf}
                }
                @inproceedings{Ghalamzanl2022,
                    title={Solar Energy Technology in Precision Agriculture and Smart Farming},
                    author=Ghalamzan E., Amir and et al.,
                    booktitle=Elsevier book on Solar Energy,
                    year={2022},
                    url={}
                }
                @inproceedings{Nazari2021,
                    title={Tactile Dynamic Behaviour Prediction Based on Robot Action},
                    author={Nazari, Kiyanoush and Mandil, Willow and Hanheide, Marc and Ghalamzan, Amir},
                    booktitle={Proceedings of TAROS 2021},
                    year={2021},
                    url={https://lcas.lincoln.ac.uk/wp/wp-content/uploads/2021/09/t5.3-TAROS2021paper18.pdf}
                }
                @inproceedings{crivellari2021deep,
                    title={Deep Robot Path Planning from Demonstrations for Breast Cancer Examination},
                    author={Crivellari, Marta and Sanni, Oluwatoyin and Zanchettin, Andrea and Esfahani, Amir Ghalamzan},
                    booktitle={Annual Conference Towards Autonomous Robotic Systems},
                    pages={260--272},
                    year={2021},
                    organization={Springer}
                  }
                @inproceedings{esfahani2021tactile,
                    title={Tactile Dynamic Behaviour Prediction Based on Robot Action},
                    author={Nazari, Kiyanoush and Mandil, Willow and Ghalamzan, Amir },
                    booktitle={Towards Autonomous Robotic Systems: 22nd Annual Conference, TAROS 2021, Lincoln, UK, September 8--10, 2021, Proceedings},
                    volume={13054},
                    pages={284},
                    year={2021},
                    organization={Springer Nature}
                }
                @article{ghalamzan2021deep,
                    title={Deep-LfD: deep robot learning from demonstrations},
                    author={Ghalamzan, Amir and Nazari, Kiyanoush and Hashempour, Hamidreza and Zhong, Fangxun},
                    journal={Software Impacts},
                    volume={9},
                    pages={100087},
                    year={2021},
                    publisher={Elsevier}
                  }

                  @article{pardi2020planning,
                    title={Planning maximum-manipulability cutting paths},
                    author={Pardi, Tommaso and Ortenzi, Valerio and Fairbairn, Colin and Pipe, Tony and Esfahani, Amir Masoud Ghalamzan and Stolkin, Rustam},
                    journal={IEEE Robotics and Automation Letters},
                    volume={5},
                    number={2},
                    pages={1999--2006},
                    year={2020},
                    publisher={IEEE}
                  }

                @inproceedings{Mavrakis2020,
                    doi = {10.1109/iros45743.2020.9341112},
                    url = {https://doi.org/10.1109/iros45743.2020.9341112},
                    year = {2020},
                    month = oct,
                    publisher = {{IEEE}},
                    author = {Nikos Mavrakis and Amir M. Ghalamzan E. and Rustam Stolkin},
                    title = {Estimating An Object's Inertial Parameters By Robotic Pushing: A Data-Driven Approach},
                    booktitle = {2020 {IEEE}/{RSJ} International Conference on Intelligent Robots and Systems ({IROS})}
                }
                @inproceedings{Parsa2020,
                    doi = {10.1109/case48305.2020.9216789},
                    url = {https://doi.org/10.1109/case48305.2020.9216789},
                    year = {2020},
                    month = aug,
                    publisher = {{IEEE}},
                    author = {Soran Parsa and Disha Kamale and Sariah Mghames and Kiyanoush Nazari and Tommaso Pardi and Aravinda R. Srinivasan and Gerhard Neumann and Marc Hanheide and Ghalamzan E. Amir},
                    title = {Haptic-guided shared control grasping: collision-free manipulation},
                    booktitle = {2020 {IEEE} 16th International Conference on Automation Science and Engineering ({CASE})}
                }
                @inproceedings{Mghames2020,
                    doi = {10.1109/iros45743.2020.9341728},
                    url = {https://doi.org/10.1109/iros45743.2020.9341728},
                    year = {2020},
                    month = oct,
                    publisher = {{IEEE}},
                    author = {Sariah Mghames and Marc Hanheide and Amir Ghalamzan E.},
                    title = {Interactive Movement Primitives: Planning to Push Occluding Pieces for Fruit Picking},
                    booktitle = {2020 {IEEE}/{RSJ} International Conference on Intelligent Robots and Systems ({IROS})}
                }
                @inproceedings{Selvaggio2019,
                    doi = {10.1109/iros40897.2019.8968109},
                    url = {https://doi.org/10.1109/iros40897.2019.8968109},
                    year = {2019},
                    month = nov,
                    publisher = {{IEEE}},
                    author = {Mario Selvaggio and Amir M. Ghalamzan E and Rocco Moccia and Fanny Ficuciello and Bruno Siciliano},
                    title = {Haptic-guided shared control for needle grasping optimization in minimally invasive robotic surgery},
                    booktitle = {2019 {IEEE}/{RSJ} International Conference on Intelligent Robots and Systems ({IROS})}
                }
                @article{GhalamzanE2018,
                    doi = {10.1016/j.robot.2017.12.001},
                    url = {https://doi.org/10.1016/j.robot.2017.12.001},
                    year = {2018},
                    month = mar,
                    publisher = {Elsevier {BV}},
                    volume = {101},
                    pages = {45--56},
                    author = {Amir M. Ghalamzan E. and Matteo Ragaglia},
                    title = {Robot learning from demonstrations: Emulation learning in environments with moving obstacles},
                    journal = {Robotics and Autonomous Systems}
                }
                @article{Osa2017,
                    doi = {10.1109/lra.2017.2653850},
                    url = {https://doi.org/10.1109/lra.2017.2653850},
                    year = {2017},
                    month = apr,
                    publisher = {Institute of Electrical and Electronics Engineers ({IEEE})},
                    volume = {2},
                    number = {2},
                    pages = {819--826},
                    author = {Takayuki Osa and Amir M. Ghalamzan Esfahani and Rustam Stolkin and Rudolf Lioutikov and Jan Peters and Gerhard Neumann},
                    title = {Guiding Trajectory Optimization by Demonstrated Distributions},
                    journal = {{IEEE} Robotics and Automation Letters}
                }
                @inproceedings{Ghalamzan2017,
                    doi = {10.1109/iros.2017.8206178},
                    url = {https://doi.org/10.1109/iros.2017.8206178},
                    year = {2017},
                    month = sep,
                    publisher = {{IEEE}},
                    author = {E. Amir M. Ghalamzan and Firas Abi-Farraj and Paolo Robuffo Giordano and Rustam Stolkin},
                    title = {Human-in-the-loop optimisation: Mixed initiative grasping for optimally facilitating post-grasp manipulative actions},
                    booktitle = {2017 {IEEE}/{RSJ} International Conference on Intelligent Robots and Systems ({IROS})}
                }
                @inproceedings{Mavrakis2017,
                    doi = {10.1109/iros.2017.8206258},
                    url = {https://doi.org/10.1109/iros.2017.8206258},
                    year = {2017},
                    month = sep,
                    publisher = {{IEEE}},
                    author = {Nikos Mavrakis and E. Amir M. Ghalamzan and Rustam Stolkin},
                    title = {Safe robotic grasping: Minimum impact-force grasp selection},
                    booktitle = {2017 {IEEE}/{RSJ} International Conference on Intelligent Robots and Systems ({IROS})}
                }
                @inproceedings{Amir2017,
                    doi = {10.1109/icrom.2017.8466161},
                    url = {https://doi.org/10.1109/icrom.2017.8466161},
                    year = {2017},
                    month = oct,
                    publisher = {{IEEE}},
                    author = {M. Amir and E. Ghalamzan and Nikos Mavrakis and Rustam Stolkin},
                    title = {Grasp that optimises objectives along post-grasp trajectories},
                    booktitle = {2017 5th {RSI} International Conference on Robotics and Mechatronics ({ICRoM})}
                }
                @inproceedings{GhalamzanE2016,
                    doi = {10.1109/iros.2016.7759158},
                    url = {https://doi.org/10.1109/iros.2016.7759158},
                    year = {2016},
                    month = oct,
                    publisher = {{IEEE}},
                    author = {Amir M. Ghalamzan E. and Nikos Mavrakis and Marek Kopicki and Rustam Stolkin and Ales Leonardis},
                    title = {Task-relevant grasp selection: A joint solution to planning grasps and manipulative motion trajectories},
                    booktitle = {2016 {IEEE}/{RSJ} International Conference on Intelligent Robots and Systems ({IROS})}
                }
                @inproceedings{Talha2016,
                    doi = {10.1109/ssrr.2016.7784294},
                    url = {https://doi.org/10.1109/ssrr.2016.7784294},
                    year = {2016},
                    month = oct,
                    publisher = {{IEEE}},
                    author = {M. Talha and E. A. M. Ghalamzan and C. Takahashi and J. Kuo and W. Ingamells and R. Stolkin},
                    title = {Towards robotic decommissioning of legacy nuclear plant: Results of human-factors experiments with tele-robotic manipulation,  and a discussion of challenges and approaches for decommissioning},
                    booktitle = {2016 {IEEE} International Symposium on Safety,  Security,  and Rescue Robotics ({SSRR})}
                }
                @inproceedings{GhalamzanE2015,
                    doi = {10.1109/icra.2015.7139467},
                    url = {https://doi.org/10.1109/icra.2015.7139467},
                    year = {2015},
                    month = may,
                    publisher = {{IEEE}},
                    author = {Amir M. Ghalamzan E. and Luca Bascetta and Marcello Restelli and Paolo Rocco},
                    title = {Estimating a Mean-Path from a set of 2-D curves},
                    booktitle = {2015 {IEEE} International Conference on Robotics and Automation ({ICRA})}
                }
                @inproceedings{GhalamzanE2015,
                    doi = {10.1109/icra.2015.7139985},
                    url = {https://doi.org/10.1109/icra.2015.7139985},
                    year = {2015},
                    month = may,
                    publisher = {{IEEE}},
                    author = {Amir M. Ghalamzan E and Chris Paxton and Gregory D. Hager and Luca Bascetta},
                    title = {An incremental approach to learning generalizable robot tasks from human demonstration},
                    booktitle = {2015 {IEEE} International Conference on Robotics and Automation ({ICRA})}
                }            
                @article{GhalamzanE2021,
                    doi = {10.1016/j.simpa.2021.100087},
                    url = {https://doi.org/10.1016/j.simpa.2021.100087},
                    year = {2021},
                    month = aug,
                    publisher = {Elsevier {BV}},
                    volume = {9},
                    pages = {100087},
                    author = {Amir Ghalamzan E. and Kiyanoush Nazari and Hamidreza Hashempour and Fangxun Zhong},
                    title = {Deep-{LfD}: Deep robot learning from demonstrations},
                    journal = {Software Impacts}
                }
                @INCOLLECTION {book,
                    author    = "Amir Ghalamzan Esfahani and Gautham P Das and Iain Gould and Payam Zarafshan and Rajendran S Vishnu and James Heselden and Amir Badiee and Isobel Wright and Simon Pearson",
                    title     = "Applications of Solar Energy in Precision Agriculture and Smart Farming",
                    booktitle = "Solar Energy Advancements in Agriculture and Food Production Systems",
                    publisher = "Elsevier",
                    year      = "In press",
                    chapter   = "10"
                }
            

intelligent manipulation laboratory

Oxford University
University of Bristol
University of Birmingham
London Imperial College
Queen Mary University
Polytechnic University of Milan