Soft robotic arms have shown great potential toward applications to human daily lives, which is mainly due to their infinite passive degrees of freedom and intrinsic safety. There are tasks in lives that require the motion of the robot to meet some certain pose constraints that have not been implemented through the soft arm, like delivering a glass of water. Because the workspace of the soft arm is affected by the loads or interaction, it is difficult to implement this task through the motion planning method. In this letter, we propose a Q-learning based approach to address the problem, directly achieving motion control with constraints under loads and interaction without planning. We first generate a controller for the soft arm based on Q-learning, which can operate the arm while satisfying the pose constraints when the arm is neither loaded nor interacted with the environment. Then, we introduce a process that adjusts corresponding Q values in the controller, which allows the controller to operate the arm with an unknown load or interaction while still satisfying the pose constraints. We implement the approach on our soft arm, i.e., the Honeycomb Pneumatic Network (HPN) Arm. The experiments show that the approach is effective, even when the arm reached an untrained situation or even beyond the workspace under the interaction.

@ARTICLE{9851517,
    author={Gan, Yinghao and Li, Peijin and Jiang, Hao and Wang, Gaotian and Jin, Yusong and Chen, Xiaoping and Ji, Jianmin},
    journal={IEEE Robotics and Automation Letters}, 
    title={A Reinforcement Learning Method for Motion Control With Constraints on an HPN Arm}, 
    year={2022},
    volume={7},
    number={4},
    pages={12006-12013},
    keywords={Motion control;Q-learning;Manipulators;Task analysis;Soft robotics;Load modeling;Data models;Machine learning for robot control;modeling;control;and learning for soft robots;soft robot applications},
    doi={10.1109/LRA.2022.3196789}}
                                              
                                              

Nonprehensile manipulation through precise pushing is an essential skill that has been commonly challenged by perception and physical uncertainties, such as those associated with contacts, object geometries, and physical properties. For this, we propose a unified framework that jointly addresses system modeling, action generation, and control. While most existing approaches either heavily rely on a priori system information for analytic modeling, or leverage a large dataset to learn dynamic models, our framework approximates a system transition function via non-parametric learning only using a small number of exploratory actions (ca. 10). The approximated function is then integrated with model predictive control to provide precise pushing manipulation. Furthermore, we show that the approximated system transition functions can be robustly transferred across novel objects while being online updated to continuously improve the manipulation accuracy. Through extensive experiments on a real robot platform with a set of novel objects and comparing against a state-of-the-art baseline, we show that the proposed unified framework is a light-weight and highly effective approach to enable precise pushing manipulation all by itself. Our evaluation results illustrate that the system can robustly ensure millimeter-level precision and can straightforwardly work on any novel object.

@misc{wang_push_2024,
                                title = {{UNO} Push: Unified Nonprehensile Object Pushing via Non-Parametric Estimation and Model Predictive Control},
                                url = {http://arxiv.org/abs/2403.13274},
                                author = {Wang, Gaotian and Ren, Kejia and Hang, Kaiyu},
                                date = {2024-03-19},
                                eprint = {2403.13274 [cs]},
                                publisher = {arXiv},
                                langid = {english},
                            }
                                            

In order to successfully perform manipulation tasks in new environments, such as grasping, robots must be proficient in segmenting unseen objects from the background and/or other objects. Previous works perform unseen object instance segmentation (UOIS) by training deep neural networks on large-scale data to learn RGB/RGB-D feature embeddings, where cluttered environments often result in inaccurate segmentations. We build upon these methods and introduce a novel approach to correct inaccurate segmentation, such as under-segmentation, of static image-based UOIS masks by using robot interaction and a designed body frame-invariant feature. We demonstrate that the relative linear and rotational velocities of frames randomly attached to rigid bodies due to robot interactions can be used to identify objects and accumulate corrected object-level segmentation masks. By introducing motion to regions of segmentation uncertainty, we are able to drastically improve segmentation accuracy in an uncertainty-driven manner with minimal, non-disruptive interactions (ca. 2-3 per scene). We demonstrate the effectiveness of our proposed interactive perception pipeline in accurately segmenting cluttered scenes by achieving an average object segmentation accuracy rate of $80.7 \%$, an increase of $28.2 \%$ when compared with other state-of-the-art UOIS methods.

@article{qian_riseg_nodate,
    title = {{RISeg}: Robot Interactive Object Segmentation via Body Frame-Invariant Features},
    author = {Qian, Howard and Lu, Yangxiao and Ren, Kejia and Wang, Gaotian and Khargonkar, Ninad and Xiang, Yu and Hang, Kaiyu},
    langid = {english},
    file = {Qian et al. - RISeg Robot Interactive Object Segmentation via B.pdf:C\:\\Users\\Vector Wang\\Zotero\\storage\\AH2HDVLC\\Qian et al. - RISeg Robot Interactive Object Segmentation via B.pdf:application/pdf},
}
                                              

The passive compliance of soft robotic arms renders the development of accurate kinematic models and modelbased controllers challenging. The most widely used model in soft robotic kinematics assumes Piecewise Constant Curvature (PCC). However, PCC introduces errors when the robot is subject to external forces or even gravity. In this paper, we establish a three-dimensional (3D) kinematic representation of a soft robotic arm with pseudo universal and prismatic joints that are capable of capturing non-constant curvature deformations of the soft segments. We theoretically demonstrate that this constitutes a more general methodology than PCC. Simulations and experiments on the real robot attest to the superior modeling accuracy of our approach in 3D motions with unknown loads. The maximum position/rotation error of the proposed model is verified $6.7 \times / 4.6 \times$ lower than the PCC model considering gravity and external forces. Furthermore, we devise an inverse kinematic controller that is capable of positioning the tip, tracking trajectories, as well as performing interactive tasks in the 3D space.

@article{wang_kinematic_nodate,
        title = {Kinematic Modeling and Control of a Soft Robotic Arm with Non-constant Curvature Deformation},
        author = {Wang, Zhanchi and Wang, Gaotian and Chen, Xiaoping and Freris, Nikolaos},
        langid = {english},
        file = {Wang et al. - Kinematic Modeling and Control of a Soft Robotic A.pdf:C\:\\Users\\Vector Wang\\Zotero\\storage\\2N79EKKQ\\Wang et al. - Kinematic Modeling and Control of a Soft Robotic A.pdf:application/pdf},
    }
                                              

The Piecewise Constant Curvature (PCC) assumption is the most widely used in the modeling and control of soft robots. However, a limitation of PCC models lies in accurately describing the deformation of a soft robot when executing dynamic tasks such as operating under gravity or interacting with humans. This paper introduces a new methodology for dynamic modeling and control of a multi-segment soft arm in the 3D space where each segment undergoes non-constant curvature deformations. In this framework, the soft manipulator is modeled as a series of segments, with each one represented by two stretchable links connected by a universal joint. Furthermore, we devise and analyze a controller for motion control in the configuration space with unknown load. The controller is implemented on a four-segment soft robotic manipulator and validated in a range of dynamic trajectory tracking tasks.

@INPROCEEDINGS{10354732,
        author={Wang, Zhanchi and Wang, Gaotian and Chen, Xiaoping and Freris, Nikolaos M.},
        booktitle={2023 IEEE International Conference on Robotics and Biomimetics (ROBIO)}, 
        title={Dynamic modeling and Control of a Soft Robotic Arm Using a Piecewise Universal Joint Model}, 
        year={2023},
        volume={},
        number={},
        pages={1-6},
        keywords={Solid modeling;Three-dimensional displays;Deformation;Motion segmentation;Dynamics;Soft robotics;Aerospace electronics},
        doi={10.1109/ROBIO58561.2023.10354732}}
                                              
                                              

It is challenging to control a soft robot, where reinforcement learning methods have been applied with promising results. However, due to the poor sample efficiency, reinforcement learning methods require a large collection of training data, which limits their applications. In this paper, we propose a Q-learning controller for a physical soft robot, in which pre-trained models using data from a rough simulator are applied to improve the performance of the controller. We implement the method on our soft robot, i.e., Honeycomb Pneumatic Network (HPN) arm. The experiments show that the usage of pre-trained models can not only reduce the amount of the real-world training data, but also greatly improve its accuracy and convergence rate.

@INPROCEEDINGS{9739524,
    author={Li, Peijin and Wang, Gaotian and Jiang, Hao and Jin, Yusong and Gan, Yinghao and Chen, Xiaoping and Ji, Jianmin},
    booktitle={2021 IEEE International Conference on Robotics and Biomimetics (ROBIO)}, 
    title={A Q-learning Control Method for a Soft Robotic Arm Utilizing Training Data from a Rough Simulator}, 
    year={2021},
    volume={},
    number={},
    pages={839-845},
    keywords={Q-learning;Costs;Conferences;Biomimetics;Biological system modeling;Training data;Soft robotics},
    doi={10.1109/ROBIO54168.2021.9739524}}