An Evaluation System of Robotic End-Effectors for Food Handling.

Owing to Japan's aging society and labor shortages, the food and agricultural industries are facing a significant demand for robotic food handling technologies. Considering the large variety of food products, available robotic end-effectors are limited. Our primary goal is to maximize the applicability of existing end-effectors and efficiently develop novel ones, and therefore, it is necessary to categorize food products and end-effectors from the viewpoint of robotic handling and establish their relationships through an effective evaluation approach. This study proposes a system for evaluating robotic end-effectors to identify appropriate ones and develop new ones. The evaluation system consists of food categorization based on food properties related to robotic handling, categorization of robotic end-effectors based on their grasping principles, a robotic system with visual recognition based on Robot Operating System 2 (ROS 2) to conduct handling tests, a scoring system for performance evaluation, and a visualization approach for presenting the results and comparisons. Based on food categorization, 14 real food items and their corresponding samples were chosen for handling tests. Seven robotic end-effectors, both commercialized and under development, were selected for evaluation. Using the proposed evaluation system, we quantitatively compared the performance of different end-effectors in handling different food items. We also observed differences in the handling of real food items and samples. The overall performance of an end-effector can be visualized and quantitatively evaluated to demonstrate its versatility in handling various food items.

A Scooping-Binding Robotic Gripper for Handling Various Food Products.

Food products are usually difficult to handle for robots because of their large variations in shape, size, softness, and surface conditions. It is ideal to use one robotic gripper to handle as many food products as possible. In this study, a scooping-binding robotic gripper is proposed to achieve this goal. The gripper was constructed using a pneumatic parallel actuator and two identical scooping-binding mechanisms. The mechanism consists of a thin scooping plate and multiple rubber strings for binding. When grasping an object, the mechanisms actively makes contact with the environment for scooping, and the object weight is mainly supported by the scooping plate. The binding strings are responsible for stabilizing the grasping by wrapping around the object. Therefore, the gripper can perform high-speed pick-and-place operations. Contact analysis was conducted using a simple beam model and a finite element model that were experimentally validated. Tension property of the binding string was characterized and an analytical model was established to predict binding force based on object geometry and binding displacement. Finally, handling tests on 20 food items, including products with thin profiles and slippery surfaces, were performed. The scooping-binding gripper succeeded in handling all items with a takt time of approximately 4 s. The gripper showed potential for actual applications in the food industry.

Circular Shell Gripper for Handling Food Products.

Pneumatically driven soft robotic grippers have been extensively studied in recent years. A majority of the grippers, especially those entirely composed of soft materials, can adapt to and handle various objects. However, there are limited studies regarding the realization of arbitrary grasping postures and twisting manipulation. Furthermore, the handling efficiency or the takt time of a handling task has not been investigated frequently. Therefore, this article proposes a circular shell gripper that consists of a rigid external shell and four soft internal air chambers. The soft chambers can be pneumatically inflated, thereby enabling it to grasp an object with a large contact area. The rigid shell allows the gripper to generate a large grasping force, while providing rigidity. This allows it to achieve arbitrary handling postures and twisting manipulations. A finite element model was constructed to simulate the chamber inflation, contact area upon grasping, lifting force, and twisting torque. An analytical model was formulated to quickly predict the lifting force and the twisting torque required to manipulate a known object. The models were validated via experimental tests on the lifting and twisting of a rigid cylinder. The experimental results indicate that the shell gripper can generate a maximum lifting force and twisting torque of 50.97 N and 0.73 Nm, respectively, with an input pressure of 10 kPa. Experimental tests on a variety of food and drink products revealed that the gripper could handle deformable, heavy, and irregularly shaped objects as well as realize arbitrary manipulation posture and twisting motion. The takt time for a pick-and-place task was found to be ∼2-5 s, using an SCARA robot; this time can be further optimized (less than 1 s) by using a parallel robot. However, the gripper was unable to handle low-profile objects, due to the downward pressing force; this was identified as a limitation of the proposed design.

Challenges and Opportunities in Robotic Food Handling: A Review.

Despite developments in robotics and automation technologies, several challenges need to be addressed to fulfill the high demand for automating various manufacturing processes in the food industry. In our opinion, these challenges can be classified as: the development of robotic end-effectors to cope with large variations of food products with high practicality and low cost, recognition of food products and materials in 3D scenario, better understanding of fundamental information of food products including food categorization and physical properties from the viewpoint of robotic handling. In this review, we first introduce the challenges in robotic food handling and then highlight the advances in robotic end-effectors, food recognition, and fundamental information of food products related to robotic food handling. Finally, future research directions and opportunities are discussed based on an analysis of the challenges and state-of-the-art developments.

Geometry Optimisation of a Hall-Effect-Based Soft Fingertip for Estimating Orientation of Thin Rectangular Objects.

Soft tactile sensors have been applied to robotic grippers for assembly. It is a challenging task to obtain contact information and object orientation using tactile sensors during grasping. Currently, the design of Hall-effect-based tactile sensors to perform such tasks is based on trial and error. We present a method of investigating the optimal geometrical design of a cylindrical soft sensor to increase its sensitivity. The finite element model of a soft fingertip was constructed in Abaqus with two design variables, i.e., hollow radius and magnet position. Then, the model was imported into Isight, with the maximisation of magnet displacement as the objective function. We found that the optimal design was at the boundary of the parameter design space. Four fingertips were fabricated with one intuitive, one optimal, and two optional sets of parameters. Experiments were performed, and object orientation was estimated by utilising linear approximation and a machine learning approach. Good agreements were achieved between optimisation and experiments. The results revealed that the estimated average error in object orientation was decreased by the optimised fingertip design. Furthermore, the 3-axis forces could successfully be estimated based on sensor outputs.

Towards patient-specific medializing calcaneal osteotomy for adult flatfoot: a finite element study.

Clinically in medializing calcaneal osteotomy (MCO), foot and ankle surgeons are facing difficulties in choosing appropriate surgical parameters due to the individual differences in deformities among flatfoot patients. Traditional cadaveric studies have provided important information regarding the biomechanical effects of tendons, ligaments, and plantar fascia, but limitations have been reached when dealing with individual differences and tailoring patient-specific surgeries. Therefore, this study aimed at implementing the finite element (FE) method to investigate the effect of different MCO parameters to help foot and ankle surgeons performing patient-specific surgeries. This study constructed FE models of a flatfoot and a healthy foot based on computed tomography (CT) images. After validating the FE models with experimental measurements, differences in plantar stress were compared between two models and a criterion was established for evaluating the performance of surgical simulations. Four MCO parameters were then studied through FE simulations. Results suggested that the transverse angle, ß, and translation distance, d, affected surgical performance. Therefore, special attentions may be recommended when choosing these two parameters clinically. However, the sagittal angle, α, and osteotomy position, p, were found to have less effect on the MCO performance.

Comparison of different soft grippers for lunch box packaging.

Automating the lunch box packaging is a challenging task due to the high deformability and large individual differences in shape and physical property of food materials. Soft robotic grippers showed potentials to perform such tasks. In this paper, we presented four pneumatic soft actuators made of different materials and different fabrication methods and compared their performances through a series of tests. We found that the actuators fabricated by 3D printing showed better linearity and less individual differences, but showed low durability compared to actuators fabricated by traditional casting process. Robotic grippers were assembled using the soft actuators, and grasping tests were performed on soft paper containers filled with food materials. Results suggested that grippers with softer actuators required lower air pressure to lift up the same weight and generated less deformation on the soft container. The actuator made of casting process with Dragon Skin 10 material lifted the most weight among different actuators.

Grasping state estimation of printable soft gripper using electro-conductive yarn.

Automatic handling of many types of food materials are required to realize the automation of production of commercially prepared box lunches. A printable soft gripper was developed for food handling which is simple to produce with a 3D printer. However, the sensing ability of the printable soft gripper was not discussed in previous research. In this paper, a novel method for estimating the grasping state of a printable soft gripper using electro-conductive yarn is presented. Electro-conductive yarn is a conductive material, and the resistance of strings is changed by stretching. It is less expensive than other sensors that can be used for measurement of grasping state. Additionally, it is easy to assemble and disassemble by hand. Electro-conductive yarn is applied to a prototype printable soft gripper, and the proposed estimation method is verified experimentally. From the experimental results, the estimated grasping state from the resistance of the electro-conductive yarn coincides with the actual grasping state of the gripper. Our proposed method of using electro-conductive yarn was successful for estimating the grasping state of a printable soft gripper.

Aerial pruning mechanism, initial real environment test.

In this research, a pruning mechanism for aerial pruning tasks is tested in a real environment. Since the final goal of the aerial pruning robot will be to prune tree branches close to power lines, some experiments related to wireless communication and pruning performance were conducted. The experiments consisted of testing the communication between two XBee RF modules for monitoring purposes as well as testing the speed control of the circular saw used for pruning tree branches. Results show that both the monitoring and the pruning tasks were successfully done in a real environment.

Disposable soft 3 axis force sensor for biomedical applications.

This paper proposes a new disposable soft 3D force sensor that can be used to calculate either force or displacement and vibrations. It uses three Hall Effect sensors orthogonally placed around a cylindrical beam made of silicon rubber. A niobium permanent magnet is inside the silicon. When a force is applied to the end of the cylinder, it is compressed and bent to the opposite side of the force displacing the magnet. This displacement causes change in the magnetic flux around the ratiomatric linear sensors (Hall Effect sensors). By analysing these changes, we calculate the force or displacement in three directions using a lookup table. This sensor can be used in minimal invasive surgery and haptic feedback applications. The cheap construction, bio-compatibility and ease of miniaturization are few advantages of this sensor. The sensor design, and its characterization are presented in this work.

Study on the human perception of incipient and overall slippages using a 2D FE fingertip model.

Slippage on the fingertips is an important phenomenon that occurs constantly in our daily life. However, the mechanism behind the slippage, especially incipient slippage, which appears prior to overall slippage, has not been fully understood. In this paper, a 2D finite element (FE) model of the human fingertip was presented to study how the human fingertip perceives slippages. The 2D geometries of the fingertip were generated based on magnetic resonance (MR) images. The fingertip model consisted of four layers: epidermis, dermis, subcutaneous tissue, and distal phalanx. The microstructures of the intermediate and limiting ridges in between the epidermis and dermis layers were manually constructed to locate four types of mechanoreceptors. Simulations of pushing and sliding motions were implemented, and mechanical measures of the acceleration and strain energy density (SED) were investigated at the locations of the mechanoreceptors. We found that both incipient and overall slippages could be clearly detected using the acceleration signal captured by the FA-I and SA-I receptors. The SED measurement does not provide useful information for the slippage detection.

Embedded electro-conductive yarn for shape sensing of soft robotic manipulators.

Flexible soft and stiffness-controllable surgical manipulators enhance the manoeuvrability of surgical tools during Minimally Invasive Surgery (MIS), as opposed to conventional rigid laparoscopic instruments. These flexible and soft robotic systems allow bending around organs, navigating through complex anatomical pathways inside the human body and interacting inherently safe with its soft environment. Shape sensing in such systems is a challenge and one essential requirement for precise position feedback control of soft robots. This paper builds on our previous work integrating multiple optical fibres into a soft manipulator to estimate the robot's pose using light intensity modulation. Here, we present an enhanced version of our embedded bending/shape sensor based on electro-conductive yarn. The new system is miniaturised and able to measure bending behaviour as well as elongation. The integrated yarn material is helically wrapped around an elastic strap and protected inside a 1.5mm outer-diameter stretchable pipe. Three of these resulting stretch sensors are integrated in the periphery of a pneumatically actuated soft manipulator for direct measurement of the actuation chamber lengths. The capability of the sensing system in measuring the bending curvature and elongation of the arm is evaluated.

Matching of feature points based on TSSC method from MR images of nonrigid deformed tissues.

Due to the nonlinear and nonuniform local deformation of nonrigid tissues, it is difficult to match a number of feature points distributing somewhat uniform in the tissues from MR images for deformation measurement. This paper proposes TSSC (TPS-SURF-SAC-Clustering) based method of feature point matching and elimination of mismatching. First, Fast-Hessian and Harris operator are utilized to extract the feature points in the initial MR image, and the matching region is identified by TPS transformation model for every query point in the deformed image. Then the SURF descriptors and the proposed Spatial Association Correspondence (SAC) method are combined to match the feature points. Finally, by clustering the coordinate differences between the matching points obtained by TPS-SURF-SAC and the matching points calculated by TPS model, most incorrectly matched points are eliminated. After every iterative processing of matching and mismatching elimination, the updated TPS model becomes more accurate and more correctly so that the matched points can be identified than those of last iteration. The experimental results show that the proposed SAC was efficient and that TSSC based method outperformed the single SURF or SIFT method.

A finite element model of flatfoot (Pes Planus) for improving surgical plan.

Flatfoot is a foot condition caused by the collapse of the medial arch of the foot, and it can result in problems such as severe pain, swelling, abnormal gait, and difficulty walking. Despite being a very common foot deformity, flatfoot is one of the least understood orthopaedic problems, and the opinions regarding its optimal treatment vary widely. In this paper, an FE model of a flatfoot is proposed that is based on CT measurements. Surface meshes of the bones and soft tissue were generated from CT images and then simplified to reduce the node density. A total of 62 ligaments, 9 tendons, and the plantar fascia were modeled manually. Volume meshes of the different components were generated and combined to form the completed flatfoot model. A dynamic FE formulation was derived, and a balanced standing simulation was performed. The model was validated by comparing stress distribution results from the simulation to experimental data.

Multimodal flexible sensor for healthcare systems.

This paper describes potential applications of our previously developed fabric sensor into wearable healthcare or nursing systems based on its sensing modalities. This sensor is knitted from tension-sensitive electro-conductive yarns; whose structure has an elastic core, wound around by two separated tension-sensitive electro-conductive threads. This makes the sensor inherently flexible and stretchable, allowing it to conform to any complicated surface. We have equipped the sensor with three modalities, including proximity that allows the sensor to estimate a distance from the sensor to human hand and activates a light touch sensing, which could initiate comfortable and friendly interfaces in order to reduce burden of patients/disable people during interactions with healthcare devices; tactile perception that can measures contact force or applied load, especially realize slippage acting on the sensor surface, which is promising to be embedded into wearable devices or smart carpets; and tensile that can quantify a volume's contraction/expansion, which can be employed to monitoring muscles activity and so on.

Viscoelastic interaction between intraocular microrobots and vitreous humor: a finite element approach.

Vitreous humor exhibits complex biomechanical properties and determination of these properties is essential for designing ophthalmic biomedical microdevices. In this paper, the viscoelastic properties of porcine vitreous humor were studied based on ex vivo creep experiments, in which a microrobot was magnetically actuated inside the vitreous. A three-dimensional (3D) finite element (FE) model was proposed to simulate the viscoelastic interaction between the microrobot and porcine vitreous humor. An optimization-based method was employed to estimate the viscoelastic parameters of the vitreous humor. The proposed model successfully validated the experimental measurements. The estimated parameters were compared with published data in literature. The model was then used to study the shape-dependent interaction of the microrobot with the vitreous humor. The methods presented in this paper can be used for the optimization of ophthalmic microrobots and microsurgical tools.

On the problem of determination of spring stiffness parameters for spring-mesh models.

On account of having real-time behavior and being easy to implement, spring meshes have been used for modeling deformable objects. Determining spring stiffness parameters for simulation of soft objects with high accuracy still remains a challenge. Allen Van Gelder derived an approximate formula for determining spring stiffness parameters based on strain analysis. Even though the experimental result showed the effectiveness, the method has not been investigated from a quantitative point of view. In this paper we propose a quantitative method for determining spring stiffness parameters. Moreover we propose a method to improve the accuracy by way of introducing torsional spring into the conventional spring mesh model.