・ 新開発のソフトマグネティックスキンは、ロボット、人間の皮膚や材料等に貼り付けられるストレッチャブルなマグネット状で触感機能を付与する。数百万個の微粒子を充填したシリコーンゴムから構成され、各微粒子の持つ N 極と S 極により発生する磁界を利用して触感を検出する。
・ 同マグネットスキンのセンサー機能の向上を図るため、内視鏡カテーテルを模倣した筒状の物体に沿った圧力の検出方法とセンシング面積の拡大(現時点では 1.5 ㎠まで）について研究を進める。また、磁性微粒子が作る磁界の拡大やマグネトメーターの配置場所の変更を試みる。
Advanced Intelligent Systems 掲載論文(フルテキスト）
Soft Magnetic Skin for Continuous Deformation Sensing
Recent progress in soft‐matter sensors has shown improved fabrication techniques, resolution, and range. However, scaling up these sensors into an information‐rich tactile skin remains largely limited by designs that require a corresponding increase in the number of wires to support each new sensing node. To address this, a soft tactile skin that can estimate force and localize contact over a continuous 15 mm2 area with a single integrated circuit and four output wires is introduced. The skin is composed of silicone elastomer loaded with randomly distributed magnetic microparticles. Upon deformation, the magnetic particles change position and orientation with respect to an embedded magnetometer, resulting in a change in the net measured magnetic field. Two experiments are reported to calibrate and estimate both location and force of surface contact. The classification algorithms can localize pressure with an accuracy of >98% on both grid and circle pattern. Regression algorithms can localize pressure to a 3 mm2 area on average. This proof‐of‐concept sensing skin addresses the increasing need for a simple‐to‐fabricate, quick‐to‐integrate, and information‐rich tactile surface for use in robotic manipulation, soft systems, and biomonitoring.
Growing interest in wearable technologies, soft robotics, and human–robot interaction has renewed focus on the development of soft sensing. These materials gather information while remaining soft and stretchable by using a wide range of sensing technologies and modalities. Many artificial or electronic skin technologies commonly use resistive or capacitive sensing, but there have also been exciting advancements in the use of piezoelectrics, triboelectricity, optics, and acoustics.1-4 Soft sensors can also be engineered that leverage the coupling of strain or pressure with changes in electrical resistance across fluidic channels embedded in the elastomer.5, 6 Composites can also be developed to add sensing properties to naturally soft host substrates, such as gel or elastomer. For example, loading elastomer with micro‐ or nano‐particles of liquid metal or carbon black can markedly improve the host’s thermal, electrical, mechanical, or radio‐frequency properties.7 However, these sensing technologies are difficult to scale‐up to large sensing areas due to corresponding challenges from fabrication, delicate interfaces, and the additional wiring that is required. Not only does integration of these larger systems become more difficult but also the probability of failure at one of the interfaces also increases.
Here, we introduce a tactile skin composed of a fixed 3‐axis magnetometer covered with a soft elastomer that is embedded with a dispersion of magnetic microparticles (Figure 1A). As deformation is applied to the surface of the composite, the microparticles are displaced with respect to the static position of the magnetometer. The magnetometer can measure the changes in the surrounding magnetic field, without direct electrical contact, and estimate the location and force of the contact (Figure 1B,C). An approximate model for this sensing mode is described in Section 1.1, Supporting Information. We also leverage morphological computation through the inherent dimension reduction performed in the material itself. Although there are many individual magnetic particles distributed throughout the skin contributing to the signal, we can measure a simple 3‐axis output that preserves information about the deformation.8-10
Sensor Overview: A) The elastomer composite loaded with magnetic microparticles is cured under a field. B) The composite retains the stretchability and flexibility of the host substrate, and is compatible with stretchable circuitry. C) The magnetic field measured at the magnetometer changes with the deformation of the elastomer. We attribute this to the change in location between each magnetic particle and the fixed magnetometer.
Magnetic and ferromagnetic elastomer composites have been well studied and reported.11-14 Previously established relationships between conductivity and applied pressure,15, 16 damping properties and external field,17 and shear modulus and external field,18 make them valuable for sensing19 and actuation.20 Recent approaches to magnetic tactile sensing measure magnetic flux with arrays of Hall effect sensors and rigid permanent magnets embedded within an elastomer.21-24 Magnetic flux can also be measured through inductance changes with giant magneto‐impedance materials.25 In contrast, our approach uses elastomers embedded with magnetic Ne‐Fe‐B microparticles that are on the order of ≈200 <?XML:NAMESPACE PREFIX = “[default] http://www.w3.org/1998/Math/MathML” NS = “http://www.w3.org/1998/Math/MathML” />μm in diameter. The use of microscale magnetic particles reduces the intensity of internal stress concentrations when mechanical load is applied and also allows the potential for sensing skins that are flexible or stretchable. Moreover, they allow for geometries that are thin or contain sharp 3D geometries.
We begin with a brief overview of the fabrication of the soft tactile skin and the method for collecting pressure data over a 15 mm2 square and 5 mm radial circle. Due to the nonuniform distribution of particles, we opt for data‐driven techniques to classify the location and estimate the depth of the contact, which have been shown to be successful for tactile data in many cases.26-30 The top five classification and regression algorithms are reported and discussed. In particular, we show that we can classify location with 98% accuracy for both 3 mm resolution 5 × 5 grid, and 5 mm radial circle with three discrete depths. Regression algorithms localize the contact to a 3 mm2 area. In summary, this work introduces a novel approach to address the need for a continuous and soft tactile surface with simple fabrication, quick integration, and adaptable geometry.
2 Results and Discussion
The skin is made by mixing a commercial silicone with magnetic microparticles and curing the composite under a magnetic field (see Section 4.1 for more details). We programmed a 4‐degree‐of‐freedom (DOF) robotic arm to automate applied pressure and collect magnetic field change and force over time (see Section 4.3 for more details). Here, we discuss two experiments: a 5 × 5 grid to demonstrate the spatial and force resolution given a fixed indentation depth and an 8‐point circle to test both depth and force resolution given a fixed distance from the magnetometer. The time‐series data was represented by a set of static features (see Section 4.2 for more details). Classification and regression algorithm comparisons can be found in Section 1.2, Supporting Information and Figure S3, Supporting Information.
2.1 Location Sensing
For the 5 × 5 grid experiment, force and magnetic field changes were collected over a 3 mm resolution 5 × 5 grid up to a 3 mm depth (Figure 2A). We collected 2750 contact samples at these 25 locations using a uniform random distribution. Each class (25 total) has about 100 samples each.
Grid Results Overview: A) The magnetic skin is sampled at 25 locations to a depth of 3 mm for a total of 25 classes. B) QDA classification results for location 13, and C) all QDA classification results grouped by class. Mean absolute error from linear regression grouped by location for D) x‐position, E) y‐position, and F) mean absolute error from k‐NN regression for force.
Several different classification algorithms were able to accurately distinguish between the 25 locations (see Section 1.2, Supporting Information). Here, we present classification results using quadratic discriminant analysis (QDA), which achieved the best performance. In the event of a misclassification, the predicted class is always adjacent to the true location (Figure 2B). Classification accuracy for every location are shown in Figure 2C and perform well across all 25 locations.
To estimate the location, we transformed the 25 discrete locations into their coordinate locations. For the 5 × 5 grid and linear regression, the x‐position has an average error of 1.1 mm and the y‐position has an average error of 3.8 mm. We attribute this difference in accuracy to the larger misalignment in the y‐axis frame. If the location grid is not perfectly centered over the magnetometer, the y signal will measure smaller changes in signal. In Figure 2D, the x‐axis looks well‐centered with very similar accuracy across 25 locations. However, the y‐axis accuracy is biased toward the right‐hand side (Figure 2E). This can be attributed to a combination of alignment, particle distribution, or chip manufacturing errors. These errors make model‐based techniques very difficult to calibrate, further supporting our use of data‐driven methods.
The output estimations near the edge of the sensor tend to have a lower accuracy and higher standard deviation. Due to the magnetic signal to distance relationship of 1/31/d3, the quality of signal is expected to decrease drastically with distance. At these points along the edge, we believe that the random distribution of particles begins to have a larger effect on output signal than the applied deformation. This leads to unusual signal changes, and is the main reason why we chose data‐driven techniques instead of function fitting. A more detailed example of this type of noise can be found in Section 1.3, Supporting Information.
2.2 Location and Depth Sensing
For the 8‐point circle experiment, the force‐controlled changes in magnetic field were measured for eight different XY locations and three different depths (dZ = 1, 2, or 3 mm) (Figure 3A). We collected 2850 contact samples for these 24 XYZ locations using a uniform random distribution. Each class (24 total) had approximately 110 samples each. See Figure S5, Supporting Information, for the experimental set‐up.
Circle Results Overview: A) The magnetic skin is sampled at 8 locations at 1, 2, or 3 mm depth for a total of 24 classes. B) QDA classification results for location 3 and a depth of 1 mm, and C) all QDA classification results grouped by class. Mean absolute error from linear regression output grouped by location for D) x‐position, E) y‐position, and F) z‐position. G) Mean absolute error from k‐NN regression grouped by location for force.
As in Section 2.1, QDA can be used to classify location based on both XY location and depth. If the predicted class is wrong, it is commonly predicted as an adjacent class (Figure 3B). Misclassification between adjacent locations is much more common than adjacent depths. The large correlation between z‐axis magnetic field and pressure can be used to easily distinguish between the three depths. Since all the tested locations are closer to the magnetometer than the 5 × 5 experiment, we do not see the same introduced noise from the particle aggregates. Classification accuracy for every location are shown in Figure 3C. In general, less applied pressure (depth = 1) leads to a smaller signal change and lower accuracy. For this sample, location 3 and depth 1 had noticeably lower classification accuracy. We attribute this to a combination of misalignment leading to smaller signals on the right‐hand side, which is also apparent in the larger error in locations 2, 3, and 4 in Figure 3D,E.
The 24 classes were transformed into their true (x,y,z) coordinates for location estimation. For the 8‐point circle and linear regression, the x‐position has a mean absolute error of 1.2 mm and the y‐position has a mean absolute error of 3.4 mm across all the classes. The difference in error between the x and y coordinates imply a small misalignment in this test as well—also shown in varied error by location in Figures 3D,E. The z‐position error is much smaller (0.03 mm) due to larger signal changes associated with 1 mm depth changes (Figure 3F).
2.3 Estimating Force
We can also estimate force with our time‐series data and a k‐nearest neighbors (k‐NN) regression. The inputs are the BBx, BBy, and BBz components of the magnetic field, the internal temperature of the magnetometer BBt, and load cell output at each time step. For the 5 × 5 grid experiments, the force estimation has a mean error of 0.44 N (Figure 2F), a minimum output of 0.03 N, and the maximum output of 1.9 N. For the 8‐point circle, the force estimation has a mean error of 0.25 N (Figure 3G), the minimum output of 0.14 N, and a maximum output of 2.4 N. The z‐axis of the magnetic field has the strongest correlation with the applied pressure, making force estimation quite accurate. However, a good signal change is dependent on the amount of deformation. Therefore, we expect that if the elastomer had a higher Young’s modulus, then the force resolution would be much larger. The force range applied during both tests was approximately between 0 and 2.5 N, which was limited by our chosen maximum depth of 3 mm.
2.4 Sensor Demonstrations
A simple use‐case of the tactile skin is demonstrated by using the magnetic elastomer as a 4‐key directional game pad. Four acrylic arrows are adhered to the surface to help the user locate where to apply pressure to input a direction command. The four commands can be identified by the changes in the X, Y, and Z components of the magnetic field. No classifier is used for this example, and instead simple thresholding is found to be adequate when the buttons are sufficiently spaced. The positive and negative X and Y changes are mapped to the four arrow keys on the keyboard to move an image around the screen. Example data for each direction is shown in Figure 4A.
Demos: A) Four direction arrow keys can be used to move images around the screen with simple thresholds and sign comparisons. B) More precise positioning requires a robotic arm and the previously trained QDA classification model to classify locations to place mines and flags in Minesweeper.
To demonstrate the speed and accuracy of the 5 × 5 grid classifier, we play a short game of Minesweeper with a robot‐controlled cylindrical indenter. Each of the 25 grid locations is mapped to a mouse location on the screen. The length of the signals (i.e., duration of applied pressure) indicates whether the user wants a left‐click to reveal the square or a right‐click to place a flag. Immediately after the signal returns to resting, the QDA classifier is used to predict the location and then the appropriate actions are performed. Raw data and classification results are shown in Figure 4B.
These demonstrations show that the magnetic skin can function with varied inputs and noise. However, all three inputs are relatively low frequency. As with most elastomer‐based sensors, we expect hysteresis to play a larger role in more dynamic applications. Since the sensing mode is dependent on the deformation of the magnetic skin, any mechanical improvements that help the skin keep up with dynamic change would be beneficial.
3 Conclusions and Future Work
In conclusion, a novel integration of magnetic elastomer with data‐driven analysis leads to a continuous interaction surface that can estimate location and depth of indentation. Classification results can distinguish between 25 grid locations in a 15 mm2 area with >98% accuracy. The algorithms can also classify 24 classes in a constant diameter circle with varied depth. Regression algorithms can localize the contact to a 3 mm2 area within the 15 mm2 active sensing area. The magnetic skin leverages morphological computation properties to inherently reduce the dimensionality of the output before analysis, thereby eliminating the need for a dense array of underlying microelectronic chips and wiring.
In the future, we plan to improve the range and resolution for force and contact location by tuning the fabrication process of the magnetic elastomer, training procedure, and adding additional magnetometers. In addition, mechanical improvements to the composite can mitigate hysteresis to enable use in more dynamic applications. We also anticipate future applications in soft robotics, medical devices, manipulation, and tactile surfaces. As necessary, the magnetic skin can be molded to conform to the geometry of the host system and be magnetically programed to respond to prescribed mechanical loads or deformations.
4 Experimental Section
The pre‐polymer and cross‐linker were shear mixed (AR‐100; Thinky) for 30 s in a 1:1 ratio. The pre‐cured elastomer mixture was immediately hand‐mixed with magnetic particles (MQP‐15‐12; Magnequench) in a 1:1 weight ratio. The composite was then poured into a 3D‐printed mold and degassed for 5 min. A thin plastic film was placed on top of the mold and excess elastomer was squeezed out. The filled mold was then placed upside down on the surface of a permanent magnet (N48; Applied Magnets). The elastomer was cured at room temperature and removed from the mold in an hour. Finally, the elastomer was adhered (Sil‐Poxy; Smooth‐On) to the top of the commercial magnetometer board (MLX90393; Sparkfun) (Figure S6, Supporting Information). The magnetic skin required no electrical connection to the underlying magnetometer board. Instead, it required proximity for the magnetic flux changes to be detected.
For this article, we chose to represent the time‐series data as a set of representative features. We chose to manually identify 21 features to aid our intuition on the results, in lieu of automated feature selection methods. The 21 features included the minimum, maximum, mean, standard deviation, median, and sum of the magnetic signal in each direction and the scalar ratios between the three axes. At the end of the contact, we calculated the features from data collected over the time of the contact, and immediately output the classification and regression results. We were most interested in supporting evidence of our claim that deformation of the randomly distributed magnetic particles created repeatable and separable signals. Classification and regression methods using these features are described in Section 1.2, Supporting Information.
Data collection was automated using a desktop 4‐DOF robotic arm (uArm Swift Pro; UFactory). The magnetic skin board was mounted onto an acrylic plate with a 500 g load cell (TAL221; SparkFun). During contact, 3‐axis magnetometer, load cell, and position data was collected and stored at approximately 50 Hz. The indention locations were programed in two patterns: a 5 × 5 grid (depth = 3 mm) and 8‐point circle (depth = 1, 2, or 3 mm). All indentations were performed at the same speed, 10 mm min−1. We used the robot arm kinematics as our ground truth location. The indenter was a cylindrical rigid punch with a radius of 1.5 mm. In both cases, the location was considered as a classification and a regression problem to focus on modeling the sensor implementation and supporting the proof‐of‐concept sensing mode. The magnetometer was located directly underneath location 13 of the 5 × 5 grid and center of the 8‐point circle.
This work was in part supported by ONR Grants N00014‐16‐2301 and N00014‐14‐10778, Code 34, Bio‐Inspired Autonomous Systems (PM: Tom McKenna), NOPP Grant N00014‐18‐12843 (PM: Reginald Beach), and National Science Foundation Graduate Research Fellowship Program under Grant No. DGE 1252522. Any opinions findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
Conflict of Interest
The authors declare no conflict of interest.