Sensorimotor control in the congenital absence of functional muscle spindles.

Hereditary sensory and autonomic neuropathy type III (HSAN III), also known as familial dysautonomia or Riley-Day syndrome, results from an autosomal recessive genetic mutation that causes a selective loss of specific sensory neurones, leading to greatly elevated pain and temperature thresholds, poor proprioception, marked ataxia and disturbances in blood pressure control. Stretch reflexes are absent throughout the body, which can be explained by the absence of functional muscle spindle afferents - assessed by intraneural microelectrodes inserted into peripheral nerves in the upper and lower limbs. This also explains the greatly compromised proprioception at the knee joint, as assessed by passive joint-angle matching. Moreover, there is a tight correlation between loss of proprioceptive acuity at the knee and the severity of gait impairment. Surprisingly, proprioception is normal at the elbow, suggesting that participants are relying more on sensory cues from the overlying skin; microelectrode recordings have shown that myelinated tactile afferents in the upper and lower limbs appear to be normal. Nevertheless, the lack of muscle spindles does affect sensorimotor control in the upper limb: in addition to poor performance in the finger-to-nose test, manual performance in the Purdue pegboard task is much worse than in age-matched healthy controls. Unlike those rare individuals with large-fibre sensory neuropathy, in which both muscle spindle and cutaneous afferents are absent, those with HSAN III present as a means of assessing sensorimotor control following the selective loss of muscle spindle afferents.

Elbow proprioception is normal in patients with a congenital absence of functional muscle spindles.

KEY POINTS: Individuals with hereditary sensory and autonomic neuropathy type III (HSAN III), also known as Riley-Day syndrome or familial dysautonomia, do not have functional muscle spindle afferents but do have essentially normal cutaneous mechanoreceptors. Lack of muscle spindle feedback from the legs may account for the poor proprioception at the knee and the ataxic gait typical of HSAN III. Given that functional muscle spindle afferents are also absent in the upper limb, we assessed whether proprioception at the elbow was likewise compromised. Passive joint angle matching showed that proprioception was normal at the elbow, suggesting that individuals with HSAN III rely more on cutaneous afferents around the elbow. ABSTRACT: Hereditary sensory and autonomic neuropathy type III (HSAN III) is a rare neurological condition that features a marked ataxic gait that progressively worsens over time. We have shown that functional muscle spindle afferents are absent in the upper and lower limbs in HSAN III, and we have argued that this may account for the ataxia. We recently used passive joint angle matching to demonstrate that proprioception of the knee joint is very poor in HSAN III but can be improved towards normal by application of elastic kinesiology tape across the knee joints, which we attribute to the presence of intact cutaneous mechanoreceptors. Here we assessed whether proprioception was equally compromised at the elbow joint, and whether it could be improved through taping. Proprioception at the elbow joint was assessed using passive joint angle matching in 12 HSAN III patients and 12 age-matched controls. There was no difference in absolute error, gradient or correlation coefficient of the relationship between joint angles of the reference and indicator arms. Unlike at the knee, taping did not improve elbow proprioception in either group. Clearly, the lack of muscle spindles compromised proprioception at the knee but not at the elbow, and we suggest that the HSAN III patients rely more on proprioceptive signals from the skin around the elbow.

Effects of tonic muscle pain on fusimotor control of human muscle spindles during isometric ankle dorsiflexion.

Studies on anesthetized animals have revealed that nociceptors can excite fusimotor neurons and thereby change the sensitivity of muscle spindles to stretch; such nociceptive reflexes have been suggested to underlie the mechanisms that lead to chronic musculoskeletal pain syndromes. However, the validity of the "vicious cycle" hypothesis in humans has yielded results contrasting with those found in animals. Given that spindle firing rates are much lower in humans than in animals, it is possible that some of the discrepancies between human experimental data and those obtained in animals could be explained by differences in background fusimotor drive when the leg muscles are relaxed. We examined the effects of tonic muscle pain during voluntary contractions of the ankle dorsiflexors. Unitary recordings were obtained from 10 fusimotor-driven muscle spindle afferents (6 primary, 4 secondary) supplying the ankle dorsiflexors via a microelectrode inserted percutaneously into the common peroneal nerve. A series of 1-min weak contractions was performed at rest and during 1 h of muscle pain induced by intramuscular infusion of 5% hypertonic saline into the tibialis anterior muscle. We did not observe any statistically significant increases in muscle spindle firing rates of six afferents followed during tonic muscle pain, although discharge variability increased slightly. Furthermore, a participant's capacity to maintain a constant level of force, while relying on proprioceptive feedback in the absence of visual feedback, was not compromised during pain. We conclude that nociceptive inputs from contracting muscle do not excite fusimotor neurons during voluntary isometric contractions in humans. NEW & NOTEWORTHY Data obtained in the cat have shown that muscle pain causes a marked increase in the firing of muscle spindles, attributed to a nociceptor-driven fusimotor reflex. However, our studies of muscle spindles in relaxed leg muscles failed to find any effect on spindle discharge. Here we showed that experimental muscle pain failed to increase the firing of muscle spindle afferents during weak voluntary contractions, when fusimotor drive sufficient to increase their firing is present.

Impaired sensorimotor control of the hand in congenital absence of functional muscle spindles.

Patients with hereditary sensory and autonomic neuropathy type III (HSAN III) exhibit marked ataxia, including gait disturbances. We recently showed that functional muscle spindle afferents in the leg, recorded via intraneural microelectrodes inserted into the peroneal nerve, are absent in HSAN III, although large-diameter cutaneous afferents are intact. Moreover, there is a tight correlation between loss of proprioceptive acuity at the knee and the severity of gait impairment. We tested the hypothesis that manual motor performance is also compromised in HSAN III, attributed to the predicted absence of muscle spindles in the intrinsic muscles of the hand. Manual performance in the Purdue pegboard task was assessed in 12 individuals with HSAN III and 11 age-matched healthy controls. The mean (±SD) pegboard score (number of pins inserted in 30 s) was 8.1 ± 1.9 and 8.6 ± 1.8 for the left and right hand, respectively, significantly lower than the scores for the controls (15.0 ± 1.3 and 16.0 ± 1.1; P < 0.0001). Performance was not improved after kinesiology tape was applied over the joints of the hand. In 5 patients we inserted a tungsten microelectrode into the ulnar nerve at the wrist. No spontaneous or stretch-evoked muscle afferent activity could be identified in any of the 11 fascicles supplying intrinsic muscles of the hand, whereas touch-evoked activity from low-threshold cutaneous mechanoreceptor afferents could readily be recorded from 4 cutaneous fascicles. We conclude that functional muscle spindles are absent in the short muscles of the hand and most likely absent in the long finger flexors and extensors, and that this largely accounts for the poor manual motor performance in HSAN III. NEW & NOTEWORTHY We describe the impaired manual motor performance in patients with hereditary sensory and autonomic neuropathy type III (Riley-Day syndrome), who exhibit congenital insensitivity to pain, poor proprioception, and marked gait ataxia. We show that functional muscle spindles are absent in the intrinsic muscles of the hand, which we argue contributes to their poor performance in a task involving the precision grip.

Eye center localization and gaze gesture recognition for human-computer interaction.

This paper introduces an unsupervised modular approach for accurate and real-time eye center localization in images and videos, thus allowing a coarse-to-fine, global-to-regional scheme. The trajectories of eye centers in consecutive frames, i.e., gaze gestures, are further analyzed, recognized, and employed to boost the human-computer interaction (HCI) experience. This modular approach makes use of isophote and gradient features to estimate the eye center locations. A selective oriented gradient filter has been specifically designed to remove strong gradients from eyebrows, eye corners, and shadows, which sabotage most eye center localization methods. A real-world implementation utilizing these algorithms has been designed in the form of an interactive advertising billboard to demonstrate the effectiveness of our method for HCI. The eye center localization algorithm has been compared with 10 other algorithms on the BioID database and six other algorithms on the GI4E database. It outperforms all the other algorithms in comparison in terms of localization accuracy. Further tests on the extended Yale Face Database b and self-collected data have proved this algorithm to be robust against moderate head poses and poor illumination conditions. The interactive advertising billboard has manifested outstanding usability and effectiveness in our tests and shows great potential for benefiting a wide range of real-world HCI applications.

Gender recognition from facial images: two or three dimensions?

This paper seeks to compare encoded features from both two-dimensional (2D) and three-dimensional (3D) face images in order to achieve automatic gender recognition with high accuracy and robustness. The Fisher vector encoding method is employed to produce 2D, 3D, and fused features with escalated discriminative power. For 3D face analysis, a two-source photometric stereo (PS) method is introduced that enables 3D surface reconstructions with accurate details as well as desirable efficiency. Moreover, a 2D+3D imaging device, taking the two-source PS method as its core, has been developed that can simultaneously gather color images for 2D evaluations and PS images for 3D analysis. This system inherits the superior reconstruction accuracy from the standard (three or more light) PS method but simplifies the reconstruction algorithm as well as the hardware design by only requiring two light sources. It also offers great potential for facilitating human computer interaction by being accurate, cheap, efficient, and nonintrusive. Ten types of low-level 2D and 3D features have been experimented with and encoded for Fisher vector gender recognition. Evaluations of the Fisher vector encoding method have been performed on the FERET database, Color FERET database, LFW database, and FRGCv2 database, yielding 97.7%, 98.0%, 92.5%, and 96.7% accuracy, respectively. In addition, the comparison of 2D and 3D features has been drawn from a self-collected dataset, which is constructed with the aid of the 2D+3D imaging device in a series of data capture experiments. With a variety of experiments and evaluations, it can be proved that the Fisher vector encoding method outperforms most state-of-the-art gender recognition methods. It has also been observed that 3D features reconstructed by the two-source PS method are able to further boost the Fisher vector gender recognition performance, i.e., up to a 6% increase on the self-collected database.

In vivo measurement of skin microrelief using photometric stereo in the presence of interreflections.

This paper proposes and describes an implementation of a photometric stereo-based technique for in vivo assessment of three-dimensional (3D) skin topography in the presence of interreflections. The proposed method illuminates skin with red, green, and blue colored lights and uses the resulting variation in surface gradients to mitigate the effects of interreflections. Experiments were carried out on Caucasian, Asian, and African American subjects to demonstrate the accuracy of our method and to validate the measurements produced by our system. Our method produced significant improvement in 3D surface reconstruction for all Caucasian, Asian, and African American skin types. The results also illustrate the differences in recovered skin topography due to the nondiffuse bidirectional reflectance distribution function (BRDF) for each color illumination used, which also concur with the existing multispectral BRDF data available for skin.

Unsupervised sub-segmentation for pigmented skin lesions.

BACKGROUND: Early identification of malignant melanoma with the surgical removal of thin lesions is the most effective treatment for skin cancers. A computer-aided diagnostic system assists to improve the diagnostic accuracy, where segmenting lesion from normal skin is usually considered as the first step. One of the challenges in the automated segmentation of skin lesions arises from the fact that darker areas within the lesion should be considered separate from the more general suspicious lesion as a whole, because these pigmented areas can provide significant additional diagnostic information. METHODS: This paper presents, for the first time, an unsupervised segmentation scheme to allow the isolation of normal skin, pigmented skin lesions, and interesting darker areas inside the lesion simultaneously. An adaptive mean-shift is first applied with a 5D spatial colour-texture feature space to generate a group of homogenous regions. Then the sub-segmentation maps are calculated by integrating maximal similarity-based region merging and the kernel k-means algorithm, where the number of segments is defined by a cluster validity measurement. RESULTS: The proposed method has been validated extensively on both normal digital photographs and dermoscopy images, which demonstrates competitive performance in achieving automatic segmentation. The isolated dark areas have proved helpful in the discrimination of malignant melanomas from atypical benign nevi. Compared with the results obtained from the asymmetry measure of the entire lesion, the asymmetry distribution of the isolated dark areas helped increase the accuracy of the identification of malignant melanoma from 65.38% to 73.07%, and this classification accuracy reached 80.77% on integrating both asymmetry descriptors. CONCLUSION: The proposed segmentation scheme gives the lesion boundary closed to the manual segmentation obtained by experienced dermatologists. The initial classification results indicate that the study of the distributions of darker areas inside the lesions is very promising in characterizing melanomas.

Diabetes mellitus prediction and diagnosis from a data preprocessing and machine learning perspective.

BACKGROUND AND OBJECTIVE: Diabetes mellitus is a metabolic disorder characterized by hyperglycemia, which results from the inadequacy of the body to secrete and respond to insulin. If not properly managed or diagnosed on time, diabetes can pose a risk to vital body organs such as the eyes, kidneys, nerves, heart, and blood vessels and so can be life-threatening. The many years of research in computational diagnosis of diabetes have pointed to machine learning to as a viable solution for the prediction of diabetes. However, the accuracy rate to date suggests that there is still much room for improvement. In this paper, we are proposing a machine learning framework for diabetes prediction and diagnosis using the PIMA Indian dataset and the laboratory of the Medical City Hospital (LMCH) diabetes dataset. We hypothesize that adopting feature selection and missing value imputation methods can scale up the performance of classification models in diabetes prediction and diagnosis. METHODS: In this paper, a robust framework for building a diabetes prediction model to aid in the clinical diagnosis of diabetes is proposed. The framework includes the adoption of Spearman correlation and polynomial regression for feature selection and missing value imputation, respectively, from a perspective that strengthens their performances. Further, different supervised machine learning models, the random forest (RF) model, support vector machine (SVM) model, and our designed twice-growth deep neural network (2GDNN) model are proposed for classification. The models are optimized by tuning the hyperparameters of the models using grid search and repeated stratified k-fold cross-validation and evaluated for their ability to scale to the prediction problem. RESULTS: Through experiments on the PIMA Indian and LMCH diabetes datasets, precision, sensitivity, F1-score, train-accuracy, and test-accuracy scores of 97.34%, 97.24%, 97.26%, 99.01%, 97.25 and 97.28%, 97.33%, 97.27%, 99.57%, 97.33, are achieved with the proposed 2GDNN model, respectively. CONCLUSION: The data preprocessing approaches and the classifiers with hyperparameter optimization proposed within the machine learning framework yield a robust machine learning model that outperforms state-of-the-art results in diabetes mellitus prediction and diagnosis. The source code for the models of the proposed machine learning framework has been made publicly available.

A new method describing border irregularity of pigmented lesions.

BACKGROUND/PURPOSE: Automatic quantitative characterization of border irregularity generating useful descriptors is a highly important task for computer-aided diagnosis of melanoma. This paper proposes a novel approach to describe the border irregularity of melanomas aiming at achieving higher recognition rates. METHODS: By introducing a boundary characteristic description, which we call a centroid distance diagram (CDD), a compact-supported mapping, called the centroid distance curve, can be extracted from this diagram. The centroid distance curve establishes the projection from angular orientations to the sum of the lengths of those line segments connecting the lesion centroid and border points. Border irregularity descriptors generated from CDDs include the non-centroid-convexity index, the maximum-minimum distance indicator, the standard deviation of centroid distance curves and the maximum magnitude of non-zero frequency elements of centroid distance curves after discrete Fourier transforms. Upper limits of the error boundaries involved in these descriptors are estimated. RESULTS: Experimental studies are based on 60 melanoma and 107 benign lesion images collected from local pigmented lesion clinics. By applying the proposed descriptors, receiver operating characteristic (ROC) curves are constructed by projecting the features into a linear space learned from samples. The optimal sensitivity and specificity for the proposed method are 74.2% and 72.6%. The total area enclosed by the corresponding ROC curve is 0.788. In addition, as the training and testing study for melanoma recognition in the literature is largely missing, a comprehensive comparative study is conducted by randomly dividing the data into two groups: one for training and one for testing. For the testing group, the best mean sensitivity obtained with the descriptors proposed in this paper reaches 71.8% and the standard deviation is 10.1%. The specificity for the testing group corresponding to the optimal sensitivity is 69.8%, with a standard deviation of 7.2%. CONCLUSION: This study suggests that in terms of sensitivity, descriptors extracted from CDDs are the most powerful ones in characterizing the border irregularity of melanomas.

Using 3D differential forms to characterize a pigmented lesion in vivo.

BACKGROUND/PURPOSE: After the formulation of ABCD rules, many new feature extraction methods are emerging to describe the asymmetry, border irregularity, color variation and diameter of malignant melanoma. In this paper, a new research direction orthogonal to ABCD rules that characterizes 3D local disruption of skin surfaces to realize automatic recognition of melanoma is described. METHODS: This paper examines 3D differential forms of skin surfaces to characterize the local geometrical properties of melanoma. Firstly, 3D data of skin surfaces are obtained using a photometric stereo device. Then differential forms of lesion surfaces are determined to describe the geometrical texture patterns involved. Using only these geometrical features, a simple least-squared error-based linear classifier can be constructed to realize the classification of malignant melanomas and benign lesions. RESULTS: As with the 3D data of 35 melanoma and 66 benign lesion samples collected from local pigmented lesion clinics, the optimal sensitivity and specificity of the constructed linear classifier are 71.4% and 86.4%, respectively. The total area enclosed by the corresponding receiver operating characteristics curve is 0.823. CONCLUSION: This study indicates that differential forms obtained from 3D data are very promising in characterizing melanoma. Combining these features with other skin features such as border irregularity and color variation might further improve the accuracy and reliability of the automatic diagnosis of melanoma.

A computer vision approach to improving cattle digestive health by the monitoring of faecal samples.

The digestive health of cows is one of the primary factors that determine their well-being and productivity. Under- and over-feeding are both commonplace in the beef and dairy industry; leading to welfare issues, negative environmental impacts, and economic losses. Unfortunately, digestive health is difficult for farmers to routinely monitor in large farms due to many factors including the need to transport faecal samples to a laboratory for compositional analysis. This paper describes a novel means for monitoring digestive health via a low-cost and easy to use imaging device based on computer vision. The method involves the rapid capture of multiple visible and near-infrared images of faecal samples. A novel three-dimensional analysis algorithm is then applied to objectively score the condition of the sample based on its geometrical features. While there is no universal ground truth for comparison of results, the order of scores matched a qualitative human prediction very closely. The algorithm is also able to detect the presence of undigested fibres and corn kernels using a deep learning approach. Detection rates for corn and fibre in image regions were of the order 90%. These results indicate the potential to develop this system for on-farm, real time monitoring of the digestive health of individual animals, allowing early intervention to effectively adjust feeding strategy.

Obtaining malignant melanoma indicators through statistical analysis of 3D skin surface disruptions.

BACKGROUND/PURPOSE: It has been observed that disruptions in skin patterns are larger for malignant melanoma (MM) than benign lesions. In order to extend the classification results achieved for 2D skin patterns, this work intends to investigate the feasibility of lesion classification using 3D skin surface texture, in the form of surface normals acquired from a previously built six-light photometric stereo device. MATERIAL AND METHODS: The proposed approach seeks to separate MM from benign lesions through analysis of the degree of surface disruptions in the tilt and slant direction of surface normals, so called skin tilt pattern and skin slant pattern. A 2D Gaussian function is used to simulate a normal region of skin for comparison with a lesion's observed tilt and slant patterns. The differences associated with the two patterns are estimated as the disruptions in the tilt and slant pattern respectively for lesion classification. RESULTS: Preliminary studies on 11 MMs and 28 benign lesions have given Receiver operating characteristic areas of 0.73 and 0.85 for tilt and slant pattern, respectively, which are better than 0.65 previously obtained for the skin line direction using the same samples. CONCLUSIONS: This paper has demonstrated an important application of 3D skin texture for computer-assisted diagnosis of MM in vivo. By taking advantage of the extra dimensional information, preliminary studies suggest that some improvements over the existing 2D skin line pattern approach for the differentiation between MM and benign lesions.

A photometric stereo-based 3D imaging system using computer vision and deep learning for tracking plant growth.

BACKGROUND: Tracking and predicting the growth performance of plants in different environments is critical for predicting the impact of global climate change. Automated approaches for image capture and analysis have allowed for substantial increases in the throughput of quantitative growth trait measurements compared with manual assessments. Recent work has focused on adopting computer vision and machine learning approaches to improve the accuracy of automated plant phenotyping. Here we present PS-Plant, a low-cost and portable 3D plant phenotyping platform based on an imaging technique novel to plant phenotyping called photometric stereo (PS). RESULTS: We calibrated PS-Plant to track the model plant Arabidopsis thaliana throughout the day-night (diel) cycle and investigated growth architecture under a variety of conditions to illustrate the dramatic effect of the environment on plant phenotype. We developed bespoke computer vision algorithms and assessed available deep neural network architectures to automate the segmentation of rosettes and individual leaves, and extract basic and more advanced traits from PS-derived data, including the tracking of 3D plant growth and diel leaf hyponastic movement. Furthermore, we have produced the first PS training data set, which includes 221 manually annotated Arabidopsis rosettes that were used for training and data analysis (1,768 images in total). A full protocol is provided, including all software components and an additional test data set. CONCLUSIONS: PS-Plant is a powerful new phenotyping tool for plant research that provides robust data at high temporal and spatial resolutions. The system is well-suited for small- and large-scale research and will help to accelerate bridging of the phenotype-to-genotype gap.

Reflectance of human skin using colour photometric stereo: with particular application to pigmented lesion analysis.

BACKGROUND/PURPOSE: The optical appearance of human skin is highly dependent on the interaction between the illumination (type and position), observer position and the skin surface structure. Different currently available photographic techniques record different aspects of this appearance, each providing its own incomplete description. This limits their usefulness, especially for pigmented skin lesion diagnosis. In this paper a new, easy to use, low-cost photographic method is described,which aims to generate an efficiently encoded yet reasonably complete representation of skin appearance. MATERIAL AND METHODS: A prototype hand-held camera was developed that rapidly acquires six colour images, each with the skin illuminated from a different direction. A novel photometric stereo processing was used to combine these into a colour image of the skin's diffuse reflectance, independent of the skin surface topography, as well as a separate representation of that topography in the form of a surface gradient image. Images of four clinical pigmented skin lesions were evaluated in comparison with conventional digital photographs by both visual judgement and automated lesion boundary detection. RESULTS: The new colour reflectance images were free from the effects of topographical shading, shadowing and specular reflections. Lesion boundaries obtained automatically from the reflectance images were always closer to the outline drawn by a dermatologist than those obtained from conventional photographs. Finally, recombining the colour reflectance and surface gradient data to form a virtual image of the skin surface that is highly realistic in appearance. CONCLUSIONS: The new colour photometric stereo camera produces images of skin and skin tumours in which the reflectance information that is related to subsurface pigment distribution is separated from the surface topographic information. The total information generated by the system, for use in visual or automated analysis, is potentially greater than that for either conventional photography or dermatoscopy alone. Its further development and broader clinical evaluation are warranted to determine its usefulness and role in a wide range of dermatological tasks, including tele-dermatology applications.

Photometric stereo for three-dimensional leaf venation extraction.

Leaf venation extraction studies have been strongly discouraged by considerable challenges posed by venation architectures that are complex, diverse and subtle. Additionally, unpredictable local leaf curvatures, undesirable ambient illuminations, and abnormal conditions of leaves may coexist with other complications. While leaf venation extraction has high potential for assisting with plant phenotyping, speciation and modelling, its investigations to date have been confined to colour image acquisition and processing which are commonly confounded by the aforementioned biotic and abiotic variations. To bridge the gaps in this area, we have designed a 3D imaging system for leaf venation extraction, which can overcome dark or bright ambient illumination and can allow for 3D data reconstruction in high resolution. We further propose a novel leaf venation extraction algorithm that can obtain illumination-independent surface normal features by performing Photometric Stereo reconstruction as well as local shape measures by fusing the decoupled shape index and curvedness features. In addition, this algorithm can determine venation polarity - whether veins are raised above or recessed into a leaf. Tests on both sides of different leaf species with varied venation architectures show that the proposed method is accurate in extracting the primary, secondary and even tertiary veins. It also proves to be robust against leaf diseases which can cause dramatic changes in colour. The effectiveness of this algorithm in determining venation polarity is verified by it correctly recognising raised or recessed veins in nine different experiments.

Innovative 3D and 2D machine vision methods for analysis of plants and crops in the field.

Machine vision systems offer great potential for automating crop control, harvesting, fruit picking, and a range of other agricultural tasks. However, most of the reported research on machine vision in agriculture involves a 2D approach, where the utility of the resulting data is often limited by effects such as parallax, perspective, occlusion and changes in background light - particularly when operating in the field. The 3D approach to plant and crop analysis described in this paper offers potential to obviate many of these difficulties by utilising the richer information that 3D data can generate. The methodologies presented, such as four-light photometric stereo, also provide advanced functionalities, such as an ability to robustly recover 3D surface texture from plants at very high resolution. This offers potential for enabling, for example, reliable detection of the meristem (the part of the plant where growth can take place), to within a few mm, for directed weeding (with all the associated cost and ecological benefits) as well as offering new capabilities for plant phenotyping. The considerable challenges associated with robust and reliable utilisation of machine vision in the field are also considered and practical solutions are described. Two projects are used to illustrate the proposed approaches: a four-light photometric stereo apparatus able to recover plant textures at high-resolution (even in direct sunlight), and a 3D system able to measure potato sizes in-the-field to an accuracy of within 10%, for extended periods and in a range of environmental conditions. The potential benefits of the proposed 3D methods are discussed, both in terms of the advanced capabilities attainable and the widespread potential uptake facilitated by their low cost.

Combination of 3D skin surface texture features and 2D ABCD features for improved melanoma diagnosis.

Two-dimensional asymmetry, border irregularity, colour variegation and diameter (ABCD) features are important indicators currently used for computer-assisted diagnosis of malignant melanoma (MM); however, they often prove to be insufficient to make a convincing diagnosis. Previous work has demonstrated that 3D skin surface normal features in the form of tilt and slant pattern disruptions are promising new features independent from the existing 2D ABCD features. This work investigates that whether improved lesion classification can be achieved by combining the 3D features with the 2D ABCD features. Experiments using a nonlinear support vector machine classifier show that many combinations of the 2D ABCD features and the 3D features can give substantially better classification accuracy than using (1) single features and (2) many combinations of the 2D ABCD features. The best 2D and 3D feature combination includes the overall 3D skin surface disruption, the asymmetry and all the three colour channel features. It gives an overall 87.8 % successful classification, which is better than the best single feature with 78.0 % and the best 2D feature combination with 83.1 %. These demonstrate that (1) the 3D features have additive values to improve the existing lesion classification and (2) combining the 3D feature with all the 2D features does not lead to the best lesion classification. The two ABCD features not selected by the best 2D and 3D combination, namely (1) the border feature and (2) the diameter feature, were also studied in separate experiments. It found that inclusion of either feature in the 2D and 3D combination can successfully classify 3 out of 4 lesion groups. The only one group not accurately classified by either feature can be classified satisfactorily by the other. In both cases, they have shown better classification performances than those without the 3D feature in the combinations. This further demonstrates that (1) the 3D feature can be used to improve the existing 2D-based diagnosis and (2) including the 3D feature with subsets of the 2D features can be used in distinguishing different benign lesion classes from MM. It is envisaged that classification performance may be further improved if different 2D and 3D feature subsets demonstrated in this study are used in different stages to target different benign lesion classes in future studies.

Distribution quantification on dermoscopy images for computer-assisted diagnosis of cutaneous melanomas.

Computerised analysis on skin lesion images has been reported to be helpful in achieving objective and reproducible diagnosis of melanoma. In particular, asymmetry in shape, colour and structure reflects the irregular growth of melanin under the skin and is of great importance for diagnosing the malignancy of skin lesions. This paper proposes a novel asymmetry analysis based on a newly developed pigmentation elevation model and the global point signatures (GPSs). Specifically, the pigmentation elevation model was first constructed by computer-based analysis of dermoscopy images, for the identification of melanin and haemoglobin. Asymmetry of skin lesions was then assessed through quantifying distributions of the pigmentation elevation model using the GPSs, derived from a Laplace-Beltrami operator. This new approach allows quantifying the shape and pigmentation distributions of cutaneous lesions simultaneously. Algorithm performance was tested on 351 dermoscopy images, including 88 malignant melanomas and 263 benign naevi, employing a support vector machine (SVM) with tenfold cross-validation strategy. Competitive diagnostic results were achieved using the proposed asymmetry descriptor only, presenting 86.36 % sensitivity, 82.13 % specificity and overall 83.43 % accuracy, respectively. In addition, the proposed GPS-based asymmetry analysis enables working on dermoscopy images from different databases and is approved to be inherently robust to the external imaging variations. These advantages suggested that the proposed method has good potential for follow-up treatment.

Enhanced 3D curvature pattern and melanoma diagnosis.

This article describes an enhanced curvature pattern based melanoma diagnosis system using convolution techniques and ensemble classifiers. We extract the 3D data of melanoma with a photometric stereo device first. Then differential forms of the melanoma surface can be extracted with the convolution method proposed. After extracting 3D based differential forms, statistical moments of enhanced principal curvatures of skin surfaces are calculated to describe the geometrical texture patterns. Finally, ensemble classifiers are constructed whose optimal mean sensitivity and specificity can reach 89.24 percent and 87.62 percent respectively. Comparisons with skin tilt/slant pattern based 3D shape characterization method and 2D methods like color variation and border irregularity are also included.