Localization and recognition of human action in 3D using transformers.

Understanding a person's behavior from their 3D motion sequence is a fundamental problem in computer vision with many applications. An important component of this problem is 3D action localization, which involves recognizing what actions a person is performing, and when the actions occur in the sequence. To promote the progress of the 3D action localization community, we introduce a new, challenging, and more complex benchmark dataset, BABEL-TAL (BT), for 3D action localization. Important baselines and evaluating metrics, as well as human evaluations, are carefully established on this benchmark. We also propose a strong baseline model, i.e., Localizing Actions with Transformers (LocATe), that jointly localizes and recognizes actions in a 3D sequence. The proposed LocATe shows superior performance on BABEL-TAL as well as on the large-scale PKU-MMD dataset, achieving state-of-the-art performance by using only 10% of the labeled training data. Our research could advance the development of more accurate and efficient systems for human behavior analysis, with potential applications in areas such as human-computer interaction and healthcare.

Exploring weight bias and negative self-evaluation in patients with mood disorders: insights from the BodyTalk Project.

Background: Negative body image and adverse body self-evaluation represent key psychological constructs within the realm of weight bias (WB), potentially intertwined with the negative self-evaluation characteristic of depressive symptomatology. Although WB encapsulates an implicit form of self-critical assessment, its exploration among people with mood disorders (MD) has been under-investigated. Our primary goal is to comprehensively assess both explicit and implicit WB, seeking to reveal specific dimensions that could interconnect with the symptoms of MDs. Methods: A cohort comprising 25 MD patients and 35 demographically matched healthy peers (with 83% female representation) participated in a series of tasks designed to evaluate the congruence between various computer-generated body representations and a spectrum of descriptive adjectives. Our analysis delved into multiple facets of body image evaluation, scrutinizing the associations between different body sizes and emotionally charged adjectives (e.g., active, apple-shaped, attractive). Results: No discernible differences emerged concerning body dissatisfaction or the correspondence of different body sizes with varying adjectives. Interestingly, MD patients exhibited a markedly higher tendency to overestimate their body weight (p = 0.011). Explicit WB did not show significant variance between the two groups, but MD participants demonstrated a notable implicit WB within a specific weight rating task for BMI between 18.5 and 25 kg/m2 (p = 0.012). Conclusions: Despite the striking similarities in the assessment of participants' body weight, our investigation revealed an implicit WB among individuals grappling with MD. This bias potentially assumes a role in fostering self-directed negative evaluations, shedding light on a previously unexplored facet of the interplay between WB and mood disorders.

The Poses for Equine Research Dataset (PFERD).

Studies of quadruped animal motion help us to identify diseases, understand behavior and unravel the mechanics behind gaits in animals. The horse is likely the best-studied animal in this aspect, but data capture is challenging and time-consuming. Computer vision techniques improve animal motion extraction, but the development relies on reference datasets, which are scarce, not open-access and often provide data from only a few anatomical landmarks. Addressing this data gap, we introduce PFERD, a video and 3D marker motion dataset from horses using a full-body set-up of densely placed over 100 skin-attached markers and synchronized videos from ten camera angles. Five horses of diverse conformations provide data for various motions from basic poses (eg. walking, trotting) to advanced motions (eg. rearing, kicking). We further express the 3D motions with current techniques and a 3D parameterized model, the hSMAL model, establishing a baseline for 3D horse markerless motion capture. PFERD enables advanced biomechanical studies and provides a resource of ground truth data for the methodological development of markerless motion capture.

Virtual Reality Exposure to a Healthy Weight Body Is a Promising Adjunct Treatment for Anorexia Nervosa.

INTRODUCTION/OBJECTIVE: Treatment results of anorexia nervosa (AN) are modest, with fear of weight gain being a strong predictor of treatment outcome and relapse. Here, we present a virtual reality (VR) setup for exposure to healthy weight and evaluate its potential as an adjunct treatment for AN. METHODS: In two studies, we investigate VR experience and clinical effects of VR exposure to higher weight in 20 women with high weight concern or shape concern and in 20 women with AN. RESULTS: In study 1, 90% of participants (18/20) reported symptoms of high arousal but verbalized low to medium levels of fear. Study 2 demonstrated that VR exposure to healthy weight induced high arousal in patients with AN and yielded a trend that four sessions of exposure improved fear of weight gain. Explorative analyses revealed three clusters of individual reactions to exposure, which need further exploration. CONCLUSIONS: VR exposure is a well-accepted and powerful tool for evoking fear of weight gain in patients with AN. We observed a statistical trend that repeated virtual exposure to healthy weight improved fear of weight gain with large effect sizes. Further studies are needed to determine the mechanisms and differential effects.

Skeletal Editing Approach to Bridge-Functionalized Bicyclo[1.1.1]pentanes from Azabicyclo[2.1.1]hexanes.

Azabicyclo[2.1.1]hexanes (aza-BCHs) and bicyclo[1.1.1]pentanes (BCPs) have emerged as attractive classes of sp3-rich cores for replacing flat, aromatic groups with metabolically resistant, three-dimensional frameworks in drug scaffolds. Strategies to directly convert, or "scaffold hop", between these bioisosteric subclasses through single-atom skeletal editing would enable efficient interpolation within this valuable chemical space. Herein, we describe a strategy to "scaffold hop" between aza-BCH and BCP cores through a nitrogen-deleting skeletal edit. Photochemical [2+2] cycloadditions, used to prepare multifunctionalized aza-BCH frameworks, are coupled with a subsequent deamination step to afford bridge-functionalized BCPs, for which few synthetic solutions currently exist. The modular sequence provides access to various privileged bridged bicycles of pharmaceutical relevance.

Fast-SNARF: A Fast Deformer for Articulated Neural Fields.

Neural fields have revolutionized the area of 3D reconstruction and novel view synthesis of rigid scenes. A key challenge in making such methods applicable to articulated objects, such as the human body, is to model the deformation of 3D locations between the rest pose (a canonical space) and the deformed space. We propose a new articulation module for neural fields, Fast-SNARF, which finds accurate correspondences between canonical space and posed space via iterative root finding. Fast-SNARF is a drop-in replacement in functionality to our previous work, SNARF, while significantly improving its computational efficiency. We contribute several algorithmic and implementation improvements over SNARF, yielding a speed-up of 150×. These improvements include voxel-based correspondence search, pre-computing the linear blend skinning function, and an efficient software implementation with CUDA kernels. Fast-SNARF enables efficient and simultaneous optimization of shape and skinning weights given deformed observations without correspondences (e.g. 3D meshes). Because learning of deformation maps is a crucial component in many 3D human avatar methods and since Fast-SNARF provides a computationally efficient solution, we believe that this work represents a significant step towards the practical creation of 3D virtual humans.

The neural coding of face and body orientation in occipitotemporal cortex.

Face and body orientation convey important information for us to understand other people's actions, intentions and social interactions. It has been shown that several occipitotemporal areas respond differently to faces or bodies of different orientations. However, whether face and body orientation are processed by partially overlapping or completely separate brain networks remains unclear, as the neural coding of face and body orientation is often investigated separately. Here, we recorded participants' brain activity using fMRI while they viewed faces and bodies shown from three different orientations, while attending to either orientation or identity information. Using multivoxel pattern analysis we investigated which brain regions process face and body orientation respectively, and which regions encode both face and body orientation in a stimulus-independent manner. We found that patterns of neural responses evoked by different stimulus orientations in the occipital face area, extrastriate body area, lateral occipital complex and right early visual cortex could generalise across faces and bodies, suggesting a stimulus-independent encoding of person orientation in occipitotemporal cortex. This finding was consistent across functionally defined regions of interest and a whole-brain searchlight approach. The fusiform face area responded to face but not body orientation, suggesting that orientation responses in this area are face-specific. Moreover, neural responses to orientation were remarkably consistent regardless of whether participants attended to the orientation of faces and bodies or not. Together, these results demonstrate that face and body orientation are processed in a partially overlapping brain network, with a stimulus-independent neural code for face and body orientation in occipitotemporal cortex.

Radical Termination via ß-Scission Enables Photoenzymatic Allylic Alkylation Using "Ene"-Reductases.

Allylations are practical transformations that forge C-C bonds while introducing an alkene for further chemical manipulations. Here, we report a photoenzymatic allylation of α-chloroamides with allyl silanes using flavin-dependent 'ene'-reductases (EREDs). An engineered ERED can catalyze annulative allylic alkylation to prepare 5, 6, and 7-membered lactams with high levels of enantioselectivity. Ultrafast transient absorption spectroscopy indicates that radical termination occurs via ß-scission of the silyl group to afford a silyl radical, a distinct mechanism by comparison to traditional radical allylations involving allyl silanes. Moreover, this represents an alternative strategy for radical termination using EREDs. This mechanism was applied to intermolecular couplings involving allyl sulfones and silyl enol ethers. Overall, this method highlights the opportunity for EREDs to catalyze radical termination strategies beyond hydrogen atom transfer.

Occlusion Boundary: A Formal Definition & Its Detection via Deep Exploration of Context.

Occlusion boundaries contain rich perceptual information about the underlying scene structure and provide important cues in many visual perception-related tasks such as object recognition, segmentation, motion estimation, scene understanding, and autonomous navigation. However, there is no formal definition of occlusion boundaries in the literature, and state-of-the-art occlusion boundary detection is still suboptimal. With this in mind, in this paper we propose a formal definition of occlusion boundaries for related studies. Further, based on a novel idea, we develop two concrete approaches with different characteristics to detect occlusion boundaries in video sequences via enhanced exploration of contextual information (e.g, local structural boundary patterns, observations from surrounding regions, and temporal context) with deep models and conditional random fields. Experimental evaluations of our methods on two challenging occlusion boundary benchmarks (CMU and VSB100) demonstrate that our detectors significantly outperform the current state-of-the-art. Finally, we empirically assess the roles of several important components of the proposed detectors to validate the rationale behind these approaches.

A pose-independent method for accurate and precise body composition from 3D optical scans.

OBJECTIVE: The aim of this study was to investigate whether digitally re-posing three-dimensional optical (3DO) whole-body scans to a standardized pose would improve body composition accuracy and precision regardless of the initial pose. METHODS: Healthy adults (n = 540), stratified by sex, BMI, and age, completed whole-body 3DO and dual-energy X-ray absorptiometry (DXA) scans in the Shape Up! Adults study. The 3DO mesh vertices were represented with standardized templates and a low-dimensional space by principal component analysis (stratified by sex). The total sample was split into a training (80%) and test (20%) set for both males and females. Stepwise linear regression was used to build prediction models for body composition and anthropometry outputs using 3DO principal components (PCs). RESULTS: The analysis included 472 participants after exclusions. After re-posing, three PCs described 95% of the shape variance in the male and female training sets. 3DO body composition accuracy compared with DXA was as follows: fat mass R2 = 0.91 male, 0.94 female; fat-free mass R2 = 0.95 male, 0.92 female; visceral fat mass R2 = 0.77 male, 0.79 female. CONCLUSIONS: Re-posed 3DO body shape PCs produced more accurate and precise body composition models that may be used in clinical or nonclinical settings when DXA is unavailable or when frequent ionizing radiation exposure is unwanted.

Separated and overlapping neural coding of face and body identity.

Recognising a person's identity often relies on face and body information, and is tolerant to changes in low-level visual input (e.g., viewpoint changes). Previous studies have suggested that face identity is disentangled from low-level visual input in the anterior face-responsive regions. It remains unclear which regions disentangle body identity from variations in viewpoint, and whether face and body identity are encoded separately or combined into a coherent person identity representation. We trained participants to recognise three identities, and then recorded their brain activity using fMRI while they viewed face and body images of these three identities from different viewpoints. Participants' task was to respond to either the stimulus identity or viewpoint. We found consistent decoding of body identity across viewpoint in the fusiform body area, right anterior temporal cortex, middle frontal gyrus and right insula. This finding demonstrates a similar function of fusiform and anterior temporal cortex for bodies as has previously been shown for faces, suggesting these regions may play a general role in extracting high-level identity information. Moreover, we could decode identity across fMRI activity evoked by faces and bodies in the early visual cortex, right inferior occipital cortex, right parahippocampal cortex and right superior parietal cortex, revealing a distributed network that encodes person identity abstractly. Lastly, identity decoding was consistently better when participants attended to identity, indicating that attention to identity enhances its neural representation. These results offer new insights into how the brain develops an abstract neural coding of person identity, shared by faces and bodies.

Active-Site Environmental Factors Customize the Photophysics of Photoenzymatic Old Yellow Enzymes.

The development of non-natural photoenzymatic systems has reinvigorated the study of photoinduced electron transfer (ET) within protein active sites, providing new and unique platforms for understanding how biological environments affect photochemical processes. In this work, we use ultrafast spectroscopy to compare the photoinduced electron transfer in known photoenzymes. 12-Oxophytodienoate reductase 1 (OPR1) is compared to Old Yellow Enzyme 1 (OYE1) and morphinone reductase (MR). The latter enzymes are structurally homologous to OPR1. We find that slight differences in the amino acid composition of the active sites of these proteins determine their distinct electron-transfer dynamics. Our work suggests that the inside of a protein active site is a complex/heterogeneous dielectric network where genetically programmed heterogeneity near the site of biological ET can significantly affect the presence and lifetime of various intermediate states. Our work motivates additional tunability of Old Yellow Enzyme active-site reorganization energy and electron-transfer energetics that could be leveraged for photoenzymatic redox approaches.

[Analysis of motor development within the first year of life: 3-D motion tracking without markers for early detection of developmental disorders]. / Analyse der Spontanmotorik im 1. Lebensjahr: Markerlose 3-D-Bewegungserfassung zur Früherkennung von Entwicklungsstörungen.

Children with motor development disorders benefit greatly from early interventions. An early diagnosis in pediatric preventive care (U2-U5) can be improved by automated screening. Current approaches to automated motion analysis, however, are expensive, require lots of technical support, and cannot be used in broad clinical application. Here we present an inexpensive, marker-free video analysis tool (KineMAT) for infants, which digitizes 3­D movements of the entire body over time allowing automated analysis in the future.Three-minute video sequences of spontaneously moving infants were recorded with a commercially available depth-imaging camera and aligned with a virtual infant body model (SMIL model). The virtual image generated allows any measurements to be carried out in 3­D with high precision. We demonstrate seven infants with different diagnoses. A selection of possible movement parameters was quantified and aligned with diagnosis-specific movement characteristics.KineMAT and the SMIL model allow reliable, three-dimensional measurements of spontaneous activity in infants with a very low error rate. Based on machine-learning algorithms, KineMAT can be trained to automatically recognize pathological spontaneous motor skills. It is inexpensive and easy to use and can be developed into a screening tool for preventive care for children.

General Movement Assessment from videos of computed 3D infant body models is equally effective compared to conventional RGB video rating.

BACKGROUND: General Movement Assessment (GMA) is a powerful tool to predict Cerebral Palsy (CP). Yet, GMA requires substantial training challenging its broad implementation in clinical routine. This inspired a world-wide quest for automated GMA. AIMS: To test whether a low-cost, marker-less system for three-dimensional motion capture from RGB depth sequences using a whole body infant model may serve as the basis for automated GMA. STUDY DESIGN: Clinical case study at an academic neurodevelopmental outpatient clinic. SUBJECTS: Twenty-nine high risk infants were assessed at their clinical follow-up at 2-4 month corrected age (CA). Their neurodevelopmental outcome was assessed regularly up to 12-31 months CA. OUTCOME MEASURES: GMA according to Hadders-Algra by a masked GMA-expert of conventional and computed 3D body model ("SMIL motion") videos of the same GMs. Agreement between both GMAs was tested using dichotomous and graded scaling with Kappa and intraclass correlations, respectively. Sensitivity and specificity to predict CP at ≥12 months CA were assessed. RESULTS: Agreement of the two GMA ratings was moderate-good for GM-complexity (κ = 0.58; ICC = 0.874 [95%CI 0.730; 0.941]) and substantial-good for fidgety movements (FMs; Kappa = 0.78, ICC = 0.926 [95%CI 0.843; 0.965]). Five children were diagnosed with CP (four bilateral, one unilateral CP). The GMs of the child with unilateral CP were twice rated as mildly abnormal with FMs. GM-complexity and somewhat less FMs, of both conventional and SMIL motion videos predicted bilateral CP comparably to published literature. CONCLUSIONS: Our computed infant 3D full body model is an attractive starting point for automated GMA in infants at risk of CP.

Photoenzymatic Hydrogenation of Heteroaromatic Olefins Using 'Ene'-Reductases with Photoredox Catalysts.

Flavin-dependent 'ene'-reductases (EREDs) are highly selective catalysts for the asymmetric reduction of activated alkenes. This function is, however, limited to enones, enoates, and nitroalkenes using the native hydride transfer mechanism. Here we demonstrate that EREDs can reduce vinyl pyridines when irradiated with visible light in the presence of a photoredox catalyst. Experimental evidence suggests the reaction proceeds via a radical mechanism where the vinyl pyridine is reduced to the corresponding neutral benzylic radical in solution. DFT calculations reveal this radical to be "dynamically stable", suggesting it is sufficiently long-lived to diffuse into the enzyme active site for stereoselective hydrogen atom transfer. This reduction mechanism is distinct from the native one, highlighting the opportunity to expand the synthetic capabilities of existing enzyme platforms by exploiting new mechanistic models.

Learning and Tracking the 3D Body Shape of Freely Moving Infants from RGB-D sequences.

Statistical models of the human body surface are generally learned from thousands of high-quality 3D scans in predefined poses to cover the wide variety of human body shapes and articulations. Acquisition of such data requires expensive equipment, calibration procedures, and is limited to cooperative subjects who can understand and follow instructions, such as adults. We present a method for learning a statistical 3D Skinned Multi-Infant Linear body model (SMIL) from incomplete, low-quality RGB-D sequences of freely moving infants. Quantitative experiments show that SMIL faithfully represents the RGB-D data and properly factorizes the shape and pose of the infants. To demonstrate the applicability of SMIL, we fit the model to RGB-D sequences of freely moving infants and show, with a case study, that our method captures enough motion detail for General Movements Assessment (GMA), a method used in clinical practice for early detection of neurodevelopmental disorders in infants. SMIL provides a new tool for analyzing infant shape and movement and is a step towards an automated system for GMA.

Asymmetric redox-neutral radical cyclization catalysed by flavin-dependent 'ene'-reductases.

Flavin-dependent 'ene'-reductases (EREDs) are exquisite catalysts for effecting stereoselective reductions. Although these reactions typically proceed through a hydride transfer mechanism, we recently found that EREDs can also catalyse reductive dehalogenations and cyclizations via single electron transfer mechanisms. Here, we demonstrate that these enzymes can catalyse redox-neutral radical cyclizations to produce enantioenriched oxindoles from α-haloamides. This transformation is a C-C bond-forming reaction currently unknown in nature and one for which there are no catalytic asymmetric examples. Mechanistic studies indicate the reaction proceeds via the flavin semiquinone/quinone redox couple, where ground-state flavin semiquinone provides the electron for substrate reduction and flavin quinone oxidizes the vinylogous α-amido radical formed after cyclization. This mechanistic manifold was previously unknown for this enzyme family, highlighting the versatility of EREDs in asymmetric synthesis.

Decoding subcategories of human bodies from both body- and face-responsive cortical regions.

Our visual system can easily categorize objects (e.g. faces vs. bodies) and further differentiate them into subcategories (e.g. male vs. female). This ability is particularly important for objects of social significance, such as human faces and bodies. While many studies have demonstrated category selectivity to faces and bodies in the brain, how subcategories of faces and bodies are represented remains unclear. Here, we investigated how the brain encodes two prominent subcategories shared by both faces and bodies, sex and weight, and whether neural responses to these subcategories rely on low-level visual, high-level visual or semantic similarity. We recorded brain activity with fMRI while participants viewed faces and bodies that varied in sex, weight, and image size. The results showed that the sex of bodies can be decoded from both body- and face-responsive brain areas, with the former exhibiting more consistent size-invariant decoding than the latter. Body weight could also be decoded in face-responsive areas and in distributed body-responsive areas, and this decoding was also invariant to image size. The weight of faces could be decoded from the fusiform body area (FBA), and weight could be decoded across face and body stimuli in the extrastriate body area (EBA) and a distributed body-responsive area. The sex of well-controlled faces (e.g. excluding hairstyles) could not be decoded from face- or body-responsive regions. These results demonstrate that both face- and body-responsive brain regions encode information that can distinguish the sex and weight of bodies. Moreover, the neural patterns corresponding to sex and weight were invariant to image size and could sometimes generalize across face and body stimuli, suggesting that such subcategorical information is encoded with a high-level visual or semantic code.

The Virtual Caliper: Rapid Creation of Metrically Accurate Avatars from 3D Measurements.

Creating metrically accurate avatars is important for many applications such as virtual clothing try-on, ergonomics, medicine, immersive social media, telepresence, and gaming. Creating avatars that precisely represent a particular individual is challenging however, due to the need for expensive 3D scanners, privacy issues with photographs or videos, and difficulty in making accurate tailoring measurements. We overcome these challenges by creating "The Virtual Caliper", which uses VR game controllers to make simple measurements. First, we establish what body measurements users can reliably make on their own body. We find several distance measurements to be good candidates and then verify that these are linearly related to 3D body shape as represented by the SMPL body model. The Virtual Caliper enables novice users to accurately measure themselves and create an avatar with their own body shape. We evaluate the metric accuracy relative to ground truth 3D body scan data, compare the method quantitatively to other avatar creation tools, and perform extensive perceptual studies. We also provide a software application to the community that enables novices to rapidly create avatars in fewer than five minutes. Not only is our approach more rapid than existing methods, it exports a metrically accurate 3D avatar model that is rigged and skinned.

Body size estimation of self and others in females varying in BMI.

Previous literature suggests that a disturbed ability to accurately identify own body size may contribute to overweight. Here, we investigated the influence of personal body size, indexed by body mass index (BMI), on body size estimation in a non-clinical population of females varying in BMI. We attempted to disentangle general biases in body size estimates and attitudinal influences by manipulating whether participants believed the body stimuli (personalized avatars with realistic weight variations) represented their own body or that of another person. Our results show that the accuracy of own body size estimation is predicted by personal BMI, such that participants with lower BMI underestimated their body size and participants with higher BMI overestimated their body size. Further, participants with higher BMI were less likely to notice the same percentage of weight gain than participants with lower BMI. Importantly, these results were only apparent when participants were judging a virtual body that was their own identity (Experiment 1), but not when they estimated the size of a body with another identity and the same underlying body shape (Experiment 2a). The different influences of BMI on accuracy of body size estimation and sensitivity to weight change for self and other identity suggests that effects of BMI on visual body size estimation are self-specific and not generalizable to other bodies.