A Bio-inspired Model for Bee Simulations.

As eusocial creatures, bees display unique macro collective behavior and local body dynamics that hold potential applications in various fields, such as computer animation, robotics, and social behavior. Unlike birds and fish, bees fly in a low-aligned zigzag pattern. Additionally, bees rely on visual cues for foraging and predator avoidance, exhibiting distinctive local body oscillations, such as body lifting, thrusting, and swaying. These inherent features pose significant challenges for realistic bee simulations in practical animation applications. In this paper, we present a bio-inspired model for bee simulations capable of replicating both macro collective behavior and local body dynamics of bees. Our approach utilizes a visually-driven system to simulate a bee's local body dynamics, incorporating obstacle perception and body rolling control for effective collision avoidance. Moreover, we develop an oscillation rule that captures the dynamics of the bee's local bodies, drawing on insights from biological research. Our model extends beyond simulating individual bees' dynamics; it can also represent bee swarms by integrating a fluid-based field with the bees' innate noise and zigzag motions. To fine-tune our model, we utilize pre-collected honeybee flight data. Through extensive simulations and comparative experiments, we demonstrate that our model can efficiently generate realistic low-aligned and inherently noisy bee swarms.

Multi-Granularity Context Network for Efficient Video Semantic Segmentation.

Current video semantic segmentation tasks involve two main challenges: how to take full advantage of multi-frame context information, and how to improve computational efficiency. To tackle the two challenges simultaneously, we present a novel Multi-Granularity Context Network (MGCNet) by aggregating context information at multiple granularities in a more effective and efficient way. Our method first converts image features into semantic prototypes, and then conducts a non-local operation to aggregate the per-frame and short-term contexts jointly. An additional long-term context module is introduced to capture the video-level semantic information during training. By aggregating both local and global semantic information, a strong feature representation is obtained. The proposed pixel-to-prototype non-local operation requires less computational cost than traditional non-local ones, and is video-friendly since it reuses the semantic prototypes of previous frames. Moreover, we propose an uncertainty-aware and structural knowledge distillation strategy to boost the performance of our method. Experiments on Cityscapes and CamVid datasets with multiple backbones demonstrate that the proposed MGCNet outperforms other state-of-the-art methods with high speed and low latency.

Interfacial Polymerization on Polyethersulfone Ultrafiltration Membrane to Prepare Nanofiltration Layers for Dye Separation.

Nanofiltration membranes are of great significance to the treatment of dye wastewater. Interfacial polymerization is a widely used method to fabricate nanofiltration membranes. In this study, the interaction of tannic acid-assisted polyethylene polyamine (PEPA) with terephthalaldehyde (TPAL) was performed on PES ultrafiltration membranes using novel nitrogen-rich amine monomers and relatively less reactive aldehyde-based monomers. A new nanofiltration membrane ((T-P-T)/PES) was prepared by interfacial polymerization. Fourier transform infrared spectroscopy, X-ray photoelectron spectroscopy, and scanning electron microscopy were used to analyze the elemental composition, bonding state, and surface morphology of the membrane surface. The effects of the PEPA deposition time, TPAL concentration, interfacial reaction time, and curing time on the nanofiltration layer were investigated. The modified membrane, prepared under optimal conditions, showed strong dye separation ability. The permeation of the modified membrane could reach 68.68 L·m-2·h-1·bar-1, and the rejection of various dyes was above 99%. In addition, the (T-P-T)/PES membrane showed good stability during long-term dye separation.

A Calibrated Force-Based Model for Mixed Traffic Simulation.

Virtual traffic benefits a variety of applications, including video games, traffic engineering, autonomous driving, and virtual reality. To date, traffic visualization via different simulation models can reconstruct detailed traffic flows. However, each specific behavior of vehicles is always described by establishing an independent control model. Moreover, mutual interactions between vehicles and other road users are rarely modeled in existing simulators. An all-in-one simulator that considers the complex behaviors of all potential road users in a realistic urban environment is urgently needed. In this work, we propose a novel, extensible, and microscopic method to build heterogeneous traffic simulation using the force-based concept. This force-based approach can accurately replicate the sophisticated behaviors of various road users and their interactions in a simple and unified manner. We calibrate the model parameters using real-world traffic trajectory data. The effectiveness of this approach is demonstrated through many simulation experiments, as well as comparisons to real-world traffic data and popular microscopic simulators for traffic animation.

MNSS: Neural Supersampling Framework for Real-Time Rendering on Mobile Devices.

Although neural supersampling has achieved great success in various applications for improving image quality, it is still difficult to apply it to a wide range of real-time rendering applications due to the high computational power demand. Most existing methods are computationally expensive and require high-performance hardware, preventing their use on platforms with limited hardware, such as smartphones. To this end, we propose a new supersampling framework for real-time rendering applications to reconstruct a high-quality image out of a low-resolution one, which is sufficiently lightweight to run on smartphones within a real-time budget. Our model takes as input the renderer-generated low resolution content and produces high resolution and anti-aliased results. To maximize sampling efficiency, we propose using an alternate sub-pixel sample pattern during the rasterization process. This allows us to create a relatively small reconstruction model while maintaining high image quality. By accumulating new samples into a high-resolution history buffer, an efficient history check and re-usage scheme is introduced to improve temporal stability. To our knowledge, this is the first research in pushing real-time neural supersampling on mobile devices. Due to the absence of training data, we present a new dataset containing 57 training and test sequences from three game scenes. Furthermore, based on the rendered motion vectors and a visual perception study, we introduce a new metric called inter-frame structural similarity (IF-SSIM) to quantitatively measure the temporal stability of rendered videos. Extensive evaluations demonstrate that our supersampling model outperforms existing or alternative solutions in both performance and temporal stability.

MobileSky: Real-Time Sky Replacement for Mobile AR.

We present MobileSky, the first automatic method for real-time high-quality sky replacement for mobile AR applications. The primary challenge of this task is how to extract sky regions in camera feed both quickly and accurately. While the problem of sky replacement is not new, previous methods mainly concern extraction quality rather than efficiency, limiting their application to our task. We aim to provide higher quality, both spatially and temporally consistent sky mask maps for all camera frames in real time. To this end, we develop a novel framework that combines a new deep semantic network called FSNet with novel post-processing refinement steps. By leveraging IMU data, we also propose new sky-aware constraints such as temporal consistency, position consistency, and color consistency to help refine the weakly classified part of the segmentation output. Experiments show that our method achieves an average of around 30 FPS on off-the-shelf smartphones and outperforms the state-of-the-art sky replacement methods in terms of execution speed and quality. In the meantime, our mask maps appear to be visually more stable across frames. Our fast sky replacement method enables several applications, such as AR advertising, art making, generating fantasy celestial objects, visually learning about weather phenomena, and advanced video-based visual effects. To facilitate future research, we also create a new video dataset containing annotated sky regions with IMU data.

Person Foreground Segmentation by Learning Multi-Domain Networks.

Separating the dominant person from the complex background is significant to the human-related research and photo-editing based applications. Existing segmentation algorithms are either too general to separate the person region accurately, or not capable of achieving real-time speed. In this paper, we introduce the multi-domain learning framework into a novel baseline model to construct the Multi-domain TriSeNet Networks for the real-time single person image segmentation. We first divide training data into different subdomains based on the characteristics of single person images, then apply a multi-branch Feature Fusion Module (FFM) to decouple the networks into the domain-independent and the domain-specific layers. To further enhance the accuracy, a self-supervised learning strategy is proposed to dig out domain relations during training. It helps transfer domain-specific knowledge by improving predictive consistency among different FFM branches. Moreover, we create a large-scale single person image segmentation dataset named MSSP20k, which consists of 22,100 pixel-level annotated images in the real world. The MSSP20k dataset is more complex and challenging than existing public ones in terms of scalability and variety. Experiments show that our Multi-domain TriSeNet outperforms state-of-the-art approaches on both public and the newly built datasets with real-time speed.

Pathway knockout and redundancy in metabolic networks.

The robustness and stability of complex cellular networks is often attributed to the redundancy of components, including genes, enzymes and pathways. Estimation of redundancy is still an open question in systems biology. Current theoretical tools to measure redundancy have various strengths and shortcomings in providing a comprehensive description of metabolic networks. Specially, there is a lack of effective measures to cover different perturbation situations. Here we present a pathway knockout algorithm to improve quantitative measure of redundancy in metabolic networks grounded on the elementary flux mode (EFM) analysis. The proposed redundancy measure is based on the average ratio of remaining EFMs after knockout of one EFM in the unperturbed state. We demonstrated with four example systems that our algorithm overcomes limits of previous measures, and provides additional information about redundancy in the situation of targeted attacks. Additionally, we compare existing enzyme knockout and our pathway knockout algorithm by the mean-field analysis, which provides mathematical expression for the average ratio of remaining EFMs after both types of knockout. Our results prove that multiple-enzymes knockout does not always yield more information than single-enzyme knockout for evaluating redundancy. Indeed, pathway knockout considers additional effects of structural asymmetry. In the metabolic networks of amino acid anabolism in Escherichia coli and human hepatocytes, and the central metabolism in human erythrocytes, we validate our mean-field solutions and prove the capacity of pathway knockout algorithm. Moreover, in the E. coli model the two sub-networks synthesizing amino acids that are essential and those that are non-essential for humans are studied separately. In contrast to previous studies, we find that redundancy of two sub-networks is similar with each other, and even sub-networks synthesizing essential amino acids can be more redundant.

Inner Surface Hydrophilic Modification of PVDF Membrane with Tea Polyphenols/Silica Composite Coating.

The use of Polyvinylidene fluoride (PVDF) membranes is constrained in wastewater treatment because of their hydrophobic nature. Therefore, a large number of researchers have been working on the hydrophilic modification of their surfaces. In this work, a superhydrophilic tea polyphenols/silica composite coating was developed by a one-step process. The composite coating can achieve not only superhydrophilic modification of the surface, but also the inner surface of the porous PVDF membrane, which endows the modified membrane with excellent water permeability. The modified membrane possesses ultrahigh water flux (15,353 L·m-2·h-1). Besides this, the modified membrane can realize a highly efficient separation of oil/water emulsions (above 96%).

Heter-Sim: Heterogeneous Multi-Agent Systems Simulation by Interactive Data-Driven Optimization.

Interactive multi-agent simulation algorithms are used to compute the trajectories and behaviors of different entities in virtual reality scenarios. However, current methods involve considerable parameter tweaking to generate plausible behaviors. We introduce a novel approach (Heter-Sim) that combines physics-based simulation methods with data-driven techniques using an optimization-based formulation. Our approach is general and can simulate heterogeneous agents corresponding to human crowds, traffic, vehicles, or combinations of different agents with varying dynamics. We estimate motion states from real-world datasets that include information about position, velocity, and control direction. Our optimization algorithm considers several constraints, including velocity continuity, collision avoidance, attraction, direction control. Other constraints are implemented by introducing a novel energy function to control the motions of heterogeneous agents. To accelerate the computations, we reduce the search space for both collision avoidance and optimal solution computation. Heter-Sim can simulate tens or hundreds of agents at interactive rates and we compare its accuracy with real-world datasets and prior algorithms. We also perform user studies that evaluate the plausible behaviors generated by our algorithm and a user study that evaluates the plausibility of our algorithm via VR.

3D Upper Body Reconstruction with Sparse Soft Sensors.

Three-dimensional (3D) reconstruction of human body has wide applications, for example, for customized design of clothes and digital avatar production. Existing vision-based systems for 3D body reconstruction require users to wear minimal or extreme-tight clothes in front of cameras, and thus suffer from privacy problems. In this work, we explore a novel solution based on a sparse number of soft sensors on a standard garment, and use it for capturing 3D upper body shape. We utilize the maximal stretching range by modeling the nonlinear performance profile for individual sensors. The body shape can be dynamically reconstructed by analyzing the relationship between mesh deformation and sensor reading, with a learning-based approach. The wearability and flexibility of our prototype allow its use in indoor/outdoor environments and for long-term breath monitoring. Our prototype has been extensively evaluated by multiple users with different body sizes and the same user for multiple days. The results show that our garment prototype is comfortable to wear, and achieves the state-of-the-art reconstruction performance with the advantages in privacy projection and application scenarios.

AgentDress: Realtime Clothing Synthesis for Virtual Agents using Plausible Deformations.

We present a CPU-based real-time cloth animation method for dressing virtual humans of various shapes and poses. Our approach formulates the clothing deformation as a high-dimensional function of body shape parameters and pose parameters. In order to accelerate the computation, our formulation factorizes the clothing deformation into two independent components: the deformation introduced by body pose variation (Clothing Pose Model) and the deformation from body shape variation (Clothing Shape Model). Furthermore, we sample and cluster the poses spanning the entire pose space and use those clusters to efficiently calculate the anchoring points. We also introduce a sensitivity-based distance measurement to both find nearby anchoring points and evaluate their contributions to the final animation. Given a query shape and pose of the virtual agent, we synthesize the resulting clothing deformation by blending the Taylor expansion results of nearby anchoring points. Compared to previous methods, our approach is general and able to add the shape dimension to any clothing pose model. Furthermore, we can animate clothing represented with tens of thousands of vertices at 50+ FPS on a CPU. We also conduct a user evaluation and show that our method can improve a user's perception of dressed virtual agents in an immersive virtual environment (IVE) compared to a realtime linear blend skinning method.

Polyelectrolyte-calcium complexes as a pre-precursor induce biomimetic mineralization of collagen.

Polyelectrolytes such as polyaspartic acid (PAsp) are critical in biomimetic mineralization as stabilizers of amorphous calcium phosphate (ACP) precursors and as nucleation inhibitors similar to non-collagenous proteins (NCPs). Nevertheless, the application of polyelectrolyte-calcium complexes as a pre-precursor, such as PAsp-Ca complexes, in the mineralization of collagen is unexplored. Herein, we propose a polyelectrolyte-Ca complex pre-precursor (PCCP) process for collagen mineralization. By combining three-dimensional (3D) STORM, potential measurements, and cryogenic transmission electron microscopy with molecular dynamics simulations, we show that liquid-like electropositive PAsp-Ca complexes along with free calcium ions infiltrate electronegative collagen fibrils. The PAsp-Ca complexes are immobilized within the fibrils via chelation and hydrogen bonds, and outward movement of free calcium ions is prevented while phosphate and hydroxide are recruited through electrostatic attractions. Afterwards, ACP instantly forms and gradually crystallizes. The PCCP process not only unites two distinct crystallization pathways (classical (free Ca/P ions) and non-classical (polyelectrolyte-Ca complexes)), but also provides a novel strategy for rapid biomimetic mineralization of collagen.

Dictionary-based Fidelity Measure for Virtual Traffic.

Aiming at objectively measuring the realism of virtual traffic flows and evaluating the effectiveness of different traffic simulation techniques, this paper introduces a general, dictionary-based learning method to evaluate the fidelity of any traffic trajectory data. First, a traffic pattern dictionary that characterizes common patterns of real-world traffic behavior is built offline from pre-collected ground truth traffic data. The corresponding learning error is set as the benchmark of the dictionary-based traffic representation. With the aid of the constructed dictionary, the realism of input simulated traffic flow data can be evaluated by comparing its dictionary-based reconstruction error with the dictionary error benchmark. This evaluation metric can be robustly applied to any simulated traffic flow data; in other words, it is independent of how the traffic data are generated. We demonstrated the effectiveness and robustness of this metric through many experiments on real-world traffic data and various simulated traffic data, comparisons with the state-of-the-art entropy-based similarity metric for aggregate crowd motions, and perceptual evaluation studies.

Endogenetic structure of filter bubble in social networks.

The filter bubble is an intermediate structure to provoke polarization and echo chambers in social networks, and it has become one of today's most urgent issues for social media. Previous studies usually equated filter bubbles with community structures and emphasized this exogenous isolation effect, but there is a lack of full discussion of the internal organization of filter bubbles. Here, we design an experiment for analysing filter bubbles taking advantage of social bots. We deployed 128 bots to Weibo (the largest microblogging network in China), and each bot consumed a specific topic (entertainment or sci-tech) and ran for at least two months. In total, we recorded about 1.3 million messages exposed to these bots and their social networks. By analysing the text received by the bots and motifs in their social networks, we found that a filter bubble is not only a dense community of users with the same preferences but also presents an endogenetic unidirectional star-like structure. The structure could spontaneously exclude non-preferred information and cause polarization. Moreover, our work proved that the felicitous use of artificial intelligence technology could provide a useful experimental approach that combines privacy protection and controllability in studying social media.

Solid Mathematical Marbling.

Years of research have been devoted to computer-generated two-dimensional marbling. However, three-dimensional marbling has yet to be explored. In this paper, we present mathematical marbling of three-dimensional solids which supports a compact random-access vector representation. Our solid marbling textures are created by composing closed-form 3D pattern tool functions. These tool functions are an injection function and five deformation functions. The injection function is used to generate basic patterns, and the deformation functions are responsible for transforming the basic pattern into complex marbling effects. The resulting representation is feature preserving and resolution-independent. Our approach can render high-quality images preserving both the sharp features and the smooth color variations of a solid texture. When implemented on the GPU, our representation enables efficient color evaluation during the real-time solid marbling texture mapping. The color of a point in the volume space is computed by the 3D pattern tool functions from its coordinates. Our method consumes very little memory because only the mathematical functions and their corresponding parameters are stored. In addition, we develop an intuitive user interface and a genetic algorithm to facilitate the solid marbling texture authoring process. We demonstrate the effectiveness of our approach through various solid marbling textures and 3D objects carved from them.

Quadruplet Network With One-Shot Learning for Fast Visual Object Tracking.

In the same vein of discriminative one-shot learning, Siamese networks allow recognizing an object from a single exemplar with the same class label. However, they do not take advantage of the underlying structure of the data and the relationship among the multitude of samples as they only rely on the pairs of instances for training. In this paper, we propose a new quadruplet deep network to examine the potential connections among the training instances, aiming to achieve a more powerful representation. We design a shared network with four branches that receive a multi-tuple of instances as inputs and are connected by a novel loss function consisting of pair loss and triplet loss. According to the similarity metric, we select the most similar and the most dissimilar instances as the positive and negative inputs of triplet loss from each multi-tuple. We show that this scheme improves the training performance. Furthermore, we introduce a new weight layer to automatically select suitable combination weights, which will avoid the conflict between triplet and pair loss leading to worse performance. We evaluate our quadruplet framework by model-free tracking-by-detection of objects from a single initial exemplar in several visual object tracking benchmarks. Our extensive experimental analysis demonstrates that our tracker achieves superior performance with a real-time processing speed of 78 frames/s. Our source code is available.

Submodular Function Optimization for Motion Clustering and Image Segmentation.

In this paper, we propose a framework of maximizing quadratic submodular energy with a knapsack constraint approximately, to solve certain computer vision problems. The proposed submodular maximization problem can be viewed as a generalization of the classic 0/1 knapsack problem. Importantly, maximization of our knapsack constrained submodular energy function can be solved via dynamic programing. We further introduce a range-reduction step prior to dynamic programing as a two-stage procedure for more efficient maximization. In order to demonstrate the effectiveness of the proposed energy function and its maximization algorithm, we apply it to two representative computer vision tasks: image segmentation and motion trajectory clustering. Experimental results of image segmentation demonstrate that our method outperforms the classic segmentation algorithms of graph cuts and random walks. Moreover, our framework achieves better performance than state-of-the-art methods on the motion trajectory clustering task.

Mesh composition on models with arbitrary boundary topology.

This paper presents a new approach for the mesh composition on models with arbitrary boundary topology. After cutting the needed parts from existing mesh models and putting them into the right pose, an implicit surface is adopted to smoothly interpolate the boundaries of models under composition. An interface is developed to control the shape of the implicit transient surface by using sketches to specify the expected silhouettes. After that, a localized Marching Cubes algorithm is investigated to tessellate the implicit transient surface so that the mesh surface of composed model is generated. Different from existing approaches in which the models under composition are required to have pairwise merging boundaries, the framework developed based on our techniques have the new function to fuse models with arbitrary boundary topology.

Realistic Data-Driven Traffic Flow Animation Using Texture Synthesis.

We present a novel data-driven approach to populate virtual road networks with realistic traffic flows. Specifically, given a limited set of vehicle trajectories as the input samples, our approach first synthesizes a large set of vehicle trajectories. By taking the spatio-temporal information of traffic flows as a 2D texture, the generation of new traffic flows can be formulated as a texture synthesis process, which is solved by minimizing a newly developed traffic texture energy. The synthesized output captures the spatio-temporal dynamics of the input traffic flows, and the vehicle interactions in it strictly follow traffic rules. After that, we position the synthesized vehicle trajectory data to virtual road networks using a cage-based registration scheme, where a few traffic-specific constraints are enforced to maintain each vehicle's original spatial location and synchronize its motion in concert with its neighboring vehicles. Our approach is intuitive to control and scalable to the complexity of virtual road networks. We validated our approach through many experiments and paired comparison user studies.