Secrets of Event-based Optical Flow, Depth and Ego-motion Estimation by Contrast Maximization.

Event cameras respond to scene dynamics and provide signals naturally suitable for motion estimation with advantages, such as high dynamic range. The emerging field of event-based vision motivates a revisit of fundamental computer vision tasks related to motion, such as optical flow and depth estimation. However, state-of-the-art event-based optical flow methods tend to originate in frame-based deep-learning methods, which require several adaptations (data conversion, loss function, etc.) as they have very different properties. We develop a principled method to extend the Contrast Maximization framework to estimate dense optical flow, depth, and ego-motion from events alone. The proposed method sensibly models the space-time properties of event data and tackles the event alignment problem. It designs the objective function to prevent overfitting, deals better with occlusions, and improves convergence using a multi-scale approach. With these key elements, our method ranks first among unsupervised methods on the MVSEC benchmark and is competitive on the DSEC benchmark. Moreover, it allows us to simultaneously estimate dense depth and ego-motion, exposes the limitations of current flow benchmarks, and produces remarkable results when it is transferred to unsupervised learning settings. Along with various downstream applications shown, we hope the proposed method becomes a cornerstone on event-based motion-related tasks. Code is available at https://github.com/tub-rip/event_based_optical_flow.

Beyond helium: hydrogen as a carrier gas in multiresidue pesticide analysis in fruits and vegetables by GC-MS/MS.

In this comprehensive study, we evaluated the feasibility of using hydrogen instead of helium as a carrier gas in a GC-MS/MS system for pesticide residue analysis, spanning three matrices: pepper, tomato, and zucchini. Initial assessments focused on the ion source's chemical inertness, employing nitrobenzene as a benchmark to monitor the hydrogenation process. A method with a duration of less than 12 minutes was developed, achieving good chromatographic peak resolution attributable to the enhanced chromatographic performance of hydrogen as a carrier gas. The study emphasized the optimization of system parameters, testing various ion source temperatures, detector voltages, and injection volumes. Sensitivity assessments, based on the DG-SANTE criteria, indicated that the majority of compounds were identifiable at a concentration of 5 µg kg-1 (81% in tomato, 84% in pepper and 73% in zucchini). Detailed validation for reproducibility, matrix effects, and linearity across 150 pesticides unveiled generally favorable outcomes, with a notable majority of compounds displaying low matrix effects, satisfactory linearity ranges and good reproducibility with most compounds returning a relative standard deviation (RSD) below 10%. When applied to 15 real samples, the hydrogen-based system's performance was juxtaposed against a helium-based counterpart, revealing that results are very comparable between both systems. This comparative approach highlights hydrogen's potential as a reliable and efficient carrier gas in pesticide residue analysis for routine food control laboratories, overcoming difficulties resulting from the lack of helium supplies.

Environmental assessment of PAHs through honey bee colonies - A matrix selection study.

The steady conditions of temperature, humidity and air flux within beehives make them a valuable location for conducting environmental monitoring of pollutants such as PAHs. In this context, the selection of an appropriate apicultural matrix plays a key role in these monitoring studies, as it maximizes the information that will be obtained in the analyses while minimizing the inaccurate results. In the present study, three apicultural matrices (honey bees, pollen and propolis) and two passive samplers (APIStrips and silicone wristbands) are compared in terms of the number and total load of PAHs detected in them. Samplings took place in a total of 11 apiaries scattered in Austria, Denmark, and Greece, with analyses performed by GC-MS/MS. Up to 14 different PAHs were identified in silicone wristbands and pollen, whereas the remaining matrices contained a maximum of five contaminants. Naphthalene, 1-methylnaphthalene, 2-methylnaphthalene, and pyrene were found to be the most prevalent substances in the environment. Recovery studies were also performed; these suggested that the chemical structure of APIStrips is likely to produce very strong interactions with PAHs, thus hindering the adequate desorption of these substances from their surface. Overall, silicone wristbands placed inside the beehives proved the most suitable matrix for PAH monitoring through honey bee colonies.

Event-Based Background-Oriented Schlieren.

Schlieren imaging is an optical technique to observe the flow of transparent media, such as air or water, without any particle seeding. However, conventional frame-based techniques require both high spatial and temporal resolution cameras, which impose bright illumination and expensive computation limitations. Event cameras offer potential advantages (high dynamic range, high temporal resolution, and data efficiency) to overcome such limitations due to their bio-inspired sensing principle. This article presents a novel technique for perceiving air convection using events and frames by providing the first theoretical analysis that connects event data and schlieren. We formulate the problem as a variational optimization one combining the linearized event generation model with a physically-motivated parameterization that estimates the temporal derivative of the air density. The experiments with accurately aligned frame- and event camera data reveal that the proposed method enables event cameras to obtain on par results with existing frame-based optical flow techniques. Moreover, the proposed method works under dark conditions where frame-based schlieren fails, and also enables slow-motion analysis by leveraging the event camera's advantages. Our work pioneers and opens a new stack of event camera applications, as we publish the source code as well as the first schlieren dataset with high-quality frame and event data.

Formulating Event-Based Image Reconstruction as a Linear Inverse Problem With Deep Regularization Using Optical Flow.

Event cameras are novel bio-inspired sensors that measure per-pixel brightness differences asynchronously. Recovering brightness from events is appealing since the reconstructed images inherit the high dynamic range (HDR) and high-speed properties of events; hence they can be used in many robotic vision applications and to generate slow-motion HDR videos. However, state-of-the-art methods tackle this problem by training an event-to-image Recurrent Neural Network (RNN), which lacks explainability and is difficult to tune. In this work we show, for the first time, how tackling the combined problem of motion and brightness estimation leads us to formulate event-based image reconstruction as a linear inverse problem that can be solved without training an image reconstruction RNN. Instead, classical and learning-based regularizers are used to solve the problem and remove artifacts from the reconstructed images. The experiments show that the proposed approach generates images with visual quality on par with state-of-the-art methods despite only using data from a short time interval. State-of-the-art results are achieved using an image denoising Convolutional Neural Network (CNN) as the regularization function. The proposed regularized formulation and solvers have a unifying character because they can be applied also to reconstruct brightness from the second derivative. Additionally, the formulation is attractive because it can be naturally combined with super-resolution, motion-segmentation and color demosaicing. Code is available at https://github.com/tub-rip/event_based_image_rec_inverse_problem.

Event-Based Motion Segmentation With Spatio-Temporal Graph Cuts.

Identifying independently moving objects is an essential task for dynamic scene understanding. However, traditional cameras used in dynamic scenes may suffer from motion blur or exposure artifacts due to their sampling principle. By contrast, event-based cameras are novel bio-inspired sensors that offer advantages to overcome such limitations. They report pixel-wise intensity changes asynchronously, which enables them to acquire visual information at exactly the same rate as the scene dynamics. We develop a method to identify independently moving objects acquired with an event-based camera, that is, to solve the event-based motion segmentation problem. We cast the problem as an energy minimization one involving the fitting of multiple motion models. We jointly solve two sub-problems, namely event-cluster assignment (labeling) and motion model fitting, in an iterative manner by exploiting the structure of the input event data in the form of a spatio-temporal graph. Experiments on available datasets demonstrate the versatility of the method in scenes with different motion patterns and number of moving objects. The evaluation shows state-of-the-art results without having to predetermine the number of expected moving objects. We release the software and dataset under an open source license to foster research in the emerging topic of event-based motion segmentation.

Event Collapse in Contrast Maximization Frameworks.

Contrast maximization (CMax) is a framework that provides state-of-the-art results on several event-based computer vision tasks, such as ego-motion or optical flow estimation. However, it may suffer from a problem called event collapse, which is an undesired solution where events are warped into too few pixels. As prior works have largely ignored the issue or proposed workarounds, it is imperative to analyze this phenomenon in detail. Our work demonstrates event collapse in its simplest form and proposes collapse metrics by using first principles of space-time deformation based on differential geometry and physics. We experimentally show on publicly available datasets that the proposed metrics mitigate event collapse and do not harm well-posed warps. To the best of our knowledge, regularizers based on the proposed metrics are the only effective solution against event collapse in the experimental settings considered, compared with other methods. We hope that this work inspires further research to tackle more complex warp models.

Event-Based Vision: A Survey.

Event cameras are bio-inspired sensors that differ from conventional frame cameras: Instead of capturing images at a fixed rate, they asynchronously measure per-pixel brightness changes, and output a stream of events that encode the time, location and sign of the brightness changes. Event cameras offer attractive properties compared to traditional cameras: high temporal resolution (in the order of µs), very high dynamic range (140 dB versus 60 dB), low power consumption, and high pixel bandwidth (on the order of kHz) resulting in reduced motion blur. Hence, event cameras have a large potential for robotics and computer vision in challenging scenarios for traditional cameras, such as low-latency, high speed, and high dynamic range. However, novel methods are required to process the unconventional output of these sensors in order to unlock their potential. This paper provides a comprehensive overview of the emerging field of event-based vision, with a focus on the applications and the algorithms developed to unlock the outstanding properties of event cameras. We present event cameras from their working principle, the actual sensors that are available and the tasks that they have been used for, from low-level vision (feature detection and tracking, optic flow, etc.) to high-level vision (reconstruction, segmentation, recognition). We also discuss the techniques developed to process events, including learning-based techniques, as well as specialized processors for these novel sensors, such as spiking neural networks. Additionally, we highlight the challenges that remain to be tackled and the opportunities that lie ahead in the search for a more efficient, bio-inspired way for machines to perceive and interact with the world.

Topically Applied Etamsylate: A New Orphan Drug for HHT-Derived Epistaxis (Antiangiogenesis through FGF Pathway Inhibition).

Hereditary hemorrhagic telangiectasia (HHT) is a vascular dysplasia characterized by recurrent and spontaneous epistaxis (nose bleeds), telangiectases on skin and mucosa, internal organ arteriovenous malformations, and dominant autosomal inheritance. Mutations in Endoglin and ACVRL1 / ALK1 , genes mainly expressed in endothelium, are responsible in 90% of the cases for the pathology. These genes are involved in the transforming growth factor-ß(TGF-ß) signaling pathway. Epistaxis remains as one of the most common symptoms impairing the quality of life of patients, becoming life-threatening in some cases. Different strategies have been used to decrease nose bleeds, among them is antiangiogenesis. The two main angiogenic pathways in endothelial cells depend on vascular endothelial growth factor and fibroblast growth factor (FGF). The present work has used etamsylate, the diethylamine salt of the 2,5-dihydroxybenzene sulfonate anion, also known as dobesilate, as a FGF signaling inhibitor. In endothelial cells, in vitro experiments show that etamsylate acts as an antiangiogenic factor, inhibiting wound healing and matrigel tubulogenesis. Moreover, etamsylate decreases phosphorylation of Akt and ERK1/2. A pilot clinical trial (EudraCT: 2016-003982-24) was performed with 12 HHT patients using a topical spray of etamsylate twice a day for 4 weeks. The epistaxis severity score (HHT-ESS) and other pertinent parameters were registered in the clinical trial. The significant reduction in the ESS scale, together with the lack of significant side effects, allowed the designation of topical etamsylate as a new orphan drug for epistaxis in HHT (EMA/OD/135/18).

Event-Based, 6-DOF Camera Tracking from Photometric Depth Maps.

Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. These cameras do not suffer from motion blur and have a very high dynamic range, which enables them to provide reliable visual information during high-speed motions or in scenes characterized by high dynamic range. These features, along with a very low power consumption, make event cameras an ideal complement to standard cameras for VR/AR and video game applications. With these applications in mind, this paper tackles the problem of accurate, low-latency tracking of an event camera from an existing photometric depth map (i.e., intensity plus depth information) built via classic dense reconstruction pipelines. Our approach tracks the 6-DOF pose of the event camera upon the arrival of each event, thus virtually eliminating latency. We successfully evaluate the method in both indoor and outdoor scenes and show that-because of the technological advantages of the event camera-our pipeline works in scenes characterized by high-speed motion, which are still inaccessible to standard cameras.

Hidden α-helical propensity segments within disordered regions of the transcriptional activator CHOP.

C/EBP-homologous protein (CHOP) is a key determinant of the apoptotic response to endoplasmic reticulum stress or DNA damage. As a member of the C/EBP family, CHOP contains a low complexity N-terminal region involved in transcriptional activation, followed by a bZIP that binds DNA after dimerization. However, in contrast to other C/EBPs, CHOP directs binding to non-canonical C/EBP sites due to unique substitutions in its DNA-binding domain. Herein, we show that the N-terminal region of CHOP is intrinsically unstructured but contains two segments presenting α-helical propensity. One of these segments is conserved in other C/EBPs and mediates essential roles of CHOP, including regulation through phosphorylation. The second segment is placed within a proteolytic-resistant portion of the protein and exhibits reduced flexibility. Moreover, the DNA-binding region of CHOP also contains a segment with α-helical character towards its most N-terminal part. Our results suggest that structure-prone segments scattered within disordered regions may be critical for macromolecular recognition during CHOP-mediated transcriptional activation.

Augmented Reality Tool for the Situational Awareness Improvement of UAV Operators.

Unmanned Aerial Vehicles (UAVs) are being extensively used nowadays. Therefore, pilots of traditional aerial platforms should adapt their skills to operate them from a Ground Control Station (GCS). Common GCSs provide information in separate screens: one presents the video stream while the other displays information about the mission plan and information coming from other sensors. To avoid the burden of fusing information displayed in the two screens, an Augmented Reality (AR) tool is proposed in this paper. The AR system has two functionalities for Medium-Altitude Long-Endurance (MALE) UAVs: route orientation and target identification. Route orientation allows the operator to identify the upcoming waypoints and the path that the UAV is going to follow. Target identification allows a fast target localization, even in the presence of occlusions. The AR tool is implemented following the North Atlantic Treaty Organization (NATO) standards so that it can be used in different GCSs. The experiments show how the AR tool improves significantly the situational awareness of the UAV operators.

Optimal Piecewise Linear Function Approximation for GPU-Based Applications.

Many computer vision and human-computer interaction applications developed in recent years need evaluating complex and continuous mathematical functions as an essential step toward proper operation. However, rigorous evaluation of these kind of functions often implies a very high computational cost, unacceptable in real-time applications. To alleviate this problem, functions are commonly approximated by simpler piecewise-polynomial representations. Following this idea, we propose a novel, efficient, and practical technique to evaluate complex and continuous functions using a nearly optimal design of two types of piecewise linear approximations in the case of a large budget of evaluation subintervals. To this end, we develop a thorough error analysis that yields asymptotically tight bounds to accurately quantify the approximation performance of both representations. It provides an improvement upon previous error estimates and allows the user to control the tradeoff between the approximation error and the number of evaluation subintervals. To guarantee real-time operation, the method is suitable for, but not limited to, an efficient implementation in modern graphics processing units, where it outperforms previous alternative approaches by exploiting the fixed-function interpolation routines present in their texture units. The proposed technique is a perfect match for any application requiring the evaluation of continuous functions; we have measured in detail its quality and efficiency on several functions, and, in particular, the Gaussian function because it is extensively used in many areas of computer vision and cybernetics, and it is expensive to evaluate.

Improvement in the signs and symptoms of dry eye disease with dobesilate eye drops.

BACKGROUND: Dry eye is a multifactor disease of the tear film and ocular surface that substantially affects quality of life. CASE PRESENTATION: Dobesilate administered as eye drops was well tolerated and effective in treating both the objective signs and subjective symptoms of dry eye disease in this 2-week study. CONCLUSION: To the best of our knowledge, this is the first clinical report of using dobesilate in eye drops. Dobesilate may provide a novel approach to treating drying diseases of the eye.

Dramatic resolution of vitreous hemorrhage after an intravitreal injection of dobesilate.

Vitreous hemorrhages are important clinical manifestations of proliferative diabetic retinopathy. Non-cleared vitreous hemorrhages could lead to hemosiderosis bulbi and glaucoma. Here, we describe the case of a type 2 diabetic patient presenting anterior segment and vitreous hemorrhages that resolved three days after treatment with a single intravitreal injection of dobesilate.

Discovery and characterization of an endogenous CXCR4 antagonist.

CXCL12-CXCR4 signaling controls multiple physiological processes and its dysregulation is associated with cancers and inflammatory diseases. To discover as-yet-unknown endogenous ligands of CXCR4, we screened a blood-derived peptide library for inhibitors of CXCR4-tropic HIV-1 strains. This approach identified a 16 amino acid fragment of serum albumin as an effective and highly specific CXCR4 antagonist. The endogenous peptide, termed EPI-X4, is evolutionarily conserved and generated from the highly abundant albumin precursor by pH-regulated proteases. EPI-X4 forms an unusual lasso-like structure and antagonizes CXCL12-induced tumor cell migration, mobilizes stem cells, and suppresses inflammatory responses in mice. Furthermore, the peptide is abundant in the urine of patients with inflammatory kidney diseases and may serve as a biomarker. Our results identify EPI-X4 as a key regulator of CXCR4 signaling and introduce proteolysis of an abundant precursor protein as an alternative concept for chemokine receptor regulation.

Diacetyloxyl derivatization of the fibroblast growth factor inhibitor dobesilate enhances its anti-inflammatory, anti-angiogenic and anti-tumoral activities.

BACKGROUND: Dobesilate (2,5-dihydroxyphenyl sulfonate, DHPS) was recently identified as the most potent member of a family of fibroblast growth factor (FGF) inhibitors headed by gentisic acid, one of the main catabolites of aspirin. Although FGFs were first described as inducers of angiogenesis, they were soon recognized as broad spectrum mitogens. Furthermore, in the last decade these proteins have been shown to participate directly in the onset of inflammation, and their potential angiogenic activity often contributes to the inflammatory process in vivo. The aim of this work was to evaluate the anti-inflammatory, anti-angiogenic and anti-tumoral activities of the derivative of DHPS obtained by acetoxylation of its two hydroxyl groups (2,5-diacetoxyphenyl sulfonate; DAPS). METHODS: Anti-inflammatory, anti-angiogenic and anti-tumoral activities of DHPS and DAPS were compared using in vivo assays of dermatitis, angiogenesis and tumorigenesis. The effects of both compounds on myeloperoxidase (MPO) and cyclooxygenase (COX) activities, cytokine production and FGF-induced fibroblast proliferation were also determined. RESULTS: Topical DAPS is more effective than DHPS in preventing inflammatory signs (increased vascular permeability, edema, leukocyte infiltration, MPO activation) caused by contact dermatitis induction in rat ears. DAPS, but not DHPS, effectively inhibits COX-1 and COX-2 activities. DAPS also reduces the increase in serum cytokine concentration induced by lipopolysaccharide in rats. Furthermore, DAPS displays higher in vivo efficacy than DHPS in inhibiting FGF-induced angiogenesis and heterotopic glioma progression, with demonstrated oral efficacy to combat both processes. CONCLUSIONS: By inhibiting both FGF-signaling and COX-mediated prostaglandin synthesis, DAPS efficiently breaks the vicious circle created by the reciprocal induction of FGF and prostaglandins, which probably sustains undesirable inflammation in many circumstances. Our findings define the enhancement of anti-inflammatory, anti-angiogenic and anti-tumoral activities by diacetyloxyl derivatization of the FGF inhibitor, dobesilate.

Increasing the Dose of Autologous Chondrocytes Improves Articular Cartilage Repair: Histological and Molecular Study in the Sheep Animal Model.

BACKGROUND: We hypothesized that implanting cells in a chondral defect at a density more similar to that of the intact cartilage could induce them to synthesize matrix with the features more similar to that of the uninjured one. METHODS: We compared the implantation of different doses of chondrocytes: 1 million (n = 5), 5 million (n = 5), or 5 million mesenchymal cells (n = 5) in the femoral condyle of 15 sheep. Tissue generated by microfracture at the trochlea, and normal cartilage from a nearby region, processed as the tissues resulting from the implantation, were used as references. Histological and molecular (expression of type I and II collagens and aggrecan) studies were performed. RESULTS: The features of the cartilage generated by implantation of mesenchymal cells and elicited by microfractures were similar and typical of a poor repair of the articular cartilage (presence of fibrocartilage, high expression of type I collagen and a low mRNA levels of type II collagen and aggrecan). Nevertheless, in the samples obtained from tissues generated by implantation of chondrocytes, hyaline-like cartilage, cell organization, low expression rates of type I collagen and high levels of mRNA corresponding to type II collagen and aggrecan were observed. These histological features, show less variability and are more similar to those of the normal cartilage used as control in the case of 5 million cells implantation than when 1 million cells were used. CONCLUSIONS: The implantation of autologous chondrocytes in type I/III collagen membranes at high density could be a promising tool to repair articular cartilage.

Influence of acidic fibroblast growth factor on bone regeneration in experimental cranial defects using spongostan and Bio-Oss as protein carriers.

The objective of this study was to valuate 2 substances as potential carriers of fibroblast growth factor 1 (FGF-1) in a rat craniectomy model: gelatin sponge (Spongostan; Ferrosan A/S, Søborg, Denmark) and natural bone mineral (Bio-Oss; Geistlich Biomaterials, Wolhusen, Switzerland).Forty-eight adult male Sprague-Dawley rats were used. A 5-mm-diameter circular craniectomy was performed in the left parietal bone. Animals were divided into 6 experimental groups of 8 rats, each group receiving a different treatment: control (no substance added), Spongostan, Bio-Oss, FGF, FGF + Spongostan, and FGF + Bio-Oss. Animals were killed 12 weeks after surgery.Descriptive histology and stereology were used, the latter to measure the volumes of regenerated bone and Bio-Oss remaining in the defect. Analysis of variance was used to determine differences in bone regeneration between groups, and Mann-Whitney U test was used to compare the volume of remaining Bio-Oss particles.Histologically, the control defects behaved like critical size defects, showing incomplete bone regeneration. Only the FGF + Spongostan group achieved nearly complete bone regeneration. Bio-Oss particles seemed to reduce centripetal bone regeneration. Spongostan by itself did not interfere with spontaneous bone healing.Stereologic measurements of the volume of new bone growth, measured in cubic millimeter, were as follows: control group, 3.86 ± 1.03; Bio-Oss, 2.26 ± 1.06; Spongostan, 3.00 ± 0.81; FGF, 3.99 ± 1.85; FGF + Bio-Oss, 3.02 ± 1.88; and FGF + Spongostan, 8.93 ± 1.28. Analysis of variance showed a statistically significant difference between the FGF + Spongostan group and the other groups (P < 0.001). Comparison among the other groups did not show significant differences.Fibroblast growth factor 1 with a Spongostan carrier has shown great efficacy for bone regeneration in cranial critical size defects in rats. Bio-Oss did not produce a regenerative effect, either alone or with FGF-1.

Heparin modulates the mitogenic activity of fibroblast growth factor by inducing dimerization of its receptor. a 3D view by using NMR.

In vitro mitogenesis assays have shown that sulfated glycosaminoglycans (GAGs; heparin and heparan sulfate) cause an enhancement of the mitogenic activity of fibroblast growth factors (FGFs). Herein, we report that the simultaneous presence of FGF and the GAG is not an essential requisite for this event to take place. Indeed, preincubation with heparin (just before FGF addition) of cells lacking heparan sulfate produced an enhancing effect equivalent to that observed when the GAG and the protein are simultaneously added. A first structural characterization of this effect by analytical ultracentrifugation of a soluble preparation of the heparin-binding domain of fibroblast growth factor receptor 2 (FGFR2) and a low molecular weight (3 kDa) heparin showed that the GAG induces dimerization of FGFR2. To derive a high resolution structural picture of this molecular recognition process, the interactions of a soluble heparin-binding domain of FGFR2 with two different homogeneous, synthetic, and mitogenically active sulfated GAGs were analyzed by NMR spectroscopy. These studies, assisted by docking protocols and molecular dynamics simulations, have demonstrated that the interactions of these GAGs with the soluble heparin-binding domain of FGFR induces formation of an FGFR dimer; its architecture is equivalent to that in one of the two distinct crystallographic structures of FGFR in complex with both heparin and FGF1. This preformation of the FGFR dimer (with similar topology to that of the signaling complex) should favor incorporation of the FGF component to form the final assemblage of the signaling complex, without major entropy penalty. This cascade of events is probably at the heart of the observed activating effect of heparin in FGF-driven mitogenesis.