Multi-Scale Marine Object Detection in Side-Scan Sonar Images Based on BES-YOLO.

Aiming at the problem of low accuracy of multi-scale seafloor target detection in side-scan sonar images with high noise and complex background texture, a model for multi-scale target detection using the BES-YOLO network is proposed. First, an efficient multi-scale attention (EMA) mechanism is used in the backbone of the YOLOv8 network, and a bi-directional feature pyramid network (Bifpn) is introduced to merge the information of different scales, finally, a Shape_IoU loss function is introduced to continuously optimize the model and improve its accuracy. Before training, the dataset is preprocessed using 2D discrete wavelet decomposition and reconstruction to enhance the robustness of the network. The experimental results show that 92.4% of the mean average accuracy at IoU of 0.5 (mAP@0.5) and 67.7% of the mean average accuracy at IoU of 0.5 to 0.95 (mAP@0.5:0.95) are achieved using the BES-YOLO network, which is an increase of 5.3% and 4.4% compared to the YOLOv8n model. The research results can effectively improve the detection accuracy and efficiency of multi-scale targets in side-scan sonar images, which can be applied to AUVs and other underwater platforms to implement intelligent detection of undersea targets.

CORAL-Catamaran for Underwater Exploration: Development of a Multipurpose Unmanned Surface Vessel for Environmental Studies.

CORAL (Catamaran fOr UndeRwAter expLoration) is a compact, unmanned catamaran-type vehicle designed and developed to assist the scientific community in exploring marine areas such as inshore regions that are not easily accessible by traditional vessels. This vehicle can operate in different modalities: completely autonomous, semi-autonomous, or remotely assisted by the operator, thus accommodating various investigative scenarios. CORAL is characterized by compact dimensions, a very low draft and a total electric propulsion system. The vehicle is equipped with a single echo-sounder, a 450 kHz Side Scan Sonar, an Inertial Navigation System assisted by a GPS receiver and a pair of high-definition cameras for recording both above and below the water surface. Here, we present results from two investigations: the first conducted in the tourist harbour in Pozzuoli Gulf and the second in the Riomaggiore-Manarola marine area within the Cinque Terre territory (Italy). Both surveys yielded promising results regarding the potentiality of CORAL to collect fine-scale submarine elements such as anthropic objects, sedimentary features, and seagrass meadow spots. These capabilities characterize the CORAL system as a highly efficient investigation tool for depicting shallow bedforms, reconstructing coastal dynamics and erosion processes and monitoring the evolution of biological habitats.

DS-SIAUG: A Self-Training Approach Using a Disrupted Student Model for Enhanced Side-Scan Sonar Image Augmentation.

Side-scan sonar is a principal technique for subsea target detection, where the quantity of sonar images of seabed targets significantly influences the accuracy of intelligent target recognition. To expand the number of representative side-scan sonar target image samples, a novel augmentation method employing self-training with a Disrupted Student model is designed (DS-SIAUG). The process begins by inputting a dataset of side-scan sonar target images, followed by augmenting the samples through an adversarial network consisting of the DDPM (Denoising Diffusion Probabilistic Model) and the YOLO (You Only Look Once) detection model. Subsequently, the Disrupted Student model is used to filter out representative target images. These selected images are then reused as a new dataset to repeat the adversarial filtering process. Experimental results indicate that using the Disrupted Student model for selection achieves a target recognition accuracy comparable to manual selection, improving the accuracy of intelligent target recognition by approximately 5% over direct adversarial network augmentation.

Wide-Area Debris Field and Seabed Characterization of a Deep Ocean Dump Site Surveyed by Autonomous Underwater Vehicles.

Disposal of industrial and hazardous waste in the ocean was a pervasive global practice in the 20th century. Uncertainty in the quantity, location, and contents of dumped materials underscores ongoing risks to marine ecosystems and human health. This study presents an analysis of a wide-area side-scan sonar survey conducted with autonomous underwater vehicles (AUVs) at a dump site in the San Pedro Basin, California. Previous camera surveys located 60 barrels and other debris. Sediment analysis in the region showed varying concentrations of the insecticidal chemical dichlorodiphenyltrichloroethane (DDT), of which an estimated 350-700 t were discarded in the San Pedro Basin between 1947 and 1961. A lack of primary historical documents specifying DDT acid waste disposal methods has contributed to the ambiguity surrounding whether dumping occurred via bulk discharge or containerized units. Barrels and debris observed during previous surveys were used for ground truth classification algorithms based on size and acoustic intensity characteristics. Image and signal processing techniques identified over 74,000 debris targets within the survey region. Statistical, spectral, and machine learning methods characterize seabed variability and classify bottom-type. These analytical techniques combined with AUV capabilities provide a framework for efficient mapping and characterization of uncharted deep-water disposal sites.

A High-Efficiency Side-Scan Sonar Simulator for High-Speed Seabed Mapping.

Side scan sonar (SSS) is a multi-purpose ocean sensing technology, but due to the complex engineering and variable underwater environment, its research process often faces many uncertain obstacles. A sonar simulator can provide reasonable research conditions for guiding development and fault diagnosis, by simulating the underwater acoustic propagation and sonar principle to restore the actual experimental scenarios. However, the current open-source sonar simulators gradually lag behind mainstream sonar technology; therefore, they cannot be of sufficient assistance, especially due to their low computational efficiency and unsuitable high-speed mapping simulation. This paper presents a sonar simulator based on a two-level network architecture, which has a flexible task scheduling system and extensible data interaction organization. The echo signal fitting algorithm proposes a polyline path model to accurately capture the propagation delay of the backscattered signal under high-speed motion deviation. The large-scale virtual seabed is the operational nemesis of the conventional sonar simulators; therefore, a modeling simplification algorithm based on a new energy function is developed to optimize the simulator efficiency. This paper arranges several seabed models to test the above simulation algorithms, and finally compares the actual experiment results to prove the application value of this sonar simulator.

Optimization Method for Wide Beam Sonar Transmit Beamforming.

Imaging and mapping sonars such as forward-looking sonars (FLS) and side-scan sonars (SSS) are sensors frequently used onboard autonomous underwater vehicles. To acquire information from around the vehicle, it is desirable for these sonar systems to insonify a large area; thus, the sonar transmit beampattern should have a wide field of view. In this work, we study the problem of the optimization of wide transmission beampatterns. We consider the conventional phased-array beampattern design problem where all array elements transmit an identical waveform. The complex weight vector is adjusted to create the desired beampattern shape. In our experiments, we consider wide transmission beampatterns (≥20∘) with uniform output power. In this paper, we introduce a new iterative-convex optimization method for narrowband linear phased arrays and compare it to existing approaches for convex and concave-convex optimization. In the iterative-convex method, the phase of the weight parameters is allowed to be complex as in disciplined convex-concave programming (DCCP). Comparing the iterative-convex optimization method and DCCP to the standard convex optimization, we see that the former methods archive optimized beampatterns closer to the desired beampatterns. Furthermore, for the same number of iterations, the proposed iterative-convex method achieves optimized beampatterns, which are closer to the desired beampattern than the beampatterns achieved by optimization with DCCP.

Towards Characterizing and Developing Formation and Migration Cues in Seafloor Sand Waves on Topology, Morphology, Evolution from High-Resolution Mapping via Side-Scan Sonar in Autonomous Underwater Vehicles.

Sand waves constitute ubiquitous geomorphology distribution in the ocean. In this paper, we quantitatively investigate the sand wave variation of topology, morphology, and evolution from the high-resolution mapping of a side scan sonar (SSS) in an Autonomous Underwater Vehicle (AUV), in favor of online sequential Extreme Learning Machine (OS-ELM). We utilize echo intensity directly derived from SSS to help accelerate detection and localization, denote a collection of Gaussian-type morphological templates, with one integrated matching criterion for similarity assessment, discuss the envelope demodulation, zero-crossing rate (ZCR), cross-correlation statistically, and estimate the specific morphological parameters. It is demonstrated that the sand wave detection rate could reach up to 95.61% averagely, comparable to deep learning such as MobileNet, but at a much higher speed, with the average test time of 0.0018 s, which is particularly superior for sand waves at smaller scales. The calculation of morphological parameters primarily infer a wave length range and composition ratio in all types of sand waves, implying the possible dominant direction of hydrodynamics. The proposed scheme permits to delicately and adaptively explore the submarine geomorphology of sand waves with online computation strategies and symmetrically integrate evidence of its spatio-temporal responses during formation and migration.

[Remotely operated underwater vehicles used in water rescue]. / Unterwasserdrohnen in der Wasserrettung.

BACKGROUND: The search for and rescue of missing persons is always coupled with a high demand on human resources. In cases of suspected drowning it is often necessary to search huge expanses of water. For depths of more than 3-5 m the search and rescue procedure needs to be performed by specialist rescue divers. Due to saturation with nitrogen caused by the higher ambient pressure during the dive, the operating time for each rescue diver is limited. In addition, each dive is linked with an increased risk. OBJECTIVE: Miniaturization of sensors, higher performance of embedded systems and high energy density of modern accumulators offer the chance to operate with unmanned flying and diving vehicles for search and rescue even with moderate financial investment. MATERIAL AND METHODS: Based on funding by the Federation of the German Live Saving Association (DLRG) the DLRG national association of Bavaria procured three different models of remotely operated underwater vehicles (ROUV) and two different systems for underwater positioning. These systems will be offered to local associations of the DLRG in Bavaria for intense testing in their waters based on a common implementation strategy to ensure comparability and reproducibility. RESULTS: Initial tests with different types of ROUV and underwater positioning have been performed in preparation of the survey. As a result, mini-ROUVs found in the lower consumer segment have been identified as insufficient as they are not able to carry the additional payload of the underwater positioning systems whilst maintaining controllability. In contrast, more complex drones are difficult to handle and require longer preparation times before they are ready for use. The ROUVs in the median range, preferably with a streamlined structure, have so far been found to be optimal; however, operating the vehicle without a positioning system is not recommended. CONCLUSION: In combination with an underwater positioning system, remotely operated underwater vehicles are identified as a reasonable supplement for rescue divers. Fast time to operation enables a preview of the operating area before starting the rescue operation and can therefore support the rescue diver team.

Speckle Noise Filtering in Side-Scan Sonar Images Based on the Tucker Tensor Decomposition.

Real signals are usually contaminated with various types of noise. This phenomenon has a negative impact on the operation of systems that rely on signals processing. In this paper, we propose a tensor-based method for speckle noise reduction in the side-scan sonar images. The method is based on the Tucker decomposition with automatically determined ranks of factoring tensors. As verified experimentally, the proposed method shows very good results, outperforming other types of speckle-noise filters.

RT-Seg: A Real-Time Semantic Segmentation Network for Side-Scan Sonar Images.

Real-time processing of high-resolution sonar images is of great significance for the autonomy and intelligence of autonomous underwater vehicle (AUV) in complex marine environments. In this paper, we propose a real-time semantic segmentation network termed RT-Seg for Side-Scan Sonar (SSS) images. The proposed architecture is based on a novel encoder-decoder structure, in which the encoder blocks utilized Depth-Wise Separable Convolution and a 2-way branch for improving performance, and a corresponding decoder network is implemented to restore the details of the targets, followed by a pixel-wise classification layer. Moreover, we use patch-wise strategy for splitting the high-resolution image into local patches and applying them to network training. The well-trained model is used for testing high-resolution SSS images produced by sonar sensor in an onboard Graphic Processing Unit (GPU). The experimental results show that RT-Seg can greatly reduce the number of parameters and floating point operations compared to other networks. It runs at 25.67 frames per second on an NVIDIA Jetson AGX Xavier on 500*500 inputs with excellent segmentation result. Further insights on the speed and accuracy trade-off are discussed in this paper.

ECNet: Efficient Convolutional Networks for Side Scan Sonar Image Segmentation.

This paper presents a novel and practical convolutional neural network architecture to implement semantic segmentation for side scan sonar (SSS) image. As a widely used sensor for marine survey, SSS provides higher-resolution images of the seafloor and underwater target. However, for a large number of background pixels in SSS image, the imbalance classification remains an issue. What is more, the SSS images contain undesirable speckle noise and intensity inhomogeneity. We define and detail a network and training strategy that tackle these three important issues for SSS images segmentation. Our proposed method performs image-to-image prediction by leveraging fully convolutional neural networks and deeply-supervised nets. The architecture consists of an encoder network to capture context, a corresponding decoder network to restore full input-size resolution feature maps from low-resolution ones for pixel-wise classification and a single stream deep neural network with multiple side-outputs to optimize edge segmentation. We performed prediction time of our network on our dataset, implemented on a NVIDIA Jetson AGX Xavier, and compared it to other similar semantic segmentation networks. The experimental results show that the presented method for SSS image segmentation brings obvious advantages, and is applicable for real-time processing tasks.

Development of a 2 MHz Sonar Sensor for Inspection of Bridge Substructures.

Hydraulic factors account for a large part of the causes of bridge collapse. Due to the nature of the underwater environment, quick and accurate inspection is required when damage occurs. In this study, we developed a 2 MHz side scan sonar sensor module and effective operation technique by improving the limitations of existing sonar. Through field tests, we analyzed the correlation of factors affecting the resolution of the sonar data such as the angle of survey, the distance from the underwater structure and the water depth. The effect of the distance and the water depth and the structure on the survey angle was 66~82%. We also derived the relationship between these factors as a regression model for effective operating techniques. It is considered that application of the developed 2 MHz side scan sonar and its operation method could contribute to prevention of bridge collapses and disasters by quickly and accurately checking the damage of bridge substructures due to hydraulic factors.

Method for Augmenting Side-Scan Sonar Seafloor Sediment Image Dataset Based on BCEL1-CBAM-INGAN.

In this paper, a method for augmenting samples of side-scan sonar seafloor sediment images based on CBAM-BCEL1-INGAN is proposed, aiming to address the difficulties in acquiring and labeling datasets, as well as the insufficient diversity and quantity of data samples. Firstly, a Convolutional Block Attention Module (CBAM) is integrated into the residual blocks of the INGAN generator to enhance the learning of specific attributes and improve the quality of the generated images. Secondly, a BCEL1 loss function (combining binary cross-entropy and L1 loss functions) is introduced into the discriminator, enabling it to focus on both global image consistency and finer distinctions for better generation results. Finally, augmented samples are input into an AlexNet classifier to verify their authenticity. Experimental results demonstrate the excellent performance of the method in generating images of coarse sand, gravel, and bedrock, as evidenced by significant improvements in the Frechet Inception Distance (FID) and Inception Score (IS). The introduction of the CBAM and BCEL1 loss function notably enhances the quality and details of the generated images. Moreover, classification experiments using the AlexNet classifier show an increase in the recognition rate from 90.5% using only INGAN-generated images of bedrock to 97.3% using images augmented using our method, marking a 6.8% improvement. Additionally, the classification accuracy of bedrock-type matrices is improved by 5.2% when images enhanced using the method presented in this paper are added to the training set, which is 2.7% higher than that of the simple method amplification. This validates the effectiveness of our method in the task of generating seafloor sediment images, partially alleviating the scarcity of side-scan sonar seafloor sediment image data.

Side-scan sonar imaging data of underwater vehicles for mine detection.

Unmanned vehicles have become increasingly popular in the underwater domain in the last decade, as they provide better operation reliability by minimizing human involvement in most tasks. Perception of the environment is crucial for safety and other tasks, such as guidance and trajectory control, mainly when operating underwater. Mine detection is one of the riskiest operations since it involves systems that can easily damage vehicles and endanger human lives if manned. Automating mine detection from side-scan sonar images enhances safety while reducing false negatives. The collected dataset contains 1170 real sonar images taken between 2010 and 2021 using a Teledyne Marine Gavia Autonomous Underwater Vehicle (AUV), which includes enough information to classify its content objects as NOn-Mine-like BOttom Objects (NOMBO) and MIne-Like COntacts (MILCO). The dataset is annotated and can be quickly deployed for object detection, classification, or image segmentation tasks. Collecting a dataset of this type requires a significant amount of time and cost, which increases its rarity and relevance to research and industrial development.

Side-Scan Sonar Image Segmentation Based on Multi-Channel CNN for AUV Navigation.

The AUV (Autonomous Underwater Vehicle) navigation process relies on the interaction of a variety of sensors. The side-scan sonar can collect underwater images and obtain semantic underwater environment information after processing, which will help improve the ability of AUV autonomous navigation. However, there is no practical method to utilize the semantic information of side scan sonar image. A new convolutional neural network model is proposed to solve this problem in this paper. The model is a standard codec structure, which extracts multi-channel features from the input image and then fuses them to reduce parameters and strengthen the weight of feature channels. Then, a larger convolution kernel is used to extract the features of large-scale sonar images more effectively. Finally, a parallel compensation link with a small-scale convolution kernel is added and spliced with features extracted from a large convolution kernel in the decoding part to obtain features of different scales. We use this model to conduct experiments on self-collected sonar data sets, which were uploaded on github. The experimental results show that ACC and MIoU reach 0.87 and 0.71, better than other classical small-order semantic segmentation networks. Furthermore, the 347.52 g FOLP and the number of parameters around 13 m also ensure the computing speed and portability of the network. The result can extract the semantic information of the side-scan sonar image and assist with AUV autonomous navigation and mapping.

Morpho-bathymetric acoustic surveys as a tool for mapping traces of anthropogenic activities on the seafloor: The case study of the Taranto area, southern Italy.

In this study, an investigation procedure for mapping the traces of anthropogenic activities on the seafloor is proposed. Analyses are based on the interpretation of acoustic data (Multibeam Echosounder and Side Scan Sonar) acquired in the Taranto coastal area. Specific GIS tools supported the acoustic data analysis, interpretation, and mapping. These analyses highlighted that the seafloor of both coastal basins included in the investigated area is affected by a high distribution of traces related to different anthropogenic activities such as dredging, shipping, and mussel farming activities. Such kind of traces resulted efficiently detectable from morpho-bathymetric acoustic data. In particular, groove traces resulted highly distributed in both basins, while sunken mussel farm facilities are widely distributed in the Mar Grande basin. The results highlight as acoustic surveys represent a useful tool for orienting effective coastal management actions. This study points out how geophysical surveys support the geo-environmental characterization of highly urbanized coastal sectors.

Offshore benthic habitat mapping based on object-based image analysis and geomorphometric approach. A case study from the Slupsk Bank, Southern Baltic Sea.

Benthic habitat mapping is a rapidly growing field of underwater remote sensing studies. This study provides the first insight for high-resolution hydroacoustic surveys in the Slupsk Bank Natura 2000 site, one of the most valuable sites in the Polish Exclusive Zone of the Southern Baltic. This study developed a quick and transparent, automatic classification workflow based on multibeam echosounder and side-scan sonar surveys to classify benthic habitats in eight study sites within the Slupsk Bank. Different predictor variables, four supervised classifiers, and the generalisation approach, improving the accuracy of the developed model were evaluated. The results suggested a very high significance for the classification performance of specific geomorphometric features that were not used in benthic habitat mapping before. These include, e.g., Fuzzy Landform Element Classification, Multiresolution Index of the Valley Bottom Flatness, and Multiresolution Index of the Ridge Top Flatness. Comparison of classification results with manual maps demonstrated that Random Forest had the highest performance of four tested supervised classifiers. Because the current needs include benthic habitat mapping for the whole area of the Polish Exclusive Economic Zone, the key findings of this study may be further applied to extensive areas in the Polish waters and other vast areas worldwide.

Experimental bottom trawling finds resilience in large-bodied infauna but vulnerability for epifauna and juveniles in the Frisian Front.

In this study, we analysed the benthic effects of two in situ fisheries disturbance experiments using a combination of side-scan sonar, high definition underwater video, sediment profile imagery, and box core sampling techniques after conventional beam trawling and box core sampling after electric pulse trawling in a southern North Sea habitat. Acoustic and optical methods visualised the morphological changes induced by experimental beam trawling, showing the flattening and homogenisation of surface sediments. Video transects found a 94% decrease in epibenthos in beam trawled sediments compared to an untrawled control site and a 74% decrease in untrawled sediments of the same transect. Box core samples taken 5.5 h, 29 h and 75 h after trawling detected a downward trend in infaunal densities and species richness that continued after the initial impact with small-bodied and juvenile taxa being especially prone to depletion. Data from shallow sediment samples showed trawl resilience in large mud shrimps and evidence of their upward movement towards the sediment surface after disturbance. Both trawl gears induced significant changes to infaunal communities, with no differential effect between the two gears. Our results suggest that in the Frisian Front, trawling may favour the survival of deep burrowers while removing surficial macrofauna.

Identification, recovery, and impact of ghost fishing gear in the Mullica River-Great Bay Estuary (New Jersey, USA): Stakeholder-driven restoration for smaller-scale systems.

The impact of ghost fishing in large coastal ecosystems has generated considerable interest. In smaller, understudied systems with fewer stakeholders, derelict fishing gear (DFGs) may have impacts similar to these larger systems at the same relative scale. Four years of side scan sonar surveys in the Mullica River-Great Bay Estuary (New Jersey, USA) supported the recovery of 1776 DFGs off-season by commercial partners. Locations with high densities of recovered DFGs (>200 DFGs/km2) occupied intersections of recreational vessel traffic and commercial crabbing activity. Condition and depth-in-sediment of recovered DFGs was used to evaluate true bycatch (terrapins, whelks, blue crabs) versus species utilizing degraded gear as habitat (juvenile tautog, oyster toadfish). Critically, gear recovered in-season with low cost sonars (an additional 225 DFGs) prevented the accumulation of new DFGs which likely generate the highest percentages of bycatch. Removal of DFGs in this system led to significant ecological (reduced bycatch), economic (>$61,000 in direct pay, reused gear), and anticipated future benefits (increased harvest).

Detecting submerged bodies: controlled research using side-scan sonar to detect submerged proxy cadavers.

Forensic investigators routinely deploy side-scan sonar for submerged body searches. This study adds to the limited body of literature by undertaking a controlled project to understand how variables affect detection of submerged bodies using side-scan sonar. Research consisted of two phases using small and medium-sized pig (Sus scrofa) carcasses as proxies for human bodies to investigate the effects of terrain, body size, frequency, swath width, and state of decomposition. Results demonstrated that a clear, flat, sandy pond floor terrain was optimal for detection of the target as irregular terrain and/or vegetation are major limitations that can obscure the target. A higher frequency towfish was preferred for small bodies, and a 20 m swath width allowed greater visibility and easier maneuverability of the boat in this environment. Also, the medium-sized carcasses were discernable throughout the 81-day study period, indicating that it is possible to detect bodies undergoing decomposition with side-scan sonar.