Deep Learning-based Enhanced Adaptive Despeckling for Ultrasound Images

- Considering the inadequacies observed in traditional medical ultrasound image de-speckling techniques, this research presents an innovative solution: a feedforward convolutional neural network (CNN) model coupled with an adaptive multi-exposure fusion framework. The study is initiated by curating a specialized ultrasound image training dataset. It then proposes a multi-exposure fusion framework with adaptive enhancement factors to improve image quality for more efficient feature extraction. The proposed method trains a speckle model through the neural network and achieves the extraction of a de-speckled image. Experimental results unequivocally demonstrate the method's unparalleled effectiveness in speckle noise reduction within medical ultrasound images while concurrently preserving intricate image details, thus exemplifying its potential in clinical applications.


I. INTRODUCTION
Ultrasound imaging [1] uses the interaction of sound waves with living tissue to generate images of tissue by receiving and processing echoes emitted from an ultrasound probe.Ultrasonic diagnostic technology is widely used in medical clinical diagnosis.Medical ultrasonic imaging has the advantages of being non-invasive, low cost, etc., and its imaging is fast, simple and portable.However, speckle noise is an unavoidable property of ultrasound images.The speckle noise inevitably generated due to the difference in the travel path of coherent sound waves will reduce the quality of medical ultrasound images to varying degrees, easily lead to a decrease in contrast and resolution, and affect the results of clinical diagnosis.Speckle noise interferes with the details of ultrasound images and increases the difficulty of quantitative measurement and diagnosis of images [2].Suppressing speckle noise is a key processing step in feature extraction, analysis, and recognition of medical images.The analysis of medical ultrasound images has an important impact on clinical diagnosis and treatment.Speckle suppression of ultrasound images is a prerequisite for improving image quality and diagnostic accuracy [3].Early traditional image de-speckling methods are divided into three categories [4]: spatial domains, anisotropic diffusion filtering, and transform domains.These techniques can eliminate speckles from noisy photos, but they cannot effectively maintain the crisp details of the original image.Such as Lee, Kuan filter [5], these filters can only reduce noise under the loss of certain edge-preserving information.Block-Matching and 3D filtering (BM3D) method [6] can be considered as one of the current better methods, but this method has high computational complexity and ignores edge information [7].Literature [8,9] proposed methods such as the partial differential equation-based anisotropic diffusion filtering method to avoid blur.The localization problem of the original image and linear diffusion linear filtering, but this type of method involves more iterations and is computationally complex.The nonparametric statistical model speckle removal method (NPSM) is based on wavelet coefficients utilized in the literature [10] to filter the speckle noise in the image by establishing a statistical model.Bivariate Dual-Tree Complex Wavelet Transform (BI-DTCWT) based on Dual-Tree complex wavelet transform has been applied to speckle removal of medical ultrasound images [11].However, the edge preservation ability is still limited.Although these methods have a certain ability to remove speckles, they involve complex optimization problems and manual parameter selection problems [12].In recent years, the explosive development of deep learning has brought new ideas to the medical field, and many scholars have successfully applied deep learning to image denoising.Deep learning is a construct of methods that can learn from example inputs and represent data-driven predictions or classification outputs.Successful methods in this category include Cascade of Shrinkage Fields (CSF) [13], Trainable Nonlinear Reaction-Diffusion (TNRD) [14], using convolutional encoderdecoder Network image restoration and othermethods [15], and so on.However, this type of method.Essentially limited to learning image prior models.Zhanget al. [16] proposed a discriminative learning modelthat uses residual learning to separate Gaussian noise from noisy images to achieve denoising effects.However, this method is aimed at Gaussian additive noise, and its ability to suppress ultrasound image noise is limited.Inspired by this, this paper makes a training set of ultrasound images, extracts features by introducing an improved adaptive multi-exposure fusion framework, constructs a deep learning model for removing multiplicative speckle noise in medical ultrasound images, and proposes a method for eliminating speckles in ultrasound images.The proposed method is compared with traditional medical ultrasound image speckle removal techniques.Experimental results show that our method can produce good speckle removal performance and preserve more image details than existing methods.

A. ULTRASOUND IMAGE NOISE MODEL
Speckles in medical ultrasound images appear granular and have both multiplicative and additive noise [17].The speckle model of medical ultrasound images can be expressed as Equation 1.

B. RESIDUAL NETWORK
Deep learning models are a class of ways to learn feature hierarchies by building high-level features from low-level features.This style of learning can be trained using supervised or unsupervised methods.The basic building block of the deep residual network [18] is shown in Figure 1: If the input layer of a network is x, the expected output result is ().It is more difficult to directly use the convolutional layer to fit a potential identity map () = .However, if the network structure is designed as () =  () + , the complex problem can be transformed into learning a residual function  () = () − , only need to satisfy  ( ) = 0, then an identity map is formed.Therefore, this network is easier to learn than directly using convolutional layerfitting.The training process can be speed up and the capacity to remove speckles improved by combining residual learning and batch normalization.

C. ADAPTIVE MULTI-EXPOSURE FUSION FRAMEWORK
Bio-inspired multi-exposure fusion framework for low-light image enhancement [19].The enhanced image is defined as Equation 4.
The enhancement problem can be divided into three parts: the determination of the multi-exposure evaluator (), the multiexposure generator (), and the multi-exposure sampler ().
Where c is the index of the color channel.R is the enhanced result.P is Enter an image.(•) is the definition formula of the BTF model; k is the entropy that calculates the optimal exposure ratio.W is the weight matrix, which is defined as Equation 6.
Among them,  is the scene light map, and  is a parameter controlling the degree of enhancement.This paper proposes according to the global standard deviation and local standard deviation [20].Among them,  ×  represents the image size, and  ×  represents the local window size.(, ) represents the pixel value of row  and column .Among them, the local and global standard deviations are used to control the enhancement degree, and the mean value is a constant to balance the overall enhancement degree.The new weight matrix ′ is defined as in Equation 7.  ′ =  ′ (7) When ′ = 0, the obtained  = , that is, no enhancement is performed.When u′ = 1, both low-and high-exposure pixels are enhanced.When ′ > 1, pixels may be saturated, resultingin a loss of  details; the adaptive enhancement factor is related to the local mean standard deviation.The larger the local standard deviation, the higher the degree of belonging to the high frequency, and the larger the value of u'; otherwise, the smaller the value of u'.

D. DE-SPECKLING MODEL ARCHITECTURE
According to the principle of residual network, this paper builds a deep convolutional network system to achieve speckle removal in medical ultrasound images.It combines an Adaptive Fusion Framework with Deep Learning to Improve Model Learning.And combine batch normalization and residual learning to improve the model's learning accuracy and training speed.According to the residual learning strategy, it is assumed that () is a specklecontaining medical ultrasound image, () is a speckle noise image, and i is a speckle-removed image.Usingresidual learning, () = () −  finds the speckle noise output of the optimal identity map.According to the literature [21], the model architecture designed in this paper has 15 layers of depth.The definition of the loss function in this paper is shown in Equation 8.
where  denotes the trainable parameters, {I x ,S x } N are noises Clean training image pair,  represents the Frobenius norm. Figure 3 is a network structure diagram.First, the daylighting device obtains the optimal exposure ratio  according to the information entropy of the input image itself; the generator synthesizes the exposure image according to the model and the exposure ratio; the evaluator adaptively assigns weights; the combiner  generates the fused image according to Equation 4. Following the principle in [22], the size of the convolutional filter is set to 3×3.Then, in the first layer of the neural network, 64 filters of size 3×3×1 are used to generate 64 feature maps from the speckle noise image, where 3×3 is the height and width of the convolution on the input image, 1 is the number of image channels.This layer ends with a ReLu activation function.Then, layers 2-14 (hidden layers) have 64 filters with a size of 3×3×64 to generate 64 feature maps again, combined with batch normalization, and then generate the ReLu activation function.Finally, layer 15 is a filter with a size of 3 × 3 × 64 convolutional layers to reconstruct the output.The speckle-removed image is obtained using the ultrasonic speckle noise image and subtracting the gradually identified residual term model in the model using the combination of batch standard and residual learning.

DATA
To test the method's validity in this paper, many experiments are carried out on simulated speckle ultrasound images and images containing real speckle noise.The experiment is simulated in the MATLAB R2016a programming environment.Simultaneously use the MatConvNet package [23] to train the data.Mat-

B. EXPERIMENTAL COMPARISON AND ANALYSIS
To achieve objective evaluation, this paper uses the following three evaluation indicators: peak signal-to-noise ratio (Peak Signal to Noise Ratio, PSNR), edge preservation (β), equivalent number of looks (Equivalent Number of Looks, ENL) [14] as Objectively evaluate the index to test the speckle removal effect of the method.The methods for experimental comparison are BI-DTCWT [11], NPSM [10], NL_means [15], BM3D [6], Local_entropy_qsp [16], DnCNN [16] image speckle removal methods.

C. SIMULATED SPECKLE ULTRASOUND IMAGE EXPERIMENT
The literature [23] pointed out that the speckle noise simulation program can better simulate the actual speckle noise medical ultrasound image.In this paper, many sample image experiments are carried out, and the training models with noise standard deviations of 0.5-0.9 are trained, respectively.Figure 4 and Figure 5 are two images of speckle removal effects on ultrasound images of the liver with simulated speckles with a standard deviation of 0.7.Figures 4(a 4(h) and Figure 5(f), Figure 5(h) have achieved better results, compared with the method in this paper, the loss of details is more.To objectively evaluate the noise suppression performance of different methods, Tables 1 to 4 objectively evaluate peak signal-to-noise ratio PSNR and edge preservation degree β of liver images 1 and 2 with simulated speckle noise standard deviation of 0.5-0.9Index value.As shown in Table 1-Table 4, the method in this paper can obtain a larger peak signal-to-noise ratio and edge preservation.Compared with other methods, the PSNR index value has increased by 0.5-6dB, and the β index value has increased by 0.1-0.4dB.The objective evaluation results are consistent with the subjective visual effects, and the method in this paper has a better anti-spot effect.

D. REAL ULTRASOUND IMAGE EXPERIMENT
Medical ultrasound images are generally affected by speckle noise.This group uses fetal ultrasound images with a size of 210×210 pixels as the experimental object, and the speckle removal effect of the compared methods is shown in Figure 6. Figure 6(c)-Figure 6(i) are the speckle removal effect diagrams of BI-DTCWT, NPSM, NL_means, BM3D, Localentrop_qsp, DnCNN, and the method in this paper, respectively.From Figure 6(c),Figure 6(d), and Figure 6(g), it can be seen that the results still contain obvious speckle noise.Although Figures 6(f) and 6(h) can remove speckle noise to a certain extent, the images are blurred to varying degrees, and some are lost in detail.In contrast, the method in this papercan obtain a better smoothing effect in terms of subjective vision, and the impact of suppressing speckle noise is better.This article adopts.The equivalent visual number is used as the evaluation index of real ultrasound image speckle removal.The white box is the marked homogeneous region.[18] Random 25 images of.For each image in the experiment, a homogenous area is marked, and the average ENL value for each approach is counted.Table 6 demonstrates that the approach used in this paper can produce a higher equivalent visual index value, which is consistent with the speckle removal effect observed subjectively.The method in this paper can effectively suppress speckle noise while maintaining more image details.

IV.Conclusion
In this paper, the proposed multi-exposure fusion framework of adaptive enhancement factor is combined with deep learning, and its feature structure is learned from the input image data and then applied to medical ultrasound images to achieve the effect of speckle removal.Combined with batch normalization and residual network, the training process accelerates the model's performance.Compared with the existing image-denoising algorithms, the proposed method can better preserve the details of medical ultrasound images, and the speckle removal effect is remarkable.At the same time, the signal-to-noise ratio and image quality can be improved.This method provides a new, effective idea for the speckle removal of medical ultrasound images and lays a good foundation for the subsequent speckle removal processing of medical ultrasound images.

Figure 1 :
Figure 1: The basic structure of the residual network

Figure 2 :
Figure 2: Network Structure Diagram ConvNet is an open-source MATLAB toolbox for convolutional neural network training data for computer vision and multimedia applications.Due to the lack of open-source training sample sets of medical ultrasound images, this paper selected the images with high imaging quality and ideal noise suppression collected by the Siemens Simens ultrasonic diagnostic system ACUSON SC2000 in the hospital's clinical ultrasound imaging department.Crop the collected images to a size of 180 × 180 pixels.Through data expansion methods such as rotation, inversion, translation, and cropping, the training sample size is set to 400 images.The test dataset is images not included in the training dataset.
) and 5(a) are images with less speckle noise.The original ultrasound images of the liver, Figure 4(b) and Figure 5(b) are the ultrasound images after the simulation plus speckle noise, Figure 4(c)-Figure 4(i), Figure 5(c)-Figure 5(i) respectively For BI-DTCWT, NPSM, NL_means, BM3D, Local_entropy_qsp, DnCNN and the speckle removal visual effect of the method in this paper.It can be seen that the method in this paper has achieved a better subjective speckle removal effect and maintained more details of the original image, which is closer to the original image.The images in Figure 4(c)-Figure 4(e), Figure 4(g) and Figure 5(c)-Figure 5(e), Figure 5(g) still contain obvious speckle noise.Although Figure 4(f), Figure

Figure 6 Figure 4
Figure 4 Comparison of simulated speckle liver ultrasound images 2 experiments

Table 1 :
PSNR results of simulated speckle liver ultrasound image 1 by different methods (dB)

Table 5
lists the comparison results of the ENL equivalent visual values after processing the real ultrasound images with related methods.The 50 ultrasound images containing real speckles in Table6are from 25 random images collected by the EPIQ5 color Doppler ultrasound diagnostic instrument in the Medical Ultrasound Department of our school hospital and provided by the Biomedical Engineering Department of the Sirindhorn International Institute of Technology

Table 5 :
ENL results of different methods for real speckle

Table 6 :
Comparison of ENL average values of different methods for 50 real speckle ultrasound images