Detection for Density Map of Oil Spills with Machine Learning Tools
In early September, Hurricane Ida caused the disruption of 90 percent of the region’s oil, and substantial oil spilled into the Gulf of Mexico. In mid October, around 144,000 gallons of oil spilled off Southern California, according to the official statistics. After the oil spills, anxiety helps nothing. We need to detect where the source of the oil slick is so that we could clean up the spills efficiently. Hence, it is crucial to get the density map of oil slicks. (Then with the knowledge of inversion methods, we could find out the source, but we will focus on the density map rather than inversion methods in this article.)
Introduction of Dataset
I found a great dataset from NASA (https://gulfoilspill.jpl.nasa.gov/cgi-bin/search.pl). It includes not only Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) but also Uninhabited Aerial Vehicles Synthetic Aperture Radar (UAVSAR), which is also popular in the research of oil spills. The reason why I choose UAVSAR is because True et al. proposed that regular SAR backscatter could be used to measure oil thickness, for backscatter contrast was claimed to be better for low-incidence angles [1]. Besides, oil usually appears as dark spots in SAR images because it has a dampening effect on the Bragg waves [2]. Thus, oil spills detection usually uses SAR images as inputs.
(If you want to see the application of AVIRIS images, you could read this.)
Three Steps of SAR Oil Spills Detection
Dark spots detection, feature extraction, and feature classification [3].
1. Dark spots detection: identification of dark spots
The following are two methods about dark spot detection:
- Thresholding Algorithms
To separate background pixels from those corresponding to the target objects.
There are numerous thresholding algorithms, so let me just name a few:
Niblack Thresholding: This method adapts the threshold on the basis of the local mean and standard deviation, in a fixed size window.
Multi-Otsu Thresholding: The algorithm exhaustively searches for the threshold that minimizes the intra-class variance.
Niblack is useful for text recognition for a single object and Multi-Otsu is better at separating different objects.
(Both Niblack and Otsu are the names of their creators)
- Semantic segmentation
Objects classified with the same pixel values are segmented with the same colormaps.
2. Feature extraction: extraction of features from the detected dark spots
To obtain the most relevant information from the original data and represent that information in a lower dimensionality space.
Criteria to choose features given by Lippman are: “Features should contain information required to distinguish between classes.”
From my viewpoint, I would say the second step should be contained in the third step, feature classification. The model would do the feature extraction automatically according to the model we build. Nonetheless, how the model extracts features is still a black box. Right here, some people may argue that some papers [4][5] provide some feature parameters, such as area, perimeter, mean border gradient, spectral texture and so on. Based on my understanding, these parameters are translated by researchers with the outputs they got. As a consequence, we could not say we want which features to be used in the model, but we can recognize what kind of factors may influence the outcome under observation.
The following are two ways that help us to know more about which part of the image has more probability to be features. (But it’s still hard for us to hold a comprehensive view of feature extraction, which is also the reason why this topic worths more research.)
- Visualize filters: The learned filters are simply weights. The dark squares indicate small weights. The light squares indicate large weights.
- Visualize feature maps: To understand what features of the input are detected or preserved in the feature maps.
3. Feature classification: classification of the dark spots into oil spills and look-alikes
How difficulty is the classification task depends on the variability of the oil spill and look-alike examples. Classifiers learn the patterns from examples (training) and in a later stage are called to take a decision (classification).
The classic machine learning algorithm is called Convolutional Neural Network (CNN). I state some common layers that are used in CNN:
And I’ve already used some types of CNN in my previous article. That is, autoencoder, stacked autoencoder, and variational autoencoder. However, CNNs have a limitation: It can only accept fixed sized inputs coming from the fully-connected layers.
Therefore, today I want to introduce another machine learning structure called Fully Convolutional Network (FCN). FCNs only have convolutional and pooling layers (no dense layers) which give them the ability to make predictions on arbitrary-sized inputs. Here is an example of FCN:
As we can see, FCN has a similar structure with autoencoder, so it would also have tying weights for each layer. Moreover, FCN would seems symmetry (in the layer sense). A convolutional layer would couple with a convolutional layer. A dropout layer would couple with a dropout layer. A maxpooling layer would couple with a upsampling layer (their functions are dual). One part of making it not fully symmetric is that there would be “a concatenation layer” (the plus “+” sign in the figure) right before we use upsampling layer.
At the final of this part, we can conclude these three steps with this flowchart.
Application: Density Map
The following are steps that I would use to attain the density map of oil spills.
- Image processing: run the image through the thresholding algorithm
- Create a mask: select the dark spot part and make it transparent
- Get the density map: paste it into the original image
- Get the density value: get the numerical value of each pixel
Before jumping into these steps, there are two points I want to remind you. Firstly, the algorithm needs the RGB of a color to assign it a new color, but we only know the RGB of white (255, 255, 255) and black (0, 0, 0). Second, we would set some parts to transparent in the second step, so we need to use RGBA (instead of RGB) where “A” means Alpha (opacity). If you set it to 0, it means “completely transparent”. If you set it to 255, it means “completely nontransparent”. Then we are ready, let’s go!
After we use K-means algorithm with K=3 (step 1), our image has the color of white, gray, and black.
The figure below is to apply the oil spill image to the figure above.
FYI, The orange background is to help us know which part is transparent.
The 1st image is the result of step 1. Step 2 goes from 2nd image to 5th image. (The reason why we goes from 4th image to 5th image is for the convenience of getting the density map, for we would use the grey scale to know its density at each point.) Step 3 is the last image. I don’t show the result of step 4 because it just gets the grey scale value of the last image.
As we can notice, there are some black noises on our mask. The way I smooth out the result of K-means is that I no longer set one color point (in RGB format) but a range of color points (in RGB format) to white or black. Then the new mask has a smoother result.
Generalization
In order to generalize the step3 from K=3 to K=5 or 10 or larger value.
(K means the number of color categories in the K-means algorithm.)
Take K=5 as an example.
The following is the algorithm that I figure out.
Conclusion
To detection the SAR images, the three steps of oil spills detection are introduced: dark spots detection, feature extraction and feature classification. In the first step, we could use thresholding algorithms or semantic segmentation. In the second step, we could observe how the model extracts features by visualizing filters and feature maps. In the last step, we introduce a machine learning model called Fully Convolutional Network.
For the density maps, we use K-means algorithm to detect dark spots. Later on, we generate a mask, and paste it into the original image. Then we get the density map. Finally, we get the generalization of any value of K for K-means algorithm.
Thanks for your reading! Hope this article may help you in some way.
Feel free to contact me if you have any question: victor850225@gmail.com
References
[1] Fingas, M., 2018. The challenges of remotely measuring oil slick thickness. Remote Sens. 10 (2), https://doi.org/10.3390/rs10020319.
[2] A. H. S. Solberg, G. Storvik, R. Solberg and E. Volden, “Automatic detection of oil spills in ERS SAR images,” in IEEE Transactions on Geoscience and Remote Sensing, vol. 37, no. 4, pp. 1916–1924, July 1999, doi: 10.1109/36.774704.
[3] Shu, Yuanming et al. “Dark-spot detection from SAR intensity imagery with spatial density thresholding for oil-spill monitoring.” Remote Sensing of Environment 114 (2010): 2026–2035.
[4] Singha, Suman & Bellerby, Tim & Trieschmann, Olaf. (2012). Detection and classification of oil spill and look-alike spots from SAR imagery using an Artificial Neural Network. International Geoscience and Remote Sensing Symposium (IGARSS). 5630–5633. 10.1109/IGARSS.2012.6352042.
[5] Topouzelis K. N. (2008). Oil Spill Detection by SAR Images: Dark Formation Detection, Feature Extraction and Classification Algorithms. Sensors (Basel, Switzerland), 8(10), 6642–6659. https://doi.org/10.3390/s8106642
