International Journal of Electrical and Computer Engineering (IJECE) Vol.
No.
February 2019, pp.
ISSN: 2088-8708.
DOI: 10.
11591/ijece.
A hybrid Pan-sharpening approach using maximum local extrema Prajakta Patil1.
Sheela Rani2.
Meenakshi Arya3 1,2Department of Computer Science and Engineering.
KLEF.
India 3Department of Information Technology.
Vidyalankar Institute of Technology.
India
Article Info
ABSTRACT
Article history:
Mixing or combining different elements for getting enhanced version, is practiced across various areas in real life.
Pan-sharpening is a similar technique used in the digital world.
a process to combine two images into a fused image that comprises more detailed information.
Images referred herein are Panchromatic (PAN) and Multispectral (MS) images.
This paper presents a pansharpening algorithm which integrates multispectral and panchromatic images to generate an improved multispectral image.
This technique merges the Discrete wavelet transform (WT) and Intensity-HueSaturation (IHS) through separate fusing criterion for choosing an approximate and detail sub-images.
Whereas the maximal local extrema are used for merging detail sub-images and finally merged high-resolution image is reconstructed through inverse transform of wavelet and IHS.
The proposed fusion approach enhances the superiority of the resultant fused image is demonstrated by quality measures like CORR.
RMSE.
PFE.
SSIM.
SNR and PSNR with the help of satellite Worldview-II images.
The proposed algorithm is correlated with the other fusion techniques through empirical outcomes proves the superiority of the final merged image in terms of resolutions than the others.
Received Feb 5, 2018 Revised Jul 30, 2018 Accepted Aug 17, 2018 Keywords:
Component substitution Hybrid approach Image fusion Multi-resolution analysis Pan-sharpening Copyright A 2019 Institute of Advanced Engineering and Science.
All rights reserved.
Corresponding Author:
Prajakta Patil.
Department of Computer Science and Engineering.
KLEF.
Green Fields,Vaddeswaram-522502.
Guntur Distric.
Andhra Pradesh.
India.
Email: praj1984ster@gmail.
INTRODUCTION
Most of remote sensing satellites are producing images with high spectral and low spatial or low spectral and high spatial resolutions i.
Multispectral and Panchromatic images respectively.
Sharp spectral resolution highlights different objects present in the image while main feature of accurate spatial resolution is to describe the outline, contents and structure of the image.
Neither a multispectral nor a panchromatic image is sufficient to best describe the earth images essential for many of the remote sensing applications.
High spatial and spectral images are suitable for observing the earth images as it involves information like feature, shape, structure and different objects.
Direct production of such images is inconvenient due to lack of bandwidth and onboard storage available at the satellite .
Hence, it is necessary to adopt the process of combining a high spatial and spectral resolution into one image is also called as Pansharpening .
This process generates a high-resolution MS image which greatly describes earth images.
Important factor to be considered is fused image should contain more interpretation capability, also addition of spatial information without distracting spectral information .
The widely used pansharpening techniques are sorted into two key categories: component substitution pansharpening technique (CS) and multiresolution analysis pansharpening technique (MRA).
Zang et.
Investigates a framework for development of image fusion techniques and is used to develop.
Journal homepage: http://iaescore.
com/journals/index.
php/IJECE A ISSN: 2088-8708 compare and analysis various fusion techniques .
Comprehensive study of various image fusion algorithm along with applications are available in the literature .
The earlier is based upon on the exchange of element with PAN image.
In this, spectral transformation of the MS image is worked as a component.
Intensity hue saturation (IHS) .
Gram Schmidt (GS) .
and principal component analysis .
are extensively used component substitution pansharpening techniques.
HIS
pan-sharpening technique works as following steps: first it converts given low resolution multispectral images into IHS model then the extracted intensity I is exchanged by high-resolution panchromatic image.
And finally, inverse transform helps to generate high-resolution multispectral image.
But the merged image generated using this technique always deficit due to spectral distortion even if contains high spatial resolution .
The latter approach is based on the partition of the images and then reconstruction of new image with spatial and spectral details.
The several categories of MRA pan-sharpening techniques are decimated wavelet transform (DWT) .
, undecimated wavelet transform (UDWT) .
Auy trousAy wavelet transform (ATWT) .
and Laplacian pyramid (LP) .
In Wavelet transform (WT) pan-sharpening technique.
WT is applied on both images and decomposed into two sub-images: approximation image which includes generalization of the source images and detail image comprises of detail description of the source images.
Then, the approximate and the detail sub-images are combined to generate a merged image.
The resultant image obtained through inverse WT.
This category of pansharpening technique preserves spectral information but with some spatial distortions like blurring .
Hybrid approach for image fusion involves integration of CS and MRA are investigated in .
In this paper, the proposed hybrid algorithm is described as, initially.
IHS transform is applied on intensity component of MS image.
SFIM and wavelet transform is applied on a result of the above step and PAN image to generate outcome with multiscale representation.
Spectral property of the resultant image is preserved using SFIM.
Information accommodated in approximate image is distinct from detail sub-image so those need to be processed separately.
Finally, the inverse wavelet transforms (IWT) as well as an inverse intensity hue saturation (IHS) transforms are implemented to get the resultant fusion image.
The superiority of the hybrid algorithm is proved through the empirical outcomes obtained with the help of Worldview-II images.
This paper is organized into different modules such as Module II defines the traditional image fusion algorithms.
Module i represents the proposed hybrid pan-sharpening algorithm.
The experimental results with analysis for images of Worldview-II dataset are showcased in module IV.
At last, conclusion is stated in module V.
RESEARCH METHOD
This section provides the concepts related to the proposed technique.
The proposed algorithm consists of a hybrid approach using image fusion techniques like IHS .
, wavelet transform (WT) .
, smoothing filter-based intensity modulation (SFIM) .
, and local extrema .
Image fusion using IHS IHS is one of the component substitution technique, where given an image, is transformed into IHS from RGB and I component is recovered by given PAN image and at the end resultant image is gained through inverse IHS transform as described by the given algorithm below.
Up sample the MS image to PAN image Transform MS image to IHS model from RGB model The intensity is considered as.
I = Oc M where Z represents band count, = 1/3 for RGB image and M denotes kth band of MS image.
The merged high-resolution image can be obtained through,F = M (P Oe I) where F is the kth band in the fused image.
Image fusion using SFIM A proportion of a MS and PAN image and itAos a smoothing filtered image used in SFIM helps to ( , ).
( , )
improve spatial resolution for spectral bands.
F =
, .
denotes image pixel co-ordinate, ( ) Mk represents each band of MS image.
P signifies PAN images.
P mean represents smoothed PAN image and F denotes the merged image.
The resultant fused image generated using SFIM improves spatial resolution whereas retains spectral information.
Int J Elec & Comp Eng.
Vol.
No.
February 2019 : 255 - 263
Int J Elec & Comp Eng
ISSN: 2088-8708
Image fusion using wavelet Transform (WT) Wavelet transform is more attractive technique of image fusion due to its multiscale characteristics which are best suited to accomplish different image resolutions.
These are classified into discrete and continuous categories .
, .
Here it considers a discrete approach.
Decompose MS and PAN image using DWT W = DWT [ MS .
, .
] = [MS .
MS .
MS .
MS ] W = DWT [PAN .
, .
] = [PAN .
PAN .
PAN .
PAN ] where W .
, .
and W .
, .
are the wavelet coefficients of MS .
, .
and PAN .
, .
at level 1.
Calculate fused wavelet coefficients W .
, .
= [F .
F .
F .
F ] at scale l with conventional substitution method .
Reconstruct fused image F .
, .
at scale 1 using inverse DWT.
F .
, .
= IDWT[W .
, .
] THE PROPOSED METHOD This research discusses about the hybrid image fusion algorithm using integration of CS and MRA algorithms as CS pansharpening technique encompasses the ability to improve spatial resolution while MRA pansharpening technique incorporates the ability to extract adequate spatial details through preserving spectral details.
The proposed algorithm uses separate fusion rule criterion for approximate and detail subimage.
The detail sub-image comprises of the exact description about the image, therefore, the maximum local extrema are selected to merge detail sub-images obtained from wavelet transform.
The proposed algorithm is described to:
Up sample the MS image to PAN image Transform MS image to IHS model from RGB mode The intensity I is defined as.
I = Oc M where Z represents band count, = 1/3 for RGB image and M denotes kth band of MS image.
Calculate new intensity value as,I where I represent intensity value.
P denotes the PAN image.
P mean represents smoothed PAN image, and I represents the calculated new intensity value.
Decompose I and PAN image using DWT
W = DWT [I
, .
] = [I W = DWT [PAN .
, .
] = [PAN .
PAN .
PAN .
PAN ] where W .
, .
and W .
, .
are the wavelet coefficients of I .
, .
and PAN .
, .
at level 1.
Calculate approximate fused wavelet coefficients W .
, .
= [F ] at scale l as.
F .
, .
= 0.
5 O (I
PAN ,
PAN ),
, .
> PAN .
, .
, .
= PAN .
, .
, .
< PAN .
, .
Calculate detail fused wavelet coefficients [F .
F .
F ] at scale l as Selection of detail coefficients are based on contrast between detail and approximation layer which is given by Slh Similarly Shl .
Shh .
Slh .
Shl and Shh Fused detail coefficients can be obtained by:
F .
, .
= PAN .
PAN ).
Slh Slh Slh .
, .
> Slh .
, .
= Slh .
, .
< Slh .
, .
, .
, .
Similarly apply the above fusion rule to get remaining detail layer coefficients i.
F .
, .
and F .
, .
A hybrid pan-sharpening approach using maximum local extrema (Prajakta Pati.
A ISSN: 2088-8708 Reconstruct fused image F .
, .
at scale 1 using inverse DWT.
F .
, .
= IDWT[F .
F .
F .
F ] The resultant merged image can be reconstructed through inverse IHS transform as.
F = M (F .
, .
Oe I), where F is the kth band in the fused image.
RESULTS AND ANALYSIS
In this paper, dataset generated by a high-resolution WorldView-II satellite is used to verify the effectivity of the hybrid algorithm.
This satellite provides multispectral and panchromatic images with 84-m and 0.
46-m resolution correspondingly.
The set of images used are available on the website http://w.
com/data/43234.
This dataset is created using Digital Globe and organized by Beijing key laboratory of digital media.
Beihang University .
The images of this dataset describe the area of city, coastal, bridge, etc.
Evaluation indicators This paper uses quality metrics like correlation coefficient, signal to noise ratio, peak signal to noise ratio, structural similarity, the root means squared error and percentage fit error for comparison of original and resultant images .
Table 1 shows in definition of the variable used in the quality metrics.
Table 1.
Definition of the Variable Used in the Quality Metrics Var
Def
Fused Resampled MS
Fmax
Maximum
gray value Standard
EFX
Covariance Oc Oc [F.
, .
Oe AA ] y Oc Oc [X.
, .
Oe AA ] Oc MyN [F.
, .
Oe AA .
, .
] Oc Higher value of PSNR indicates noise in the merged image.
The structural similarity (SSIM) .
, .
, .
is defined as:
( AA AA
(AA
SSIM is used to find similarity among MS and fused image.
More the SSIM value then the chance to get similar images is also more.
The root mean squared error (RMSE) .
, .
measures difference between two images F and MS.
The smaller difference specifies improved fusion result.
It is represented as:
RMSE(X.
F) =
Mean The definition of the peak signal to noise ratio (PSNR) .
, .
is represented a:
SSIM(X.
F) =
p, q Pixel [F.
, .
Oe AA ][X.
, .
Oe AA ] Oc PSNR(X.
F) = 10lg No.
The correlation coefficient (CC) .
, .
can be used to measure a presence of spectral information.
The resultant value of equation reveals the correlativity of fused and MS image and CC is defined as:
CC(X.
F) =
No.
Oc Oc [F.
, .
Oe AA .
, .
] Percentage fit error (PFE) .
calculates the proportion of norm of the gap among the MS image and merged image to the norm of the MS image.
The less value of PFE specifies improved fusion result.
It is computed as:
PFE(X.
F) =
( , )
( , )
( , )
O 100
Int J Elec & Comp Eng.
Vol.
No.
February 2019 : 255 - 263
Int J Elec & Comp Eng
ISSN: 2088-8708
Signal to noise ratio (SNR) .
is sensitivity measure of the image.
The higher value is most preferable as it indicates similarity of MS and fused image.
SNR(X.
F) = 10lg
[ ( , )]
[ ( , ) AA ( , )]
Experimental results This paper also considers execution of other image fusion techniques like IHS .
DWT .
, .
BT .
and quality measures like RMSE.
PFE.
MAE.
CORR.
SNR.
PSNR.
QI and SSIM for two different groups of images where each group contains 2 pairs of input images.
This experiment and implementation is conducted in MATLAB2011a on a Windows8 computer.
Experimental results shown in Tables 2-5 supports that the proposed method provides an improved result than the other techniques like IHS.
BT, and DWT.
The highlighted experimental results indicate the superiority of the technique in an evaluation.
It also indicates that the hybrid method improves the result with RMSE.
PFE.
MAE.
SNR.
PSNR, and SSIM for both sets of images but not able to improve the result for some images in terms of CORR present in Table 4.
We also observed that our proposed method generates a resultant image with high spectral and spatial resolution from human eyeAos visual perception as compared to others.
The value of RMSE.
PFE.
SNR.
PSNR, and SSIM specified in Tables 2-5 verifies that the proposed algorithm works best for the given set of images over the other pansharpening algorithms.
It is observed that DWT slight edge over the proposed algorithm for Worldview-II seaside images of set 1 on correlation parameters, shown in Table 4.
Table 2.
Quality Measure Analysis with the Worldview-II Urban Images Set 1
Data Shown in Figure 1
Measures RMSE
PFE
MAE
CORR
SNR
PSNR
SSIM
IHS
Methods DWT Proposed Table 3.
Quality Measure Analysis with the Worldview-II Urban Images Set 2
Data Shown in Figure 2
Measures RMSE
PFE
MAE
CORR
SNR
PSNR
SSIM
IHS
Methods DWT Proposed Table 4.
Quality Measure Analysis with the Worldview-II Seaside Images Set 1
Data Shown in Figure 3
Measures RMSE
PFE
MAE
CORR
SNR
PSNR
SSIM
IHS
Methods DWT Proposed A hybrid pan-sharpening approach using maximum local extrema (Prajakta Pati.
A ISSN: 2088-8708 Table 5.
Quality Measure Analysis with the Worldview-II Seaside Images Set 2
Data Shown in Figure 4
Measures RMSE
PFE
MAE
CORR
SNR
PSNR
SSIM
IHS
Methods DWT Proposed Figures 1-4 .
indicate the resampled low-resolution MS images of Worldview-II.
The corresponding high spatial resolution PAN images are displayed in Figures 1-4 .
The resultant fused image obtained by traditional fusion methods like BT.
IHS.
DWT and the proposed method are presented in Figures 1-4 .
, .
, .
Resultant fused image after applying proposed algorithm improves the clarity of the image with more accurate geographical details are observed in Figures 1-4 .
Figure 1.
Pan-sharpening outcomes about Worldview-II urban images set 1: .
Input MS image after resampling .
Input PAN image .
BT .
IHS .
DWT .
Proposed hybrid technique .
Figure 2.
Pan-sharpening outcomes about Worldview-II urban images set 2: .
Input MS image after resampling .
Input PAN image .
BT .
IHS .
DWT .
Proposed hybrid technique Int J Elec & Comp Eng.
Vol.
No.
February 2019 : 255 - 263
Int J Elec & Comp Eng
ISSN: 2088-8708
Figure 3.
Pan-sharpening outcomes about Worldview-II seaside images set 1: .
Input MS image after resampling .
Input PAN image .
BT .
IHS .
DWT .
Proposed hybrid technique .
Figure 4.
Pan-sharpening outcomes about Worldview-II seaside images set 2: .
Input MS image after resampling .
Input PAN image .
BT .
IHS .
DWT .
Proposed hybrid technique CONCLUSION The research work is focused on implementation of a hybrid image fusion algorithm which merges CS approach and MRA approach to overcome drawbacks of individual approach.
This hybrid approach is designed to enhance superiority of images.
Separate fusion rules are adopted in this hybrid approach for the approximate and the detail sub-images.
The research work also involves comparison of hybrid pansharpening technique with some traditional algorithm like IHS.
DWT.
BT.
Experiments are conducted on 4 sets of images from Worldview-II.
The Superiority of a resultant merged image is calculated in terms of RMSE.
PFE.
MAE.
CORR.
SNR.
PSNR, and SSIM as shown in Table 2-5.
Empirical outcomes demonstrate that the hybrid algorithm gives improved fusion result than the others.
The fused image obtained from the hybrid algorithm progresses best in resolution, so it can be used further for image processing applications for classification, feature extraction etc.
Current work can be altered in future through implementing different fusion rules for merging approximate and detail sub-images.
Also, this work can be extended by developing a different hybrid approach for satellite images generated from different satellites.
A hybrid pan-sharpening approach using maximum local extrema (Prajakta Pati.
A ISSN: 2088-8708
REFERENCES