Addressing Sample Inconsistency for Semisupervised Object Detection in Remote Sensing Images
Yuhao Wang, Lifan Yao, Gang Meng, Xinyue Zhang, Jiayun Song and Haopeng Zhang*
IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing (JSTARS), 2024
Abstract   BibTeX  

The emergence of semisupervised object detection (SSOD) techniques has greatly enhanced object detection performance. SSOD leverages a limited amount of labeled data along with a large quantity of unlabeled data. However, there exists a problem of sample inconsistency in remote sensing images, which manifests in two ways. First, remote sensing images are diverse and complex. Conventional random initialization methods for labeled data are insufficient for training teacher networks to generate high-quality pseudolabels. Finally, remote sensing images typically exhibit a long-tailed distribution, where some categories have a significant number of instances, while others have very few. This distribution poses significant challenges during model training. In this article, we propose the utilization of SSOD networks for remote sensing images characterized by a long-tailed distribution. To address the issue of sample inconsistency between labeled and unlabeled data, we employ a labeled data iterative selection strategy based on the active learning approach. We iteratively filter out high-value samples through the designed selection criteria. The selected samples are labeled and used as data for supervised training. This method filters out valuable labeled data, thereby improving the quality of pseudolabels. Inspired by transfer learning, we decouple the model training into the training of the backbone and the detector. We tackle the problem of sample inconsistency in long-tail distribution data by training the detector using balanced data across categories. Our approach exhibits an approximate 1% improvement over the current state-of-the-art models on both the DOTAv1.0 and DIOR datasets.

@ARTICLE{10463140,
  author={Wang, Yuhao and Yao, Lifan and Meng, Gang and Zhang, Xinye and Song, Jiayun and Zhang, Haopeng},
  journal={IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing},
  title={Addressing Sample Inconsistency for Semisupervised Object Detection in Remote Sensing Images},
  year={2024},
  volume={17},
  number={},
  pages={6933-6944},
  keywords={Training;Remote sensing;Object detection;Measurement;Detectors;Tail;Labeling;Active learning;long-tailed distribution;remote sensing;semisupervised object detection (SSOD)},
  doi={10.1109/JSTARS.2024.3374820}
}
            

Histopathology language-image representation learning for fine-grained digital pathology cross-modal retrieval
Dingyi Hu, Zhiguo Jiang, Jun Shi, Fengying Xie, Kun Wu, Kunming Tang, Ming Cao, Jianguo Huai and Yushan Zheng*
Medical Image Analysis, 2024
Abstract   BibTeX   Code

Large-scale digital whole slide image (WSI) datasets analysis have gained significant attention in computer-aided cancer diagnosis. Content-based histopathological image retrieval (CBHIR) is a technique that searches a large database for data samples matching input objects in both details and semantics, offering relevant diagnostic information to pathologists. However, the current methods are limited by the difficulty of gigapixels, the variable size of WSIs, and the dependence on manual annotations. In this work, we propose a novel histopathology language-image representation learning framework for fine-grained digital pathology cross-modal retrieval, which utilizes paired diagnosis reports to learn fine-grained semantics from the WSI. An anchor-based WSI encoder is built to extract hierarchical region features and a prompt-based text encoder is introduced to learn fine-grained semantics from the diagnosis reports. The proposed framework is trained with a multivariate cross-modal loss function to learn semantic information from the diagnosis report at both the instance level and region level. After training, it can perform four types of retrieval tasks based on the multi-modal database to support diagnostic requirements. We conducted experiments on an in-house dataset and a public dataset to evaluate the proposed method. Extensive experiments have demonstrated the effectiveness of the proposed method and its advantages to the present histopathology retrieval methods. The code is available at https://github.com/hudingyi/FGCR.

@article{HU2024103163,
title = {Histopathology language-image representation learning for fine-grained digital pathology cross-modal retrieval},
journal = {Medical Image Analysis},
volume = {95},
pages = {103163},
year = {2024},
issn = {1361-8415},
doi = {https://doi.org/10.1016/j.media.2024.103163},
url = {https://www.sciencedirect.com/science/article/pii/S1361841524000884},
author = {Dingyi Hu and Zhiguo Jiang and Jun Shi and Fengying Xie and Kun Wu and Kunming Tang and Ming Cao and Jianguo Huai and Yushan Zheng},
keywords = {CBHIR, Cross-modal, Diagnosis reports, Digital pathology},
abstract = {Large-scale digital whole slide image (WSI) datasets analysis have gained significant attention in computer-aided cancer diagnosis. Content-based histopathological image retrieval (CBHIR) is a technique that searches a large database for data samples matching input objects in both details and semantics, offering relevant diagnostic information to pathologists. However, the current methods are limited by the difficulty of gigapixels, the variable size of WSIs, and the dependence on manual annotations. In this work, we propose a novel histopathology language-image representation learning framework for fine-grained digital pathology cross-modal retrieval, which utilizes paired diagnosis reports to learn fine-grained semantics from the WSI. An anchor-based WSI encoder is built to extract hierarchical region features and a prompt-based text encoder is introduced to learn fine-grained semantics from the diagnosis reports. The proposed framework is trained with a multivariate cross-modal loss function to learn semantic information from the diagnosis report at both the instance level and region level. After training, it can perform four types of retrieval tasks based on the multi-modal database to support diagnostic requirements. We conducted experiments on an in-house dataset and a public dataset to evaluate the proposed method. Extensive experiments have demonstrated the effectiveness of the proposed method and its advantages to the present histopathology retrieval methods. The code is available at https://github.com/hudingyi/FGCR.}
}
            

High-Resolution Feature Generator for Small-Ship Detection in Optical Remote Sensing Images
Haopeng Zhang*, Sizhe Wen, Zhaoxiang Wei and Zhuoyi Chen
IEEE Transactions on Geoscience and Remote Sensing (TGRS), 2024
Abstract   BibTeX  

Ship detection in optical remote sensing (RS) images remains a persistent challenge in current research. While prevailing methods achieve satisfactory outcomes in detecting large ship objects within RS images, the identification of small-ship objects poses greater difficulty due to their limited pixel information. To address this challenge, the utilization of generative adversarial network-based (GAN-based) super-resolution (SR) techniques proves effective. Therefore, in this article, we present a high-resolution feature generator (HRFG) specifically tailored for small-ship detection. Different from previous GAN-based methods that rely on image-level SR or feature sharing between SR and detection, we design a new architecture that uses an additional network branch, that is, high-resolution feature extractor (HRFE), to extract real high-resolution (HR) feature as a feature-level supervisory signal. The intuition is that real HR features may guide the generator network to extract HR features from low-resolution (LR) images directly. Consequently, the feature for detection is extracted and enhanced at the same time so that a large amount of calculation brought by image-level SR is avoided. Additionally, we introduce a background degradation strategy within the HRFE to improve the performance of small object recognition. Extensive experiments on a self-assembled ship dataset and two other public datasets show the superiority of the proposed method in small-ship detection tasks.

@ARTICLE{10473137,
  author={Zhang, Haopeng and Wen, Sizhe and Wei, Zhaoxiang and Chen, Zhuoyi},
  journal={IEEE Transactions on Geoscience and Remote Sensing},
  title={High-Resolution Feature Generator for Small-Ship Detection in Optical Remote Sensing Images},
  year={2024},
  volume={62},
  number={},
  pages={1-11},
  keywords={Feature extraction;Marine vehicles;Detectors;Superresolution;Optical imaging;Remote sensing;Generators;Generative adversarial network (GAN);optical image;remote sensing (RS);ship detection},
  doi={10.1109/TGRS.2024.3377999}
}
            

Satellite Video Super-Resolution via Unidirectional Recurrent Network and Various Degradation Modeling
Xiaoyuan Wei, Haopeng Zhang* and Zhiguo Jiang
IEEE International Geoscience and Remote Sensing Symposium (IGARSS), 2024
Abstract   BibTeX  

Satellite video images contain temporal contextual information that is unavailable in single-frame images. Therefore, using a sequence of frames for super-resolution can significantly enhance the reconstruction effect. However, most existing satellite Video Super-Resolution (VSR) methods focus on improving the network’s presentation ability, overlooking the complex degradation processes present in real-world satellite videos which appear as a blind SR problem. In this paper, we propose an effective satellite VSR method based on a unidirectional recurrent network named URD-VSR. Simultaneously, a network independent of the SR structure is utilized to model the degradation process. Experiments on real satellite video datasets and integration with object detection demonstrate the effectiveness of the proposed method.

Coming Soon
            

A Closed-Loop Network for Single Infrared Remote Sensing Image Super-Resolution in Real World
Haopeng Zhang*, Cong Zhang, Fengying Xie and Zhiguo Jiang
Remote Sensing (RS), 2023
Abstract   BibTeX  

Single image super-resolution (SISR) is to reconstruct a high-resolution (HR) image from a corresponding low-resolution (LR) input. It is an effective way to solve the problem that infrared remote sensing images are usually suffering low resolution due to hardware limitations. Most previous learning-based SISR methods just use synthetic HR-LR image pairs (obtained by bicubic kernels) to learn the mapping from LR images to HR images. However, the underlying degradation in the real world is often different from the synthetic method, i.e., the real LR images are obtained through a more complex degradation kernel, which leads to the adaptation problem and poor SR performance. To handle this problem, we propose a novel closed-loop framework that can not only make full use of the learning ability of the channel attention module but also introduce the information of real images as much as possible through a closed-loop structure. Our network includes two independent generative networks for down-sampling and super-resolution, respectively, and they are connected to each other to get more information from real images. We make a comprehensive analysis of the training data, resolution level and imaging spectrum to validate the performance of our network for infrared remote sensing image super-resolution. Experiments on real infrared remote sensing images show that our method achieves superior performance in various training strategies of supervised learning, weakly supervised learning and unsupervised learning. Especially, our peak signal-to-noise ratio (PSNR) is 0.9 dB better than the second-best unsupervised super-resolution model on PROBA-V dataset.

@Article{rs15040882,
AUTHOR = {Zhang, Haopeng and Zhang, Cong and Xie, Fengying and Jiang, Zhiguo},
TITLE = {A Closed-Loop Network for Single Infrared Remote Sensing Image Super-Resolution in Real World},
JOURNAL = {Remote Sensing},
VOLUME = {15},
YEAR = {2023},
NUMBER = {4},
ARTICLE-NUMBER = {882},
URL = {https://www.mdpi.com/2072-4292/15/4/882},
ISSN = {2072-4292},
DOI = {10.3390/rs15040882}
}
            

Kernel Attention Transformer for Histopathology Whole Slide Image Analysis and Assistant Cancer Diagnosis
Yushan Zheng, Jun Li, Jun Shi, Fengying Xie, Jianguo Huai, Ming Cao and Zhiguo Jiang*
IEEE Transactions on Medical Imaging(TMI), 2023
Abstract   BibTeX  

Transformer has been widely used in histopathology whole slide image analysis. However, the design of token-wise self-attention and positional embedding strategy in the common Transformer limits its effectiveness and efficiency when applied to gigapixel histopathology images. In this paper, we propose a novel kernel attention Transformer (KAT) for histopathology WSI analysis and assistant cancer diagnosis. The information transmission in KAT is achieved by cross-attention between the patch features and a set of kernels related to the spatial relationship of the patches on the whole slide images. Compared to the common Transformer structure, KAT can extract the hierarchical context information of the local regions of the WSI and provide diversified diagnosis information. Meanwhile, the kernel-based cross-attention paradigm significantly reduces the computational amount. The proposed method was evaluated on three large-scale datasets and was compared with 8 state-of-the-art methods. The experimental results have demonstrated the proposed KAT is effective and efficient in the task of histopathology WSI analysis and is superior to the state-of-the-art methods.

@ARTICLE{10093771,
  author={Zheng, Yushan and Li, Jun and Shi, Jun and Xie, Fengying and Huai, Jianguo and Cao, Ming and Jiang, Zhiguo},
  journal={IEEE Transactions on Medical Imaging}, 
  title={Kernel Attention Transformer for Histopathology Whole Slide Image Analysis and Assistant Cancer Diagnosis}, 
  year={2023},
  volume={42},
  number={9},
  pages={2726-2739},
  keywords={Transformers;Histopathology;Feature extraction;Kernel;Cancer;Task analysis;Training;WSI;transformer;cross-attention;gastric cancer;endometrial cancer},
  doi={10.1109/TMI.2023.3264781}}
            

Cervical Cell Classification Using Multi-Scale Feature Fusion and Channel-Wise Cross-Attention
Jun Shi, Xinyu Zhu, Yuan Zhang, Yushan Zheng*, Zhiguo Jiang and Liping Zheng
IEEE 20th International Symposium on Biomedical Imaging(ISBI), 2023
Abstract   BibTeX  

Cervical cancer is one of the prevalent malignant tumors in women, and accurate cervical cell classification is clinically significant for early screening of cervical cancer. In this paper, we propose a novel cervical cell classification method based on multi-scale feature fusion and channel-wise cross-attention. Specifically, the multi-scale cell features are combined from the perspective of channels, and then the fused multi-scale features are fed into multi-head channel-wise cross-attention to explore the channel dependencies and non-local semantic information, which are encoded into the high-level CNN features through Multi-Layer Perceptron (MLP) with residual structure. More importantly, the Re-Attention is applied to exploit the correlation among different attention heads. Experiments on three public cervical cell datasets, SIPaKMeD, Herlev and Motic, demonstrate the effectiveness of the method for cervical cell classification.

@INPROCEEDINGS{10230475,
  author={Shi, Jun and Zhu, Xinyu and Zhang, Yuan and Zheng, Yushan and Jiang, Zhiguo and Zheng, Liping},
  booktitle={2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI)}, 
  title={Cervical Cell Classification Using Multi-Scale Feature Fusion and Channel-Wise Cross-Attention}, 
  year={2023},
  volume={},
  number={},
  pages={1-5},
  keywords={Head;Correlation;Malignant tumors;Semantics;Robustness;Cervical cancer;Biomedical imaging;Cervical cancer;cervical cell classification;cross-attention mechanism;multi-scale feature;deep learning},
  doi={10.1109/ISBI53787.2023.10230475}}
            

A Key-Points Based Anchor-Free Cervical Cell Detector
Tong Shu, Jun Shi, Yushan Zheng*, Zhiguo Jiang and Lanlan Yu
IEEE Engineering in Medicine & Biology Society (EMBC), 2023
Abstract   BibTeX  

Cervical cell detection is crucial to cervical cytology screening at early stage. Currently most cervical cell detection methods use anchor-based pipeline to achieve the localization and classification of cells, e.g. faster R-CNN and YOLOv3. However, the anchors generally need to be pre-defined before training and the detection performance is inevitably sensitive to these pre-defined hyperparameters (e.g. number of anchors, anchor size and aspect ratios). More importantly, these preset anchors fail to conform to the cells with different morphology at inference phase. In this paper, we present a key-points based anchor-free cervical cell detector based on YOLOv3. Compared with the conventional YOLOv3, the proposed method applies a key-points based anchor-free strategy to represent the cells in the initial prediction phase instead of the preset anchors. Therefore, it can generate more desirable cell localization effect through refinement. Furthermore, PAFPN is applied to enhance the feature hierarchy. GIoU loss is also introduced to optimize the small cell localization in addition to focal loss and smooth L1 loss. Experimental results on cervical cytology ROI datasets demonstrate the effectiveness of our method for cervical cell detection and the robustness to different liquid-based preparation styles (i.e. drop-slide, membrane-based and sedimentation).

@inproceedings{inproceedings,
author = {Shu, Tong and Shi, Jun and Zheng, Yushan and Jiang, Zhiguo and Yu, Lanlan},
year = {2023},
month = {07},
pages = {1-5},
title = {A Key-Points Based Anchor-Free Cervical Cell Detector},
volume = {2023},
doi = {10.1109/EMBC40787.2023.10341092}
}
            

Position-Aware Masked Autoencoder for Histopathology WSI Representation Learning
Kun Wu, Yushan Zheng*, Jun Shi*, Fengying Xie and Zhiguo Jiang
Medical Image Computing and Computer Assisted Intervention (MICCAI), 2023
Abstract   BibTeX   Code

Transformer-based multiple instance learning (MIL) framework has been proven advanced for whole slide image (WSI) analysis. However, existing spatial embedding strategies in Transformer can only represent fixed structural information, which are hard to tackle the scale-varying and isotropic characteristics of WSIs. Moreover, the current MIL cannot take advantage of a large number of unlabeled WSIs for training. In this paper, we propose a novel self-supervised whole slide image representation learning framework named position-aware masked autoencoder (PAMA), which can make full use of abundant unlabeled WSIs to improve the discrimination of slide features. Moreover, we propose a position-aware cross-attention (PACA) module with a kernel reorientation (KRO) strategy, which makes PAMA able to maintain spatial integrity and semantic enrichment during the training. We evaluated the proposed method on a public TCGA-Lung dataset with 3,064 WSIs and an in-house Endometrial dataset with 3,654 WSIs, and compared it with 6 state-of-the-art methods. The results of experiments show our PAMA is superior to SOTA MIL methods and SSL methods. The code will be available at https://github.com/WkEEn/PAMA.

@InProceedings{10.1007/978-3-031-43987-2_69,
author="Wu, Kun
and Zheng, Yushan
and Shi, Jun
and Xie, Fengying
and Jiang, Zhiguo",
editor="Greenspan, Hayit
and Madabhushi, Anant
and Mousavi, Parvin
and Salcudean, Septimiu
and Duncan, James
and Syeda-Mahmood, Tanveer
and Taylor, Russell",
title="Position-Aware Masked Autoencoder for Histopathology WSI Representation Learning",
booktitle="Medical Image Computing and Computer Assisted Intervention -- MICCAI 2023",
year="2023",
publisher="Springer Nature Switzerland",
address="Cham",
pages="714--724",
abstract="Transformer-based multiple instance learning (MIL) framework has been proven advanced for whole slide image (WSI) analysis. However, existing spatial embedding strategies in Transformer can only represent fixed structural information, which are hard to tackle the scale-varying and isotropic characteristics of WSIs. Moreover, the current MIL cannot take advantage of a large number of unlabeled WSIs for training. In this paper, we propose a novel self-supervised whole slide image representation learning framework named position-aware masked autoencoder (PAMA), which can make full use of abundant unlabeled WSIs to improve the discrimination of slide features. Moreover, we propose a position-aware cross-attention (PACA) module with a kernel reorientation (KRO) strategy, which makes PAMA able to maintain spatial integrity and semantic enrichment during the training. We evaluated the proposed method on a public TCGA-Lung dataset with 3,064 WSIs and an in-house Endometrial dataset with 3,654 WSIs, and compared it with 6 state-of-the-art methods. The results of experiments show our PAMA is superior to SOTA MIL methods and SSL methods. The code will be available at https://github.com/WkEEn/PAMA.",
isbn="978-3-031-43987-2"
}
            

WSODet: A Weakly Supervised Oriented Detector for Aerial Object Detection
Zhiwen Tan, Zhiguo Jiang, Chen Guo and Haopeng Zhang*
IEEE Transactions on Geoscience and Remote Sensing (TGRS), 2023
Abstract   BibTeX  

In contrast to natural objects, aerial targets are usually non-axis aligned with arbitrary orientations. However, mainstream weakly supervised object detection (WSOD) methods can only predict horizontal bounding boxes (HBBs) from existing proposals generated by offline algorithms. To predict oriented bounding boxes (OBBs) for aerial targets while testing images end-to-end without proposals, WSODet is designed leveraging on layerwise relevance propagation (LRP) and point set representation (RepPoints). To be specific, based on the mainstream WSOD framework, LRP on multiple instance learning branch (MIL-LRP) is conducted to decrease the uncertainty and ambiguity of feature map. Then, a pseudo oriented label generation algorithm is designed to obtain OBB pseudolabels, which serve as supervision to train an oriented RepPoint Net under the guidance of improved oriented loss function (IOLF). During the test, input images are sent to oriented RepPoint branch (ORB) to obtain OBB predictions without proposals. Extensive experiments on the detection in optical remote sensing images (DIOR), Northwestern Polytechnical University (NWPU) VHR-10.v2, and HRSC2016 datasets demonstrate the effectiveness of our method to predict precise oriented aerial objects, achieving 22.2%, 46.5%, and 43.3% mAP, respectively. Moreover, training jointly with ORB boosts the results of the original WSOD framework compared with the existing WSOD methods even if there is no specific design for the original structure.

@ARTICLE{10049586,
  author={Tan, Zhiwen and Jiang, Zhiguo and Guo, Chen and Zhang, Haopeng},
  journal={IEEE Transactions on Geoscience and Remote Sensing},
  title={WSODet: A Weakly Supervised Oriented Detector for Aerial Object Detection},
  year={2023},
  volume={61},
  number={},
  pages={1-12},
  keywords={Proposals;Object detection;Uncertainty;Detectors;Prediction algorithms;Feature extraction;Convolutional neural networks;Deep learning;layerwise relevance propagation (LRP);oriented object detection;point set representation (RepPoints);weakly supervised object detection (WSOD)},
  doi={10.1109/TGRS.2023.3247578}
}
            

Deep-Learning-Based Direct Attitude Estimation for Uncooperative Known Space Objects
Sijia Qiao, Haopeng Zhang*, Fengying Xie and Zhiguo Jiang
IEEE Transactions on Aerospace and Electronic Systems (TAES), 2023
Abstract   BibTeX  

Attitude estimation for uncooperative known space objects plays a critical role in intelligent space technology. In recent years, deep-learning-based methods for estimating the attitude of space objects have made significant progress, particularly through indirect methods based on key point detection. While highly accurate, the indirect method requires high texture detail and is time-consuming, which limits its application in CNN-based methods. Therefore, we investigate a direct attitude estimation method that is faster, more generalized, and better suited for space applications. We propose a fast and accurate network called DSOAE-Net for estimating the attitude of space objects using monocular images. We introduce a pose representation method that is particularly suitable for space object attitude estimation, which enhances the performance of attitude estimation. We also propose a general pipeline for directly estimating attitude and explore and experiment with each stage of the pipeline. Through detailed experiments, we validate that our proposed method achieves optimal results within this framework. The proposed method demonstrates a balance between speed and accuracy, with the best accuracy results achieved among compared methods on a weakly textured dataset.

@ARTICLE{10288068,
  author={Qiao, Sijia and Zhang, Haopeng and Xie, Fengying and Jiang, Zhiguo},
  journal={IEEE Transactions on Aerospace and Electronic Systems},
  title={Deep-Learning-Based Direct Attitude Estimation for Uncooperative Known Space Objects},
  year={2023},
  volume={},
  number={},
  pages={1-18},
  keywords={Estimation;Feature extraction;Space vehicles;Task analysis;Deep learning;Cameras;Three-dimensional displays;Attitude estimation;deep learning;uncooperative known space object},
  doi={10.1109/TAES.2023.3325801}
}
            

Dynamic Informative Proposal-Based Iterative Training Procedure for Weakly Supervised Object Detection in Remote Sensing Images
Zhiwen Tan, Zhiguo Jiang, Chen Guo and Haopeng Zhang*
IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing (JSTARS), 2023
Abstract   BibTeX  

Weakly supervised object detection (WSOD) is an increasingly important task in remote sensing images. However, mainstream WSOD methods often rely on low-quality proposals due to the complex backgrounds of remote sensing images. Moreover, applying strong data augmentations directly in WSOD methods can introduce significant noise, which can hinder training procedures that rely only on image-level ground truth labels. To address these issues, we propose a dynamic informative proposal-based iterative WSOD training procedure. Specifically, we implement an informative proposal reconstruction (IPR) method to generate more informative proposals dynamically. We also use a proposal-based contrastive learning (PBCL) technique to steadily improve the quality of generated proposals. In addition, we employ a pseudolabel learning-based multistage (PLMS) training procedure to progressively improve the quality of new informative proposals while alleviating the noise catastrophe caused by strong data augmentations. Extensive experiments demonstrate the effectiveness of our proposed method in generating higher quality proposals and enhancing model generalization. Our method achieves state-of-the-art results in optical remote sensing images (DIOR), Northwestern Polytechnical University (NWPU) VHR-10.v2, and HRSC2016 datasets.

@ARTICLE{10183994,
  author={Tan, Zhiwen and Jiang, Zhiguo and Guo, Chen and Zhang, Haopeng},
  journal={IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing}, 
  title={Dynamic Informative Proposal-Based Iterative Training Procedure for Weakly Supervised Object Detection in Remote Sensing Images}, 
  year={2023},
  volume={16},
  number={},
  pages={6614-6626},
  keywords={Proposals;Training;Remote sensing;Object detection;Feature extraction;Data augmentation;Intellectual property;Contrastive learning;deep learning;feature similarity;pseudolabel learning;weakly supervised object detection (WSOD)},
  doi={10.1109/JSTARS.2023.3295732}}
            

Infrared Small Target Detection Based on Prior Weighed Sparse Decomposition
Dongning Yang, Haopeng Zhang*, Fengying Xie and Zhiguo Jiang
Image and Graphics Technologies and Applications (IGTAS), 2023
Abstract   BibTeX  

Infrared small target detection is a critical topic and research focus in target detection. Compared to visible light and radar detection, infrared imaging-based detection can effectively avoid illumination limitations and potential exposure risks. However, detecting small infrared targets with complex backgrounds and significant noise is challenging, and existing algorithms often have low detection rates, high false alarm rates, long calculation times, and unsatisfactory performance. To address these issues, we proposes an infrared small target detection algorithm based on sparse representation. The algorithm enhances target sparsity through multi-scale contrast saliency mapping and global gray value fusion, leveraging the low rank of the background. We evaluate the proposed method on SIRST dataset and compare its performance with traditional and recent algorithms. The results demonstrate the superiority of our algorithm in terms of detection rate, false alarm rate, and calculation time.

@InProceedings{10.1007/978-981-99-7549-5_11,
author="Yang, Dongning
and Zhang, Haopeng
and Xie, Fengying
and Jiang, Zhiguo",
editor="Yongtian, Wang
and Lifang, Wu",
title="Infrared Small Target Detection Based on Prior Weighed Sparse Decomposition",
booktitle="Image and Graphics Technologies and Applications",
year="2023",
publisher="Springer Nature Singapore",
address="Singapore",
pages="141--153",
isbn="978-981-99-7549-5"
}
            

Unsupervised Multi-Spectral Image Super-Resolution Based on Conditional Variational Autoencoder
Zhexin Han, Ning Zhao, Haopeng Zhang* and Zhiguo Jiang
IEEE International Geoscience and Remote Sensing Symposium (IGARSS), 2023
Abstract   BibTeX  

Unsupervised super-resolution aims to enhance the quality of images without high-resolution (HR) labels during the training stage, making it applicable to real-world scenarios. However, unsupervised super-resolution methods face the challenge of effectively learning the internal structure of images due to the absence of high-quality HR images as references. Moreover, multi-spectral remote sensing images often contain stochastic features caused by cloud and fog occlusions. These occlusions make it difficult to achieve accurate reconstruction of occluded areas through direct modeling of deep features in multi-spectral images. In this paper, we propose a method inspired by conditional variational autoencoders to address the issue of stochastic features in unsupervised multi-spectral super-resolution. Additionally, we introduce a channel attention feature fusion module to combine two types of features. We evaluated our unsupervised multi-spectral image super-resolution method using a real satellite remote sensing dataset. Experimental results demonstrate the qualitative and quantitative effectiveness of our approach.

@INPROCEEDINGS{10282224,
  author={Han, Zhexin and Zhao, Ning and Zhang, Haopeng and Jiang, Zhiguo},
  booktitle={IGARSS 2023 - 2023 IEEE International Geoscience and Remote Sensing Symposium},
  title={Unsupervised Multi-Spectral Image Super-Resolution Based on Conditional Variational Autoencoder},
  year={2023},
  volume={},
  number={},
  pages={5127-5130},
  keywords={Training;Adaptation models;Satellites;Fuses;Superresolution;Stochastic processes;Sensors;image super-resolution;unsupervised learning;condition variational autoencoder;remote sensing},
  doi={10.1109/IGARSS52108.2023.10282224}
}
            

Kernel Attention Transformer (KAT) for Histopathology Whole Slide Image Classification
Yushan Zheng*, Jun Li, Jun Shi, Fengying Xie and Zhiguo Jiang
Medical Image Computing and Computer Assisted Intervention (MICCAI), 2022
Abstract   BibTeX   Code

Transformer has been widely used in histopathology whole slide image (WSI) classification for the purpose of tumor grading, prognosis analysis, etc. However, the design of token-wise self-attention and positional embedding strategy in the common Transformer limits the effectiveness and efficiency in the application to gigapixel histopathology images. In this paper, we propose a kernel attention Transformer (KAT) for histopathology WSI classification. The information transmission of the tokens is achieved by cross-attention between the tokens and a set of kernels related to a set of positional anchors on the WSI. Compared to the common Transformer structure, the proposed KAT can better describe the hierarchical context information of the local regions of the WSI and meanwhile maintains a lower computational complexity. The proposed method was evaluated on a gastric dataset with 2040 WSIs and an endometrial dataset with 2560 WSIs, and was compared with 5 state-of-the-art methods. The experimental results have demonstrated the proposed KAT is effective and efficient in the task of histopathology WSI classification and is superior to the state-of-the-art methods.

@inproceedings{zheng2022kernel,
author    = {Yushan Zheng, Jun Li, Jun Shi, Fengying Xie, Zhiguo Jiang},
title     = {Kernel Attention Transformer (KAT) for Histopathology Whole Slide Image Classification},
booktitle = {Medical Image Computing and Computer Assisted Intervention 
            -- MICCAI 2022},
year      = {2022}
}
            

Histopathology Cross-Modal Retrieval based on Dual-Transformer Network
Dingyi Hu, Yushan Zheng*, Fengying Xie, Zhiguo Jiang and Jun Shi
IEEE International Conference on Bioinformatics and Bioengineering (BIBE), 2022
Abstract   BibTeX  

Computer-aided cancer diagnosis (CAD) methods based on the histopathological images have achieved great development. The content-based whole slide image (WSI) retrieval is one of the important application that can search for the informative data to assist clinical diagnosis. It is notable that the current retrieval system are mainly developed based on the image content and image labels. The diagnosis report for the WSIs given by the pathologists are also valuable data, but have not yet been adequately considered in modeling. In this paper, we propose a cross-modal retrieval framework based on histopathology WSIs and diagnosis report, which can simultaneously achieve four retrieval tasks for histopathology database across WSIs and diagnosis reports. The compact binary features from both WSIs and diagnosis reports are first extracted, and then built in a common vision-language semantic feature space by the constraint of the designed cross hashing loss function. The method was verified on a gastric histopathology dataset that contains 932 gastric cases with 4 lesion categories. Experimental results have demonstrated the effectiveness of the proposed method in the cross-modal retrieval tasks for digital pathology system.

@INPROCEEDINGS{9973633,
  author={Hu, Dingyi and Xie, Fengying and Jiang, Zhiguo and Zheng, Yushan and Shi, Jun},
  booktitle={2022 IEEE 22nd International Conference on Bioinformatics and Bioengineering (BIBE)}, 
  title={Histopathology Cross-Modal Retrieval based on Dual-Transformer Network}, 
  year={2022},
  volume={},
  number={},
  pages={97-102},
  keywords={Solid modeling;Design automation;Histopathology;Databases;Semantics;Feature extraction;Data models;Histopathology image analysis;Cross-modal retrieval;CAD},
  doi={10.1109/BIBE55377.2022.00028}}
            

Lesion-Aware Contrastive Representation Learning For Histopathology Whole Slide Images Analysis
Jun Li, Yushan Zheng*, Kun Wu, Jun Shi*, Fengying Xie and Zhiguo Jiang
Medical Image Computing and Computer Assisted Intervention (MICCAI), 2022
Abstract   BibTeX   Code

Local representation learning has been a key challenge to promote the performance of the histopathological whole slide images analysis. The previous representation learning methods almost follow the supervised learning paradigm. However, manual annotation for large-scale WSIs is time-consuming and labor-intensive. Hence, the self-supervised contrastive learning has recently attracted intensive attention. Contrastive learning on the design of informative of the positive and negative pairs, but common contrastive learning methods treat each sample as a single class, which leads to the class collision problems, especially in the domain of histopathology image analysis. In this paper, we proposed a novel contrastive representation learning framework called Lesion-Aware Contrastive Learning (LACL) for histopathology whole slide image analysis. We built a module called lesion queue based on the memory bank structure to store the representations of different classes of WSIs, which allowed the model to selectively sample the negative pairs during the training. Moreover, We designed a queue refinement strategy to purify the representations stored in the lesion queue. The experimental results demonstrate that LACL achieves the best performance in histopathology image representation learning on different datasets, and outperforms state-of-the-art methods under different WSI classification benchmarks.

@inproceedings{li2022lesion,
author    = {Jun Li, Yushan Zheng*, Kun Wu, Jun Shi, Fengying Xie, Zhiguo Jiang},
title     = {Lesion-Aware Contrastive Representation Learning For Histopathology Whole Slide Images Analysis},
booktitle = {Medical Image Computing and Computer Assisted Intervention 
              -- MICCAI 2022},
year      = {2022}
}
            

Multi-Frame Super-Resolution With Raw Images Via Modified Deformable Convolution
Gongzhe Li, Linwei Qiu, Haopeng Zhang, Fengying Xie, Zhiguo Jiang
IEEE ICASSP, 2022
Abstract   BibTeX  

In this paper we propose a novel model towards multi-frame super-resolution, which leverages multiple RAW images and yields a super-resolved RGB image. To facilitate the pixel misalignment in burst photography, we apply a refined Pyramid Cascading and Deformable Convolution (PCD) feature alignment module. A new 3D deformable convolution fusion module is proposed subsequently to merge the information from all frames adaptively. In addition, we employ an encoder-decoder network to restore color and details in sRGB space after super-resolving images in linear space. Extensive experiments demonstrate the superiority of our architecture and the strength of multi-frame super-resolution with RAW images.

@inproceedings{li2022multi,
  author={Li, Gongzhe and Qiu, Linwei and Zhang, Haopeng and Xie, Fengying and Jiang, Zhiguo},
  booktitle={ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, 
  title={Multi-Frame Super-Resolution With Raw Images Via Modified Deformable Convolution}, 
  year={2022},
  pages={2155-2159},
  doi={10.1109/ICASSP43922.2022.9747407}
}
            

A Two-Stage Shake-Shake Network for Long-tailed Recognition of SAR Aerial View Objects
Gongzhe Li, Linpeng Pan, Linwei Qiu, Zhiwen Tan, Fengying Xie, Haopeng Zhang*
18th IEEE Workshop on Perception Beyond the Visible Spectrum (PBVS Workshop 2022) in conjunction with CVPR 2022
PDF Abstract   BibTeX   Code

Synthetic Aperture Radar (SAR) has received more attention due to its complementary superiority on capturing significant information in the remote sensing area. However, for an Aerial View Object Classification (AVOC) task, SAR images still suffer from the long-tailed distribution of the aerial view objects. This disparity limit the performance of classification methods, especially for the data-sensitive deep learning models. In this paper, we propose a twostage shake-shake network to tackle the long-tailed learning problem. Specifically, it decouples the learning procedure into the representation learning stage and the classification learning stage. Moreover, we apply the test time augmentation (TTA) and the classification with alternating normalization (CAN) to improve the accuracy. In the PBVS 1 2022 Multi-modal Aerial View Object Classification Challenge Track 1, our method achieves 21.82% and 27.97% accuracy in the development phase and testing phase respectively, which wins the top-tier among all the participants

@inproceedings{
  author={Gongzhe Li, Linpeng Pan, Linwei Qiu, Zhiwen Tan, Fengying Xie, Haopeng Zhang},
  booktitle={18th IEEE Workshop on Perception Beyond the Visible Spectrum (PBVS Workshop 2022) in conjunction with CVPR 2022}, 
  title={A Two-Stage Shake-Shake Network for Long-tailed Recognition of SAR Aerial View Objects}, 
  year={2022},
  pages={245-256},
}
            

Few-Shot Multi-Class Ship Detection in Remote Sensing Images Using Attention Feature Map and Multi-Relation Detector
Haopeng Zhang*, Xingyu Zhang, Gang Meng, Chen Guo, Zhiguo Jiang
Remote Sensing, 2022
PDF Abstract   BibTeX  

Monitoring and identification of ships in remote sensing images is of great significance for port management, marine traffic, marine security, etc. However, due to small size and complex background, ship detection in remote sensing images is still a challenging task. Currently, deep-learning-based detection models need a lot of data and manual annotation, while training data containing ships in remote sensing images may be in limited quantities. To solve this problem, in this paper, we propose a few-shot multi-class ship detection algorithm with attention feature map and multi-relation detector (AFMR) for remote sensing images. We use the basic framework of You Only Look Once (YOLO), and use the attention feature map module to enhance the features of the target. In addition, the multi-relation head module is also used to optimize the detection head of YOLO. Extensive experiments on publicly available HRSC2016 dataset and self-constructed REMEX-FSSD dataset validate that our method achieves a good detection performance.

@Article{zhang2022few,
AUTHOR = {Zhang, Haopeng and Zhang, Xingyu and Meng, Gang and Guo, Chen and Jiang, Zhiguo},
TITLE = {Few-Shot Multi-Class Ship Detection in Remote Sensing Images Using Attention Feature Map and Multi-Relation Detector},
JOURNAL = {Remote Sensing},
VOLUME = {14},
YEAR = {2022},
NUMBER = {12},
ARTICLE-NUMBER = {2790},
URL = {https://www.mdpi.com/2072-4292/14/12/2790},
ISSN = {2072-4292},
DOI = {10.3390/rs14122790}
}
            

Encoding Histopathology Whole Slide Images with Location-aware Graphs for Diagnostically Relevant Regions Retrieval
Yushan Zheng, Zhiguo Jiang*, Jun Shi*, Fengying Xie, Haopeng Zhang, Wei Luo, Dingyi Hu, Shujiao Sun, Zhongmin Jiang and Chenghai Xue
Medical Image Analysis, 2022
PDF Abstract   BibTeX   Code

Content-based histopathological image retrieval (CBHIR) has become popular in recent years in histopathological image analysis. CBHIR systems provide auxiliary diagnosis information for pathologists by searching for and returning regions that are contently similar to the region of interest (ROI) from a pre-established database. It is challenging and yet significant in clinical applications to retrieve diagnostically relevant regions from a database consisting of histopathological whole slide images (WSIs). In this paper, we propose a novel framework for regions retrieval from WSI database based on location-aware graphs and deep hash techniques. Compared to the present CBHIR framework, both structural information and global location information of ROIs in the WSI are preserved by graph convolution and self-attention operations, which makes the retrieval framework more sensitive to regions that are similar in tissue distribution. Moreover, benefited from the graph structure, the proposed framework has good scalability for both the size and shape variation of ROIs. It allows the pathologist to define query regions using free curves according to the appearance of tissue. Thirdly, the retrieval is achieved based on the hash technique, which ensures the framework is efficient and adequate for practical large-scale WSI database. The proposed method was evaluated on an in-house endometrium dataset with 2650 WSIs and the public ACDC-LungHP dataset. The experimental results have demonstrated that the proposed method achieved a mean average precision above 0.667 on the endometrium dataset and above 0.869 on the ACDC-LungHP dataset in the task of irregular region retrieval, which are superior to the state-of-the-art methods. The average retrieval time from a database containing 1855 WSIs is 0.752 ms.

@Article{zheng2022encoding,
  author  = {Zheng, Yushan and Jiang, Zhiguo and Shi, Jun and Xie, Fengying and Zhang, Haopeng and
              Luo, Wei and Hu, Dingyi and Sun, Shujiao and Jiang, Zhongmin and Xue, Chenghai},
  title   = {Encoding histopathology whole slide images with location-aware graphs for diagnostically relevant regions retrieval},
  journal = {Medical Image Analysis},
  year    = {2022},
  volumn  = {76},
  pages   = {102308},
  doi     = {https://doi.org/10.1016/j.media.2021.102308},
}
              

Weakly Supervised Histopathological Image Representation Learning based on Contrastive Dynamic Clustering
Jun Li, Zhiguo Jiang, Yushan Zheng*, Haopeng Zhang, Jun Shi, Dingyi Hu, Wei Luo, Zhongmin Jiang and Chenghai Xue
SPIE Medical Imaging, 2022
PDF Abstract   BibTeX   Code

Feature representations of histopathology whole slide images (WSIs) are crucial to the downstream applications for computer-aided cancer diagnosis, including whole slide image classification, region of interest detection, hash retrieval, prognosis analysis, and other high-level inference tasks. State-of-the-art methods for whole slide image feature extraction generally rely on supervised learning algorithms based on fine-grained manual annotations, unsupervised learning algorithms without annotation, or directly use pre-trained features. At present, there is a lack of research on weakly supervised feature learning methods that only utilize WSI-level labeling. In this paper, we propose a weakly supervised framework that learns the feature representations of various lesion areas from histopathology whole slide images. The proposed framework consists of a contrastive learning network as the backbone and a designed contrastive dynamic clustering (CDC) module to embedding the lesion information into the feature representations. The proposed method was evaluated on a large scale endometrial whole slide image dataset. The experimental results have demonstrated that our method can learn discriminative feature representations for histopathology image classification and the quantitative performance of our method is close to the fully-supervision learning methods

@inproceedings{li2021weakly,
  author    = {Jun Li, Zhiguo Jiang, Yushan Zheng, Haopeng Zhang, Jun Shi, Dingyi Hu,
               Wei Luo, Zhongmin Jiang, and Chenghai Xue},
  title     = {Weakly Supervised Histopathological Image Representation Learning based on Contrastive Dynamic Clustering},
  booktitle = {SPEI Medical Imaging 2022},
  year      = {2022},
}
              

Global-local attention network for weakly supervised cervical cytology ROI analysis
Jun Shi*, Kun Wu, Yushan Zheng*, Yuxin He, Jun Li, Zhiguo Jiang and Lanlan Yu
IEEE 19th International Symposium on Biomedical Imaging, 2022
PDF Abstract   BibTeX  

Existing supervised Convolutional Neural Network (CNN) approaches for cervical cytology image analysis generally rely on the heavy manual annotation for each cell or cell mass and thus lead to extensive time and effort. In this paper, we propose a global-local network for weakly supervised cervical cytology region of interest (ROI) analysis. It aims to perform the classification for ROIs and further classify the cells only with the ROI labels. Specifically, the proposed method firstly detects the cells within ROI and extracts the CNN features of cells. Then attention-based bidirectional LSTM (Att-BLSTM) is applied to explore the global contextual information within ROI. On the other hand, the Vision Transformer (ViT) is used to exploit the local attentive representations of the cells in ROIs. The cross attention (CA) is applied to incorporate the global contextual features and local patterns and thus generates more discriminative feature representation of ROI. More importantly, the CA score is used as the pseudo label to select top and least attentive cells. Therefore, the in-theclass and out-of-the-class CA branches are trained to achieve the cell classification. Experimental results demonstrate the effectiveness of our method for cervical cytology ROI and cell classification, and the weak supervision of the image-level label has great potential to promote the automatic whole slide cervical image analysis and alleviate the workload of cytologists.

@Article{shi2022global,
  author  = {Jun Shi*, Kun Wu, Yushan Zheng*, Yuxin He, Jun Li, Zhiguo Jiang and Lanlan Yu},
  title   = {Global-local attention network for weakly supervised cervical cytology ROI analysis},
  booktitle = {IEEE 19th International Symposium on Biomedical Imaging},
  year    = {2022}
}
              

Hyperspectral Image Classification using Spectral-spatial Hypergraph Convolution Neural Network
Zhongtian Ma, Zhiguo Jiang, Haopeng Zhang
SPIE Remote Sensing, 2021
PDF Abstract   BibTeX  

Deep learning methods, especially convolutional neural networks(CNN), have been widely used in hyperspectral image(HSI) classification. Recently, graph convolutional networks (GCN) have shown great potential in HSI classification problem. However, the existing GCN-based methods have several problems. First, the existing methods rely too much on the adjacency matrix, which cannot be changed during training. Furthermore, most of them can only use a single kind of feature, and fail to extract the spectral-spatial information from the HSI. Finally, for the existing GCN-based methods, it is difficult to achieve the same accuracy as the mature CNN methods. In this paper, we propose a spectral-spatial hypergraph convolutional neural network (S2HCN) for HSI classification. Compared with the existing GCN-based methods, S2HCN has the following advantages. Different from the adjacency matrix that is fixed during training of GCN, S2HCN can dynamically update the weight of the hyperedge during training, which reduces the reliance on prior information to a certain extent. In addition, S2HCN generates hyperedges from the spectral and spatial features independently, and adopts the incidence matrix composed of all hyperedges to replace the adjacency matrix in GCN. In this way, the spectral and spatial features can be better integrated. Finally, compared to a simple graph structure, the hypergraph structure can express the high-dimensional relationships in the data, which is beneficial to classification problems. Sufficient experiments on two popular HSI datasets have proved the effectiveness of S2HCN.

@inproceedings{10.1117/12.2599787,
author = {Zhongtian Ma and Zhiguo Jiang and Haopeng Zhang},
title = {{Hyperspectral image classification using spectral-spatial hypergraph convolution neural network}},
volume = {11862},
booktitle = {Image and Signal Processing for Remote Sensing XXVII},
editor = {Lorenzo Bruzzone and Francesca Bovolo},
organization = {International Society for Optics and Photonics},
publisher = {SPIE},
pages = {133 -- 140},
keywords = {graph convolution networks, hypergraph learning, hyperspectral image classification, feature fusion, deep learning},
year = {2021},
doi = {10.1117/12.2599787},
URL = {https://doi.org/10.1117/12.2599787}
}

              

Hyperspectral Image Classification using Feature Fusion Hypergraph Convolution Neural Network
Zhongtian Ma, Zhiguo Jiang, Haopeng Zhang
IEEE Transactions on Geoscience and Remote Sensing, 2021
PDF Abstract   BibTeX   Code

Convolutional neural networks (CNN) and graph representation learning are two common methods for hyperspectral image (HSI) classification. Recently, graph convolutional neural networks (GCN), a combination of CNN and graph representation learning, have shown great potential in HSI classification problem. However, the existing GCN-based methods have many problems, such as over dependence on the adjacency matrix, usage of a single modal feature, and lower accuracy than the mature CNN method. In this paper, we propose a feature fusion hypergraph convolutional neural network (F2HNN) for HSI classification. F2HNN first generates hyperedges from features of different modalities to construct a hypergraph representing multi-modal features in HSI. Then, the HSI and the extracted hypergraph are input into the hypergraph convolutional neural network for learning. In addition, we proposes three feature fusion strategies. The first strategy is the most basic spatial and spectral feature fusion. The second strategy fuses the spectral features extracted by a pre-trained multilayer perceptron (MLP) with the spatial features to reduce the redundant information of the original spectral features. The third strategy uses the fusion of CNN features, spectral features and spatial features to explore the capabilities of F2HNN. Sufficient experiments on four datasets have proved the effectiveness of F2HNN.

@ARTICLE{9590574,
  author={Ma, Zhongtian and Jiang, Zhiguo and Zhang, Haopeng},
  journal={IEEE Transactions on Geoscience and Remote Sensing},
  title={Hyperspectral Image Classification using Feature Fusion Hypergraph Convolution Neural Network},
  year={2021},
  volume={},
  number={},
  pages={1-1},
  doi={10.1109/TGRS.2021.3123423}
}

              

Self-Attention Fusion Module for Single Remote Sensing Image Super-Resolution
Han Mei, Haopeng Zhang, Zhiguo Jiang
IEEE International Geoscience and Remote Sensing Symposium IGARSS, 2021
PDF Abstract   BibTeX  

Single image super-resolution (SISR) is an important procedure to improve many remote sensing applications. Global features play an important role in pixel generation of SISR. In this paper, we proposed a self-attention fusion module named as SAF module which combines spatial attention and channel attention in parallel to handle this problem. Our self-attention fusion module can be flexibly added to many popular deep-learning-based SISR models to further improve their representation ability and learn global features. Experiments on UC Merced dataset indicate that SAF module can improve the performance of classic SISR models and achieve state-of-the-art super-resolution results.

@INPROCEEDINGS{9553766,
  author={Mei, Han and Zhang, Haopeng and Jiang, Zhiguo},
  booktitle={2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS},
  title={Self-Attention Fusion Module for Single Remote Sensing Image Super-Resolution},
  year={2021},
  volume={},
  number={},
  pages={2883-2886},
  doi={10.1109/IGARSS47720.2021.9553766}}

              

Few Shot Object Detection in Remote Sensing Images
Xingyu Zhang, Haopeng Zhang, Zhiguo Jiang
SPIE Remote Sensing, 2021
PDF Abstract   BibTeX  

Object detection in remote sensing images has far-reaching significance. However, compared with object detection tasks in the field of natural images, there are still some challenge problems that need to be improved in remote sensing, due to the similarity and unbalanced scale between objects in remote sensing images. Currently, object detection algorithms based on deep learning, i.e. Faster-RCNN, YOLO, SSD, etc., have reached an incredible grade with the concerted efforts of researchers, and have been used in various aspects such as remote sensing, face recognition, pedestrian detection and so on. However, these methods always need a lot of labeled data, while collecting a dataset is labor intensive and time consuming. In order to overcome these difficulties, we introduce a few shot learning algorithm with attention mechanism for object detection in remote sensing images, aiming to detect objects of unknown classes with only a few labeled remote sensing images. We use the structure of Siamese-network to extract the features of the target from support images and query images, and then use the features of support images as a kernel to do a depth-wise convolution on the feature map of query image. By this way, we can enhance the characteristics of the target category and weaken the characteristics of other categories and the background. For unbalanced scale problem, our method takes care of various objects of different scales in the process of extracting features. We integrate the information of multi-layer feature maps and make predictions on three feature maps of different scales. Experiments on HRSC-2016 dataset validate the effectiveness of our method.

@inproceedings{10.1117/12.2598444,
author = {Xingyu Zhang and Haopeng Zhang and Zhiguo Jiang},
title = {{Few shot object detection in remote sensing images}},
volume = {11862},
booktitle = {Image and Signal Processing for Remote Sensing XXVII},
editor = {Lorenzo Bruzzone and Francesca Bovolo},
organization = {International Society for Optics and Photonics},
publisher = {SPIE},
pages = {76 -- 81},
keywords = {Few shot learning, Object detection, Remote sensing, Deep learning},
year = {2021},
doi = {10.1117/12.2598444},
URL = {https://doi.org/10.1117/12.2598444}
}
              

Frequency-based Convolutional Neural Network for Efficient Segmentation of Histopathology Whole Slide Images
Wei Luo, Yushan Zheng, Dingyi Hu, Jun Li, Chenghai Xue, and Zhiguo Jiang
International Conference on Image and Graphics (ICIG) 2021
PDF Abstract   BibTeX  

CNN-based methods for WSI segmentation are time-consuming under the limits of communication bandwidth and memory usage, due to the high pixel resolution of WSIs. In this paper, we propose a novel framework for accelerating the segmentation of digital histopathology WSIs in the frequency domain. Based on the characteristics of the JPEG format in data storage and transmission on the existing digital histopathological diagnosis cloud platform, we extract DCT coefficients from the JEPG decoding and compress them into the DCT feature cubes by a frequency selection block. Based on the DCT feature cubes, we propose an extremely light-weighted model named Efficient DCT-Network (EDCT-Net). The size of the input data, as well as the bandwidth requirement for CPU-GPU transmitting, for EDCT-net reduces by 96% compared to the common CNN-based methods. And, the number of model parameters and the floating-point operations (FLOPs) for EDCT-Net decreases by 98% and 94% compared to the baseline method. The experimental results have demonstrated that our method achieves a Dice score of 0.811 with only 8 frequency channels in the task of endometrial histopathology WSI segmentation, which is comparable with state-of-the-art methods.

@inproceedings{luo2021frequency,
  title      = "Frequency-Based Convolutional Neural Network for Efficient Segmentation of Histopathology Whole Slide Images.",
  author     = "Wei {Luo} and Yushan {Zheng} and Dingyi {Hu} and Jun {Li} and Chenghai {Xue} and Zhiguo {Jiang}",
  booktitle  = "International Conference on Image and Graphics",
  pages      = "584--596",
  year       = "2021"
}
              

Diagnostic Regions Attention Network (DRA-Net) for Histopathology WSI Recommendation and Retrieval
Yushan Zheng, Zhiguo Jiang*, Jun Shi, Fengying Xie, Haopeng Zhang, Huai Jianguo, Cao Ming, and Yang Xiaomiao
IEEE Transactions on Medical Imaging, 2021
PDF Abstract   BibTeX   Code

The development of whole slide imaging techniques and online digital pathology platforms have accelerated the popularization of telepathology for remote tumor diagnoses. During a diagnosis, the behavior information of the pathologist can be recorded by the platform and then archived with the digital case. The browsing path of the pathologist on the WSI is one of the valuable information in the digital database because the image content within the path is expected to be highly correlated with the diagnosis report of the pathologist. In this paper, we proposed a novel approach for computer-assisted cancer diagnosis named session-based histopathology image recommendation (SHIR) based on the browsing paths on WSIs. To achieve the SHIR, we developed a novel diagnostic regions attention network (DRA-Net) to learn the pathology knowledge from the image content associated with the browsing paths. The DRA-Net does not rely on the pixel-level or region-level annotations of pathologists. All the data for training can be automatically collected by the digital pathology platform without interrupting the pathologists' diagnoses. The proposed approaches were evaluated on a gastric dataset containing 983 cases within 5 categories of gastric lesions. The quantitative and qualitative assessments on the dataset have demonstrated the proposed SHIR framework with the novel DRA-Net is effective in recommending diagnostically relevant cases for auxiliary diagnosis. The MRR and MAP for the recommendation are respectively 0.816 and 0.836 on the gastric dataset.

@Article{zheng2020diagnostic,
  author  = {Zheng, Yushan and Jiang, Zhiguo and Shi, Jun and Xie, Fengying and
             Zhang, Haopeng and Huai, Jianguo and Cao, Ming and Yang, Xiaomiao},
  title   = {Diagnostic Regions Attention Network (DRA-Net) for Histopathology
             WSI Recommendation and Retrieval},
  journal = {IEEE Transactions on Medical Imaging},
  volume  = {40},
  number  = {3},
  pages   = {1090--1103}
  year    = {2021},
  doi     = {10.1109/TMI.2020.3046636},
}
              

Stain Standardization Capsule For Application-driven Histopathological Image Normalization
Yushan Zheng, Zhiguo Jiang*, Haopeng Zhang, Fengying Xie, Dingyi Hu, Shujiao Sun, Jun Shi, and Chenghai Xue
IEEE Journal of Biomedical and Health Informatics, 2021
PDF Abstract   BibTeX   Code

Color consistency is crucial to developing robust deep learning methods for histopathological image analysis. With the increasing application of digital histopathological slides, the deep learning methods are probably developed based on the data from multiple medical centers. This requirement makes it a challenging task to normalize the color variance of histopathological images from different medical centers. In this paper, we propose a novel color standardization module named stain standardization capsule based on the capsule network and the corresponding dynamic routing algorithm. The proposed module can learn and generate uniform stain separation outputs for histopathological images in various color appearance without the reference to manually selected template images. The proposed module is light and can be jointly trained with the application-driven CNN model. The proposed method was validated on three histopathology datasets and a cytology dataset, and was compared with state-of-the-art methods. The experimental results have demonstrated that the SSC module is effective in improving the performance of histopathological image analysis and has achieved the best performance in the compared methods.

@article{zheng2020stain,
  author    = {Zheng, Yushan and Jiang, Zhiguo and Zhang, Haopeng and Xie, Fengying and Hu, Dingyi
               and Sun, Shujiao and Shi, Jun and Xue, Chenghai},
  title     = {Stain standardization capsule for application-driven histopathological image normalization},
  journal   = {IEEE Journal of Biomedical and Health Informatics},
  volume    = {25},
  number    = {2},
  pages     = {337--347}
  year      = {2021},
}
              

Out-of-Region Keypoint Localization for 6D Pose Estimation
Xin Zhang, Zhiguo Jiang, and Haopeng Zhang*
Image and Vision Computing, 2020
PDF   Abstract   BibTeX  

This paper addresses the problem of instance level 6D pose estimation from a single RGB image. Our approach simultaneously detects objects and recovers poses by predicting the 2D image locations of the object's 3D bounding box vertices. Specifically, we focus on the challenge of locating virtual keypoints outside the object region proposals, and propose a boundary-based keypoint representation which incorporates classification and regression schemes to reduce output space. Moreover, our method predicts localization confidences and alleviates the influence of difficult keypoints by a voting process. We implement the proposed method based on 2D detection pipeline, meanwhile bridge the feature gap between detection and pose estimation. Our network has real-time processing capability, which runs 30 fps on a GTX 1080Ti GPU. For single object and multiple objects pose estimation on two benchmark datasets, our approach achieves competitive or superior performance compared with state-of-the-art RGB based pose estimation methods.

       
            
@Article{ZhangIVC2020,
AUTHOR = {Xin Zhang and Zhiguo Jiang and Haopeng Zhang},
TITLE = {Out-of-Region Keypoint Localization for 6D Pose Estimation},
JOURNAL = {Image and Vision Computing},
VOLUME = {93},
YEAR = {2020},
NUMBER = {C},
PAGES = {103854},
ISSN = {0262-8856},
DOI = {10.1016/j.imavis.2019.103854}
}   
                  

Non-pairwise-trained Cycle Convolutional Neural Network for Single Remote Sensing Image Super-Resolution
Haopeng Zhang, Pengrui Wang and Zhiguo Jiang
IEEE Transactions on Geoscience and Remote Sensing, 2020
PDF   Abstract   BibTeX  

Single image super-resolution (SISR) is to recover the high spatial resolution image from a single low spatial resolution one, which is a useful procedure for many remote sensing applications. Most previous convolutional neural network (CNN)-based methods adopt supervised learning. However, paired highresolution and low-resolution remote sensing images are actually hard to acquire for supervised learning SR methods. To handle this problem, we propose a novel cycle convolutional neural network (Cycle-CNN). Our network consists of two generative CNNs for down-sampling and SR separately and can be trained with unpaired data. We perform comprehensive experiments on panchromatic and multispectral images of the GaoFen-2 satellite and the UC Merced land use data set. Experimental results indicate that our method achieves state-of-the-art CNN-based SR results and is robust against noise and blur in remote sensing images. Comprehensively considering super-resolved image quality and time costs, our proposed method outperforms the compared learning-based SISR approaches.

       
            
@ARTICLE{9151194,
  author    = {H. {Zhang} and P. {Wang} and Z. {Jiang}},
  journal   = {IEEE Transactions on Geoscience and Remote Sensing}, 
  title     = {Nonpairwise-Trained Cycle Convolutional Neural Network for Single Remote Sensing Image Super-Resolution}, 
  year      = {2020},
  volume    = {},
  number    = {},
  pages     = {1-12},
  }           
  
                  

Tracing Diagnosis Paths on Histopathology WSIs for Diagnostically Relevant Case Recommendation
Yushan Zheng, Zhiguo Jiang, Haopeng Zhang, Fengying Xie, and Jun Shi
Medical Image Computing and Computer Assisted Interventions (MICCAI), 2020
PDF   Abstract   BibTeX  

Telepathology has enabled the remote cancer diagnosis based on digital pathological whole slide images (WSIs). During the diagnosis, the behavior information of the pathologist can be recorded by the platform and then archived with the digital cases. The diagnosis path of the pathologist on a WSI is valuable information since the image content within the path is highly correlated with the diagnosis report of the pathologist. In this paper, we proposed a novel diagnosis path network (DPathNet). DPathNet utilizes the diagnosis paths of pathologists on the WSIs as the supervision to learn the pathology knowledge from the image content. Based on the DPathNet, we develop a novel approach for computer-aided cancer diagnosis named session-based histopathology image recommendation (SHIR). SHIR summaries the information of a WSI while the pathologist browsing the WSI and actively recommends the relevant cases within similar image content from the database. The proposed approaches are evaluated on a gastric dataset containing 983 cases within 5 categories of gastric lesions. The experimental results have demonstrated the effectiveness of the DPathNet to the SHIR task and the supervision of the diagnosis path is sufficient to train the DPathNet. The MRR and MAP of the proposed SHIR framework are respectively 0.741 and 0.777 on the gastric dataset.

       
            
  @inproceedings{zheng2020tracing,
  author    = {Zheng, Yushan and Jiang, Zhiguo and Zhang, Haopeng and Xie, Fengying and Shi, Jun},
  title     = {Tracing Diagnosis Paths on Histopathology WSIs for 
               Diagnostically Relevant Case Recommendation},
  booktitle = {Medical Image Computing and Computer Assisted Intervention 
               -- MICCAI 2020},
  year      = {2020},
}              
                  

Informative retrieval framework for histopathology whole slides images based on deep hashing network
Dingyi Hu, Yushan Zheng*, Haopeng Zhang, Jun Shi, Fengying Xie, and Zhiguo Jiang
IEEE 17th International Symposium on Biomedical Imaging (ISBI), 2020
PDF   Abstract   BibTeX  

Histopathology image retrieval is an emerging application for Computer-aided cancer diagnosis. However, most of current retrieval methods have ignored the characteristic of histopathology images. It causes repeated results within similar image content from the same slide. Meanwhile, the data sets cannot be sufficiently utilized. To solve these issues, we proposed an informative retrieval framework based on deep hashing network. Specifically, a novel loss function for the hashing network and a retrieval strategy are designed, which contributes to more informative retrieval results without reducing the retrieval precision. The proposed method was verified on the ACDC-LungHP dataset and compared with the state-of-the-art method. The experimental results have demonstrated the effectiveness of our method in the retrieval of large-scale database containing histopathology while slide images.

@inproceedings{hu2020informative,
  author    = {Hu, Dingyi and Zheng, Yushan and Zhang, Haopeng and Shi, Jun and
               Xie, Fengying and Jiang, Zhiguo},
  title     = {Informative retrieval framework for histopathology 
               whole slides images based on deep hashing network},
  booktitle = {IEEE 17th International Symposium on Biomedical Imaging (ISBI)},
  year      = {2020},
}              
                  

Cancer Sensitive Cascaded Networks (CSC-Net) for Efficient Histopathology Whole Slide Image Segmentation
Shujiao Sun, Huining Yuan, Yushan Zheng*, Haopeng Zhang, Dingyi Hu, and Zhiguo Jiang
IEEE 17th International Symposium on Biomedical Imaging (ISBI), 2020
PDF   Abstract   BibTeX  

Automatic segmentation of histopathological whole slide images (WSIs) is challenging due to the high resolution and large scale. In this paper, we proposed a cascade strategy for fast segmentation of WSIs based on convolutional neural networks. Our segmentation framework consists of two U-Net structures which are trained with samples from different magnifications. Meanwhile, we designed a novel cancer sensitive loss (CSL), which is effective in improving the sensitivity of cancer segmentation of the first network and reducing the false positive rate of the second network. We conducted experiments on ACDC-LungHP dataset and compared our method with 2 state-of-the-art segmentation methods. The experimental results have demonstrated that the proposed method can improve the segmentation accuracy and meanwhile reduce the amount of computation. The dice score coefficient and precision of lung cancer segmentation are 0.694 and 0.947, respectively, which are superior to the compared methods.

@inproceedings{sun2020cancer,
  author    = {Sun, Shujiao and Yuan, Huining and Zheng, Yushan and Zhang, Haopeng
               and Dingyi Hu and Jiang, Zhiguo},
  title     = {Cancer sensitive cascaded networks (CSC-Net) for efficient histopathology 
               whole slide image segmentation},
  booktitle = {IEEE 17th International Symposium on Biomedical Imaging (ISBI)},
  year      = {2020},
}              
                  

Deep Learning Based Electric Pylon Detection in Remote Sensing Images
Sijia Qiao,Yu Sun and Haopeng Zhang
Remote Sensing(RS), 2020
PDF   Abstract   BibTeX  

The working condition of power network can significantly influence urban development. Among all the power facilities, electric pylon has an important effect on the normal operation of electricity supply. Therefore, the work status of electric pylons requires continuous and real-time monitoring. Considering the low efficiency of manual detection, we propose to utilize deep learning methods for electric pylon detection in high-resolution remote sensing images in this paper. To verify the effectiveness of electric pylon detection methods based on deep learning, we tested and compared the comprehensive performance of 10 state-of-the-art deep-learning-based detectors with different characteristics. Extensive experiments were carried out on a self-made dataset containing 1500 images. Moreover, 50 relatively complicated images were selected from the dataset to test and evaluate the adaptability to actual complex situations and resolution variations. Experimental results show the feasibility of applying deep learning methods to electric pylon detection. The comparative analysis can provide reference for the selection of specific deep learning model in actual electric pylon detection task.

       
            
@article{Qiao_2020, 
  title     = {Deep Learning Based Electric Pylon Detection in Remote Sensing Images}, 
  volume    = {12}, 
  ISSN      = {2072-4292}, 
  url       = {http://dx.doi.org/10.3390/rs12111857}, 
  DOI       = {10.3390/rs12111857}, 
  number    = {11}, 
  journal   = {Remote Sensing}, 
  publisher = {MDPI AG}, 
  author    = {Sijia Qiao, Yu Sunand Haopeng Zhang}, 
  year      = {2020}, 
  month     = {Jun}, 
  pages     = {1857}
}              
                  

A Comparable Study of CNN-Based Single Image Super-Resolution for Space-Based Imaging Sensors
Haopeng Zhang, Pengrui Wang, Cong Zhang, and Zhiguo Jiang
Sensors, 2019
Abstract   BibTeX  

In the case of space-based space surveillance (SBSS), images of the target space objects captured by space-based imaging sensors usually suffer from low spatial resolution due to the extremely long distance between the target and the imaging sensor. Image super-resolution is an effective data processing operation to get informative high resolution images. In this paper, we comparably study four recent popular models for single image super-resolution based on convolutional neural networks (CNNs) with the purpose of space applications. We specially fine-tune the super-resolution models designed for natural images using simulated images of space objects, and test the performance of different CNN-based models in different conditions that are mainly considered for SBSS. Experimental results show the advantages and drawbacks of these models, which could be helpful for the choice of proper CNN-based super-resolution method to deal with image data of space objects.

  @article{zhang2019comparable,
    title={A Comparable Study of CNN-Based Single Image Super-Resolution for Space-Based Imaging Sensors},
    author={Zhang, Haopeng and Wang, Pengrui and Zhang, Cong and Jiang, Zhiguo},
    journal={Sensors},
    volume={19},
    number={14},
    pages={3234},
    year={2019},
    publisher={Multidisciplinary Digital Publishing Institute}
  }              
                

Stain Standardization Capsule: A pre-processing module for histopathological image analysis
Yushan Zheng, Zhiguo Jiang, Haopeng Zhang, Jun Shi, and Fengying Xie
MICCAI 2019 Computational Pathology Workshop (COMPAY19)
PDF   Abstract   BibTeX  

Color consistency is crucial to developing robust deep learning methods for histopathological image analysis. With the increasing use of digital histopathological images, the deep learning methods are likely developed based on the data from multiple medical centers. This requirement makes it a challenge task to normalize the color variance of histopathological images from different medical centers. In this paper, we proposed a novel color standardization module named stain standardization capsule (SSC) based on the paradigm of capsules network and the corresponding dynamic routing algorithm. The proposed module can learn and generate uniform stain separation outputs for histopathological images in various color appearance without the reference to manually selected template images. The SSC module is light and can be trained end-to-end with the application-driven CNN model. The proposed method was validated on two public datasets and compared with the state-of-the-art methods. The experimental results have demonstrated that the SSC module is effective in color normalization for histopathological images and achieves the best performance in the compared methods.

@inproceedings{zheng2019stain,
  author    = {Zheng, Yushan and Jiang, Zhiguo and Zhang, Haopeng 
                and Shi, Jun and Xie, Fengying},
  title     = {Stain Standardization Capsule: A pre-processing module
                for histopathological image analysis},
  booktitle = {MICCAI 2019 Workshop -- COMPAY19},
  year      = {2019},
}
              

A Comparative Study of CNN and FCN for Histopathology Whole Slide Image Analysis
Shujiao Sun, Bonan Jiang, Yushan Zheng, and Fengying Xie
International Conference on Image and Graphics (ICIG), 2019
PDF   Abstract  

Automatic analysis of histopathological whole slide images (WSIs) is a challenging task. In this paper, we designed two deep learning structures based on a fully convolutional network (FCN) and a convolutional neural network (CNN), to achieve the segmentation of carcinoma regions from WSIs. FCN is developed for segmentation problems and CNN focuses on classification. We designed experiments to compare the performances of the two methods. The results demonstrated that CNN performs as well as FCN when applied to WSIs in high resolution. Furthermore, to leverage the advantages of CNN and FCN, we integrate the two methods to obtain a complete framework for lung cancer segmentation. The proposed methods were evaluated on the ACDC-LungHP dataset. The final dice coefficient for cancerous region segmentation is 0.770.

Real-time 6D pose estimation from a single RGB image
Xin Zhang, Zhiguo Jiang, and Haopeng Zhang
Image and Vision Computing, 2019
PDF   Abstract   BibTeX  

We propose an end-to-end deep learning architecture for simultaneously detecting objects and recovering 6D poses in an RGB image. Concretely, we extend the 2D detection pipeline with a pose estimation module to indirectly regress the image coordinates of the object's 3D vertices based on 2D detection results. Then the object's 6D pose can be estimated using a Perspective-n-Point algorithm without any post-refinements. Moreover, we elaborately design a backbone structure to maintain spatial resolution of low level features for pose estimation task. Compared with state-of-the-art RGB based pose estimation methods, our approach achieves competitive or superior performance on two benchmark datasets at an inference speed of 25 fps on a GTX 1080Ti GPU, which is capable of real-time processing.

  @article{zhang2019real,
    title={Real-time 6D pose estimation from a single RGB image},
    author={Zhang, Xin and Jiang, Zhiguo and Zhang, Haopeng},
    journal={Image and Vision Computing},
    volume={89},
    pages={1--11},
    year={2019},
    publisher={Elsevier}
  }
                

Vision-Based Satellite Recognition and Pose Estimation Using Gaussian Process Regression
Haopeng Zhang, Cong Zhang, Zhiguo Jiang, Yuan Yao, and Gang Meng
International Journal of Aerospace Engineering Volume, 2019
Abstract   BibTeX  

In this paper, we address the problem of vision-based satellite recognition and pose estimation, which is to recognize the satellite from multiviews and estimate the relative poses using imaging sensors. We propose a vision-based method to solve these two problems using Gaussian process regression (GPR). Assuming that the regression function mapping from the image (or feature) of the target satellite to its category or pose follows a Gaussian process (GP) properly parameterized by a mean function and a covariance function, the predictive equations can be easily obtained by a maximum-likelihood approach when training data are given. These explicit formulations can not only offer the category or estimated pose by the mean value of the predicted output but also give its uncertainty by the variance which makes the predicted result convincing and applicable in practice. Besides, we also introduce a manifold constraint to the output of the GPR model to improve its performance for satellite pose estimation. Extensive experiments are performed on two simulated image datasets containing satellite images of 1D and 2D pose variations, as well as different noises and lighting conditions. Experimental results validate the effectiveness and robustness of our approach.

  @article{zhang2019vision,
    title={Vision-Based Satellite Recognition and Pose Estimation Using Gaussian Process Regression},
    author={Zhang, Haopeng and Zhang, Cong and Jiang, Zhiguo and Yao, Yuan and Meng, Gang},
    journal={International Journal of Aerospace Engineering},
    volume={2019},
    year={2019},
    publisher={Hindawi}
  }
                    
                

Unsupervised Saliency Model with Color Markov Chain for Oil Tank Detection
Ziming Liu, Danpei Zhao, Zhenwei Shi, and Zhiguo Jiang
Remote Sensing, 2019
Abstract   BibTeX    

Traditional oil tank detection methods often use geometric shape information. However, it is difficult to guarantee accurate detection under a variety of disturbance factors, especially various colors, scale differences, and the shadows caused by view angle and illumination. Therefore, we propose an unsupervised saliency model with Color Markov Chain (US-CMC) to deal with oil tank detection. To avoid the influence of shadows, we make use of the CIE Lab space to construct a Color Markov Chain and generate a bottom-up latent saliency map. Moreover, we build a circular feature map based on a radial symmetric circle, which makes true targets to be strengthened for a subjective detection task. Besides, we combine the latent saliency map with the circular feature map, which can effectively suppress other salient regions except for oil tanks. Extensive experimental results demonstrate that it outperforms 15 saliency models for remote sensing images (RSIs). Compared with conventional oil tank detection methods, US-CMC has achieved better results and is also more robust for view angle, shadow, and shape similarity problems.

  @article{liu2019unsupervised,
    title={Unsupervised Saliency Model with Color Markov Chain for Oil Tank Detection},
    author={Liu, Ziming and Zhao, Danpei and Shi, Zhenwei and Jiang, Zhiguo},
    journal={Remote Sensing},
    volume={11},
    number={9},
    pages={1089},
    year={2019},
    publisher={Multidisciplinary Digital Publishing Institute}
  }
              

Star Detection and Accurate Centroiding for the Geosynchronous Interferometric Infrared Sounder of Fengyun-4A
Haopeng Zhang, Yi Su, Lei Yang, Jian Shang, Chengbao Liu, Jing Wang, and Shengxiong Zhou
IEEE Access, 2019
PDF   Abstract   BibTeX    

Extracting accurate star centroids in the observed star images is one of the key problems for image navigation of the geosynchronous interferometric infrared sounder (GIIRS) of Fengyun-4A Satellite (FY-4A), the first scientific experimental satellite of the new generation of Chinese geostationary meteorological satellite Fengyun-4 series. Compared with star sensors which are widely used for star observation, it is challenging to detect the 2×2 star spot from the focused star images of GIIRS and calculate the star centroid in high precision. In this paper, we propose a star detection and centroiding method based on trajectory search and trajectory fitting. Since the launch of FY-4A in December 2016, our centroiding method has been tested in-orbit for over two years. The extensive experiments show that the star centroiding error of our method is less than 0.3 pixels, which makes an important contribution to image navigation of FY-4A.

  @article{zhang2019star,
    title={Star Detection and Accurate Centroiding for the Geosynchronous Interferometric Infrared Sounder of Fengyun-4A},
    author={Zhang, Haopeng and Su, Yi and Yang, Lei and Shang, Jian and Liu, Chengbao and Wang, Jing and Zhou, Shengxiong and Jiang, Zhiguo and Zhang, Zhiqing},
    journal={IEEE Access},
    volume={7},
    pages={18510--18520},
    year={2019},
    publisher={IEEE}
  }
              

Subjective Saliency Model Driven by Multi-Cues Stimulus for Airport Detection
Danpei Zhao, Jiayi Li, Zhenwei Shi, Zhiguo Jiang, and Cai Meng
IEEE Access, 2019
PDF   Abstract   BibTeX    

Traditional saliency models designed for natural scene images usually use human visual characteristics to detect the target, but those salient areas in the remote sensing images (RSIs) may not be the targets we are really interested in. Taking both remote sensing image attributes and airport characteristics into account, we put forward a subjective saliency model driven by multi-cues stimulus for the airport's detection (MCS-SSM). Different from traditional saliency models, this model mainly relies on the subjective target detection task to find specific target area eliminating disturbance from other salient targets. Based on the low-level features, we train an LDA classifier by only small target samples and then build an object feature map. In the meantime, the shape information based on line density is extracted to get a shape map. Depending on the fusion result of two saliency maps, we optimize the subjective saliency map with SVM classifier. Moreover, MST density map is generated to suppress background and highlight interesting airport regions in the subjective saliency map. Consequently, MCS-SSM can respectively take the target, the background, and the detection task as multiple cues to quickly locate interest airport targets in RSI with a large cover area. MCS-SSM breaks through the limitations on color, texture, and other low-level characteristics compared with traditional saliency models, which are more targeted to detect the specific targets. The extensive experimental results demonstrate that the proposed MCS-SSM outperforms nine state-of-the-art saliency models. Besides, it has a higher detection rate and better effective performance than other three airport detection approaches.

  @article{zhao2019subjective, 
    title={Subjective Saliency Model Driven by Multi-Cues Stimulus for Airport Detection}, 
    @article{zhao2019subjective,
      title={Subjective Saliency Model Driven by Multi-Cues Stimulus for Airport Detection},
      author={Zhao, Danpei and Li, Jiayi and Shi, Zhenwei and Jiang, Zhiguo and Meng, Cai},
      journal={IEEE Access},
      volume={7},
      pages={32118--32127},
      year={2019},
      publisher={IEEE}
    }
              

Analytical entry guidance for coordinated flight with multiple no-fly-zone constraints
Wenbin Yu， Wanchun Chen， Zhiguo Jiang， Wanqing Zhang， and Penglei Zhao
Aerospace Science and Technology, 2019
Abstract   BibTeX    

This paper addresses the problem of coordinating a group of Hypersonic Glide Vehicles (HGVs) for the goal of simultaneous arrival in the presence of multiple No-Fly Zones (NFZs). Firstly, a high-precision analytical solution of flight time is derived from the nonlinear entry dynamics model built over a spherical and rotating Earth. Next, an entry guidance considering multi-NFZs and flight-time constraints is designed based on the new analytical flight-time formula as well as the existing 3-D analytical gliding-trajectory formulae. In the longitudinal part of the guidance, the flight-time and downrange formulae are used jointly to plan the longitudinal reference profile by considering both energy-management and flight-time requirements. In the lateral part, the downrange and crossrange formulae are used to plan the bank-reversal sequence according to the NFZ constraints. Additionally, in order to improve the accuracy of terminal time, speed, and altitude, a multi-objective iterative planning scheme employing onboard trajectory simulation is put forward and enabled at a time between the last two bank reversals to fine-tune the remaining short trajectory. In this scheme, the quasi-Newton method is improved by the use of directional derivatives such that the number of the trajectory simulations required to calculate the Jacobian matrix is reduced from 3 to 2 in each iteration, which greatly reduced the amount of calculation. Finally, a flight-time coordination scheme is developed for multiple HGVs to determine the starting times of entry flight, which can further determine the launch times once the boost guidance is specified. The superior performance of the guidance is demonstrated by Monte-Carlo simulations in stochastic disturbed circumstances.

  @article{yu2019analytical,
    title={Analytical entry guidance for coordinated flight with multiple no-fly-zone constraints},
    author={Yu, Wenbin and Chen, Wanchun and Jiang, Zhiguo and Zhang, Wanqing and Zhao, Penglei},
    journal={Aerospace Science and Technology},
    volume={84},
    pages={273--290},
    year={2019},
    publisher={Elsevier}
  }
              

Adaptive Color Deconvolution For Histological WSI Normalization
Yushan Zheng, Zhiguo Jiang, Haopeng Zhang, Fengying Xie, Jun Shi, and Chenghai Xue
Computer Methods and Programs in Biomedicine, 2019
PDF   Abstract   BibTeX   Code  

Background and Objective Color consistency of histological images is significant for developing reliable computer-aided diagnosis (CAD) systems. However, the color appearance of digital histological images varies across different specimen preparations, staining, and scanning situations. This variability affects the diagnosis and decreases the accuracy of CAD approaches. It is important and challenging to develop effective color normalization methods for digital histological images. Methods We proposed a novel adaptive color deconvolution (ACD) algorithm for stain separation and color normalization of hematoxylin-eosin-stained whole slide images (WSIs). To avoid artifacts and reduce the failure rate of normalization, multiple prior knowledges of staining are considered and embedded in the ACD model. To improve the capacity of color normalization for various WSIs, an integrated optimization is designed to simultaneously estimate the parameters of the stain separation and color normalization. The solving of ACD model and application of the proposed method involves only pixel-wise operation, which makes it very efficient and applicable to WSIs. Results The proposed method was evaluated on four WSI-datasets including breast, lung and cervix cancers and was compared with 6 state-of-the-art methods. The proposed method achieved the most consistent performance in color normalization according to the quantitative metrics. Through a qualitative assessment for 500 WSIs, the failure rate of normalization was 0.4% and the structure and color artifacts were effectively avoided. Applied to CAD methods, the area under receiver operating characteristic curve for cancer image classification was improved from 0.842 to 0.914. The average time of solving the ACD model is 2.97 s. Conclusions The proposed ACD model has prone effective for color normalization of hematoxylin-eosin-stained WSIs in various color appearances. The model is robust and can be applied to WSIs containing different lesions. The proposed model can be efficiently solved and is effective to improve the performance of cancer image recognition, which is adequate for developing automatic CAD programs and systems based on WSIs.

@article{zhengCMPB2019,
  title   = {Adaptive color deconvolution for histological WSI normalization},
  author  = {Yushan Zheng and Zhiguo Jiang and Haopeng Zhang and Fengying Xie and Jun Shi and Chenghai Xue},
  journal = {Computer Methods and Programs in Biomedicine},
  volume  = {170},
  pages   = {107-120},
  doi     = {doi.org/10.1016/j.cmpb.2019.01.008},
  year    = {2019}
}
              

On-Board Ship Detection in Micro-Nano Satellite Based on Deep Learning and COTS Component
Yuan Yao, Zhiguo Jiang, Haopeng Zhang, and Yu Zhou
Remote Sensing, 2019
Dataset : https://pan.baidu.com/s/16of1rN6U1dVB5DcCe0ytBQ passwd:uj3n
PDF   Abstract   BibTeX    

Micro-nano satellites have provided a large amount of remote sensing images for many earth observation applications. However, the hysteresis of satellite-ground mutual communication of massive remote sensing images and the low efficiency of traditional information processing flow have become the bottlenecks for the further development of micro-nano satellites. To solve this problem, this paper proposes an on-board ship detection scheme based on deep learning and Commercial Off-The-Shelf (COTS) component, which can be used to achieve near real-time on-board processing by micro-nano satellite computing platform. The on-board ship detection algorithm based on deep learning consists of a feature extraction network, Region Proposal Network (RPN) with square anchors, Global Average Pooling (GAP), and Bigger-Left Non-Maximum Suppression (BL-NMS). With the help of high performance COTS components, the proposed scheme can extract target patches and valuable information from remote sensing images quickly and accurately. A ground demonstration and verification system is built to verify the feasibility and effectiveness of our scheme. Our method achieves the performance with 95.9% recall and 80.5% precision in our dataset. Experimental results show that the scheme has a good application prospect in micro-nano satellites with limited power and computing resources.

@article{yaoRS2019,
  title   = {On-Board Ship Detection in Micro-Nano Satellite Based on Deep Learning and COTS Component},
  author  = {Yuan Yao and Zhiguo Jiang and Haopeng Zhang and Yu Zhou},
  journal = {Remote Sensing},
  volume  = {11},
  doi     = {10.3390/rs11070762},
  year    = {2019}
}
              

Online Exemplar-Based Fully Convolutional Network for Aircraft Detection in Remote Sensing Images
Bowen Cai, Zhiguo Jiang, Haopeng Zhang, Yuan Yao and Shanlan Nie
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, 2018
PDF   Abstract   BibTeX    

Convolutional neural network obtains remarkable achievements on target detection, due to its prominent capability on feature extraction. However, it still needs further study for aircraft detection task, since intraclass variation still restricts the accuracy of aircraft detection in remote sensing images. In this letter, we adopt regularity of aircraft circle response to design our end-to-end fully convolutional network (FCN), and embed online exemplar mining into our network to handle intraclass variation. The mined exemplars are employed to capture different intraclass characteristics, which effectively reduces the burden of network training. Specifically, we first select basic exemplars based on labeled information and initialize the relationships between exemplars and aircraft examples. Then, these relationships will be updated by the similarity of these examples in high-level features space. Finally, aircraft examples will be used to train different exemplar detectors according to updated relationships. Motivated by the geometric shape of aircraft, a circle response map is developed to construct our FCN to achieve more efficient aircraft detection. The comparative experiments indicate that superior performance of our network in accurate and efficient aircraft detection..

@article{caiRS2018,
  title   = {Online Exemplar-Based Fully Convolutional Network for Aircraft Detection in Remote Sensing Images},
  author  = {Bowen Cai and Zhiguo Jiang and Haopeng Zhang and Yuan Yao and Shanlan Nie},
  journal = {IEEE GEOSCIENCE AND REMOTE SENSING LETTERS},
  volume  = {15},
  pages   = {1095-1099},
  doi     = {10.1109/LGRS.2018.2829147},
  year    = {2018}
}
              

Higher Order Support Vector Random Fields for Hyperspectral Image Classification
Junli Yang, Zhiguo Jiang, Shuang Hao and Haopeng Zhang
ISPRS International Journal of Geo-Information, 2018
Abstract   BibTeX  

This paper addresses the problem of contextual hyperspectral image (HSI) classification. A novel conditional random fields (CRFs) model, known as higher order support vector random fields (HSVRFs), is proposed for HSI classification. By incorporating higher order potentials into a support vector random fields with a Mahalanobis distance boundary constraint (SVRFMC) model, the HSVRFs model not only takes advantage of the support vector machine (SVM) classifier and the Mahalanobis distance boundary constraint, but can also capture higher level contextual information to depict complicated details in HSI. The higher order potentials are defined on image segments, which are created by a fast unsupervised over-segmentation algorithm. The higher order potentials consider the spectral vectors of each of the segment's constituting pixels coherently, and weight these pixels with the output probability of the support vector machine (SVM) classifier in our framework. Therefore, the higher order potentials can model higher-level contextual information, which is useful for the description of challenging complex structures and boundaries in HSI. Experimental results on two publicly available HSI datasets show that the HSVRFs model outperforms traditional and state-of-the art methods in HSI classification, especially for datasets containing complicated details.

@article{yangIJGI2018,
  title   = {Higher Order Support Vector Random Fields for Hyperspectral Image Classification},
  author  = {Junli Yang and Zhiguo Jiang and Shuang Hao and Haopeng Zhang},
  journal = {2018 ISPRS INTERNATIONAL JOURNAL OF GEO-INFORMATION(IJGI)},
  volume  = {7},
  number  = {1},
  pages   = {19},
  doi     = {10.3390/ijgi7010019},
  year    = {2018}
}
              

Generating Region Proposals for Histopathological Whole Slide Image Retrieval
Yibing Ma, Zhiguo Jiang, Haopeng Zhang, Fengying Xie, Yushan Zheng, Huaqiang Shi, Yu Zhao and Jun Shi
Computer Methods and Programs in Biomedicine, 2018
Abstract   BibTeX  

Background and objective Content-based image retrieval is an effective method for histopathological image analysis. However, given a database of huge whole slide images (WSIs), acquiring appropriate region-of-interests (ROIs) for training is significant and difficult. Moreover, histopathological images can only be annotated by pathologists, resulting in the lack of labeling information. Therefore, it is an important and challenging task to generate ROIs from WSI and retrieve image with few labels. Methods This paper presents a novel unsupervised region proposing method for histopathological WSI based on Selective Search. Specifically, the WSI is over-segmented into regions which are hierarchically merged until the WSI becomes a single region. Nucleus-oriented similarity measures for region mergence and Nucleus–Cytoplasm color space for histopathological image are specially defined to generate accurate region proposals. Additionally, we propose a new semi-supervised hashing method for image retrieval. The semantic features of images are extracted with Latent Dirichlet Allocation and transformed into binary hashing codes with Supervised Hashing. Results The methods are tested on a large-scale multi-class database of breast histopathological WSIs. The results demonstrate that for one WSI, our region proposing method can generate 7.3 thousand contoured regions which fit well with 95.8% of the ROIs annotated by pathologists. The proposed hashing method can retrieve a query image among 136 thousand images in 0.29 s and reach precision of 91% with only 10% of images labeled. Conclusions The unsupervised region proposing method can generate regions as predictions of lesions in histopathological WSI. The region proposals can also serve as the training samples to train machine-learning models for image retrieval. The proposed hashing method can achieve fast and precise image retrieval with small amount of labels. Furthermore, the proposed methods can be potentially applied in online computer-aided-diagnosis systems.

@article{maCMPB2018,
  title   = {Generating region proposals for histopathological whole slide image retrieval},
  author  = {Yibing Ma and Zhiguo Jiang and Haopeng Zhang and Fengying Xie 
            and Yushan Zheng and Huaqiang Shi and Yu Zhao and Jun Shi},
  journal = {Computer Methods and Programs in Biomedicine},
  volume  = {159},
  pages   = {1 - 10},
  year    = {2018},
  issn    = {0169-2607},
  url     = {http://www.sciencedirect.com/science/article/pii/S0169260717312154},
}      
              

Histopathological Whole Slide Image Analysis Using Context-based CBIR
Yushan Zheng, Zhiguo Jiang, Haopeng Zhang, Fengying Xie, Yibing Ma, Huaqiang Shi and Yu Zhao
IEEE Transactions on Medical Imaging, 2018
PDF   Abstract   BibTeX   Supplementary  

Histopathological image classification (HIC) and content-based histopathological image retrieval (CBHIR) are two promising applications for histopathological whole slide image (WSI) analysis. HIC can efficiently predict the type of lesion involved in a histopathological image. In general, HIC can aid pathologists in locating high-risk cancer regions from a WSI by providing a cancerous probability map for the WSI. In contrast, CBHIR was developed to allow searches for regions with similar content for a region of interest (ROI) from a database consisting of historical cases. Sets of cases with similar content are accessible to pathologists, which can provide more valuable references for diagnosis. A drawback of the recent CBHIR framework is that a query ROI needs to be manually selected from a WSI. An automatic CBHIR approach for a WSI-wise analysis needs to be developed. In this paper, we propose a novel aided-diagnosis framework of breast cancer using whole slide images, which shares the advantages of both HIC and CBHIR. In our framework, CBHIR is automatically processed throughout the WSI, based on which a probability map regarding the malignancy of breast tumors is calculated. Through the probability map, the malignant regions in WSIs can be easily recognized. Furthermore, the retrieval results corresponding to each sub-region of the WSIs are recorded during the automatic analysis and are available to pathologists during their diagnosis. Our method was validated on fully annotated WSI datasets of breast tumors. The experimental results certify the effectiveness of the proposed method.

@article{zhengTMI18,
  author  = {Yushan Zheng and Zhiguo Jiang and Haopeng Zhang and Fengying Xie
             and Yibing Ma and Huaqiang Shi and Yu Zhao},
  title   = {Histopathological Whole Slide Image Analysis Using Context-based CBIR},
  journal = {IEEE Transactions on Medical Imaging},
  doi     = {10.1109/TMI.2018.2796130},
  year    = {Epub 2018 January 23}
}
              

Robust Spacecraft Component Detection in Point Clouds
Quanmao Wei, Zhiguo Jiang and Haopeng Zhang
Sensors, 2018
PDF   Abstract   BibTeX   Supplementary   Code  

Automatic component detection of spacecraft can assist in on-orbit operation and space situational awareness. Spacecraft are generally composed of solar panels and cuboidal or cylindrical modules. These components can be simply represented by geometric primitives like plane, cuboid and cylinder. Based on this prior, we propose a robust automatic detection scheme to automatically detect such basic components of spacecraft in three-dimensional (3D) point clouds. In the proposed scheme, cylinders are first detected in the iteration of the energy-based geometric model fitting and cylinder parameter estimation. Then, planes are detected by Hough transform and further described as bounded patches with their minimum bounding rectangles. Finally, the cuboids are detected with pair-wise geometry relations from the detected patches. After successive detection of cylinders, planar patches and cuboids, a mid-level geometry representation of the spacecraft can be delivered. We tested the proposed component detection scheme on spacecraft 3D point clouds synthesized by computer-aided design (CAD) models and those recovered by image-based reconstruction, respectively. Experimental results illustrate that the proposed scheme can detect the basic geometric components effectively and has fine robustness against noise and point distribution density.

@article{weiSensors18,
  author  = {Quanmao Wei and Zhiguo Jiang and Haopeng Zhang},
  title   = {Robust Spacecraft Component Detection in Point Clouds},
  journal = {Sensors},
  volume  = {18},
  year    = {2018},
  number  = {4},
  article number = {933},
  url     = {http://www.mdpi.com/1424-8220/18/4/933},
  issn    = {1424—8220},
  doi     = {10.3390/s18040933}
}
              

Vision-based Pose Estimation for Textureless Space Objects by Contour Points Matching
Xin Zhang, Zhiguo Jiang, Haopeng Zhang and Quanmao Wei
IEEE Transactions on Aerospace and Electronic Systems, 2018
Preprint   Abstract   BibTeX  

This paper presents a novel vision-based method to solve the 6-degree-of-freedom pose estimation problem of textureless space objects from a single monocular image. Our approach follows a coarse-to-fine procedure, utilizing only shape and contour information of the input image. To achieve invariance to initialization, we select a series of projection images which are similar to the input image and establish many-to-one 2D-3D correspondences by contour feature matching. Intensive attention is focused on outlier rejection and we introduce an innovative strategy to fully utilize geometric matching information to guide pose calculation. Experiments based on simulated images are carried out, and the results manifest that pose estimation error of our approach is about 1% even in situations with heavy outlier correspondences.

@article{zhangTAES18,
  author  = {Xin Zhang and Zhiguo Jiang and Haopeng Zhang and Quanmao Wei},
  journal = {IEEE Transactions on Aerospace and Electronic Systems},
  title   = {Vision-based Pose Estimation for Textureless Space Objects
             by Contour Points Matching},
  year    = {2018},
  month   = {},
  volume  = {PP},
  number  = {99},
  pages   = {1—-1},
  issn    = {0018-9251},
  doi     = {10.1109/TAES.2018.2815879}
}
              

Size-scalable Content-based Histopathological Image Retrieval from Ratabase that Ronsists of WSIs
Yushan Zheng, Zhiguo Jiang, Haopeng Zhang, Fengying Xie, Yibing Ma, Huaqiang Shi and Yu Zhao
IEEE Journal of Biomedical and Health Informatics, 2017
PDF   Abstract   BibTeX  

Content-based image retrieval (CBIR) has been widely researched for histopathological images. It is challenging to retrieve contently similar regions from histopathological whole slide images (WSIs) for regions of interest (ROIs) in different size. In this paper, we propose a novel CBIR framework for database that consists of WSIs and size-scalable query ROIs. Each WSI in the database is encoded into a matrix of binary codes. When retrieving, a group of region proposals that have similar size with the query ROI are firstly located in the database through an efficient table-lookup approach. Then, these regions are ranked by a designed multi-binary-code-based similarity measurement. Finally, the top relevant regions and their locations in the WSIs as well as the corresponding diagnostic information are returned to assist pathologists. The effectiveness of the proposed framework is evaluated on a fine-annotated WSI database of epithelial breast tumors. The experimental results have proved that the proposed framework is effective for retrieval from database that consists of WSIs. Specifically, for query ROIs of 4096$\times$4096 pixels, the retrieval precision of the top 20 return has reached 96\% and the retrieval time is less than 1.5 second.

@article{zhengJBHI17,
  author  = {Yushan Zheng and Zhiguo Jiang Haopeng Zhang and Fengying Xie
             and Yibing Ma and Huaqiang Shi and Yu Zhao},
  title   = {Size-scalable Content-based Histopathological Image Retrieval
             from Database that Consists of WSIs},
  journal = {IEEE journal of biomedical and health informatics},
  doi     = {10.1109/jbhi.2017.2723014},
  year    = {2017}
}
              

Feature Extraction from Histopathological Images Based on Nucleus-guided Convolutional Neural Network for Breast Lesion Classification
Yushan Zheng, Zhiguo Jiang, Fengying Xie, Haopeng Zhang, Yibing Ma, Huaqiang Shi and Yu Zhao
Pattern Recognition, 2017
PDF   Abstract   BibTeX  

Feature extraction is a crucial and challenging aspect in the computer-aided diagnosis of breast cancer with histopathological images. In recent years, many machine learning methods have been introduced to extract features from histopathological images. In this study, a novel nucleus-guided feature extraction framework based on convolutional neural network is proposed for histopathological images. The nuclei are first detected from images, and then used to train a designed convolutional neural network with three hierarchy structures. Through the trained network, image-level features including the pattern and spatial distribution of the nuclei are extracted. The proposed features are evaluated through the classification experiment on a histopathological image database of breast lesions. The experimental results show that the extracted features effectively represent histopathological images, and the proposed framework achieves a better classification performance for breast lesions than the compared state-of-the-art methods.

@article{zhengPR17,
  author  = {Yushan Zheng and Zhiguo Jiang and Fengying Xie and Haopeng Zhang
             and Yibing Ma and Huaqiang Shi and Yu Zhao},
  title   = {Feature Extraction from Histopathological Images Based on Nucleus-guided
             Convolutional Neural Network for Breast Lesion Classification},
  journal = {Pattern Recognition},
  year    = {2017},
  volume  = {71},
  pages   = {14—-25},
  issn    = {0031-3203},
  doi     = {10.1016/j.patcog.2017.05.010}
}
              

3D Reconstruction of Space Objects from Multi-Views by A Visible Sensor
Haopeng Zhang, Quanmao Wei and Zhiguo Jiang
Sensors, 2017
PDF   Abstract   BibTeX  

In this paper, a novel 3D reconstruction framework is proposed to recover the 3D structural model of a space object from its multi-view images captured by a visible sensor. Given an image sequence, this framework first estimates the relative camera poses and recovers the depths of the surface points by the structure from motion (SFM) method, then the patch-based multi-view stereo (PMVS) algorithm is utilized to generate a dense 3D point cloud. To resolve the wrong matches arising from the symmetric structure and repeated textures of space objects, a new strategy is introduced, in which images are added to SFM in imaging order. Meanwhile, a refining process exploiting the structural prior knowledge that most sub-components of artificial space objects are composed of basic geometric shapes is proposed and applied to the recovered point cloud. The proposed reconstruction framework is tested on both simulated image datasets and real image datasets. Experimental results illustrate that the recovered point cloud models of space objects are accurate and have a complete coverage of the surface. Moreover, outliers and points with severe noise are effectively filtered out by the refinement, resulting in an distinct improvement of the structure and visualization of the recovered points.

@article{zhangSensors17,
  author  = {Haopeng Zhang and Quanmao Wei and Zhiguo Jiang},
  title   = {3D Reconstruction of Space Objects from Multi-Views by A Visible Sensor},
  journal = {Sensors},
  volume  = {17},
  year    = {2017},
  number  = {7},
  article number = {1689},
  url     = {http://www.mdpi.com/1424-8220/17/7/1689},
  issn    = {1424-8220},
  doi     = {10.3390/s17071689}
}
              

Spacecraft Component Detection in Point Clouds
Quanmao Wei, Zhiguo Jiang, Haopeng Zhang and Nie Shanlan
Image and Graphics Technologies and Applications (IGTA), 2017
PDF   Abstract   BibTeX   Code  

Component detection of spacecraft is significant for on-orbit operation and space situational awareness. Solar wings and main body are the major components of most spacecrafts, and can be described by geometric primitives like planes, cuboid or cylinder. Based on this prior, pipeline to automatically detect the basic components of spacecraft in 3D point clouds is presented, in which planes, cuboid and cylinder are successively detected. The planar patches are first detected as possible solar wings in point clouds of the recorded object. As for detection of the main body, inferring a cuboid main body from the detected patches is first attempted, and a further attempt to extract a cylinder main body is made if no cuboid exists. Dimensions are estimated for each component. Experiments on satellite point cloud data that are recovered by image-based reconstruction demonstrated effectiveness and accuracy of this pipeline.

@inproceedings{weiIGTA17,
  author    = {Quanmao Wei and Zhiguo Jiang and Haopeng Zhang and Shanlan Nie},
  editor    = {Yongtian Wang and Shengjin Wang and Yue Liu and Jian Yang
               and Xiaoru Yuan and Ran He and Henry Been-Lirn Duh},
  title     = {Spacecraft Component Detection in Point Clouds},
  booktitle = {Advances in Image and Graphics Technologies},
  year      = {2017},
  publisher = {Springer Singapore},
  address   = {Singapore},
  pages     = {210—-218},
  isbn      = {978-981-10-7389-2}
}
              

Airport Detection Using End-to-End Convolutional Neural Network with Hard Example Mining
Bowen Cai, Zhiguo Jiang, Haopeng Zhang, Danpei Zhao and Yuan Yao
Remote Sensing, 2017
PDF   Abstract   BibTeX  

Deep convolutional neural network (CNN) achieves outstanding performance in the field of target detection. As one of the most typical targets in remote sensing images (RSIs), airport has attracted increasing attention in recent years. However, the essential challenge for using deep CNN to detect airport is the great imbalance between the number of airports and background examples in large-scale RSIs, which may lead to over-fitting. In this paper, we develop a hard example mining and weight-balanced strategy to construct a novel end-to-end convolutional neural network for airport detection. The initial motivation of the proposed method is that backgrounds contain an overwhelming number of easy examples and a few hard examples. Therefore, we design a hard example mining layer to automatically select hard examples by their losses, and implement a new weight-balanced loss function to optimize CNN. Meanwhile, the cascade design of proposal extraction and object detection in our network releases the constraint on input image size and reduces spurious false positives. Compared with geometric characteristics and low-level manually designed features, the hard example mining based network could extract high-level features, which is more robust for airport detection in complex environment. The proposed method is validated on a multi-scale dataset with complex background collected from Google Earth. The experimental results demonstrate that our proposed method is robust, and superior to the state-of-the-art airport detection models.

@article{caiRS17,
  author  = {Bowen Cai and Zhiguo Jiang and Haopeng Zhang and Danpei Zhao and Yuan Yao},
  title   = {Airport Detection Using End-to-End Convolutional Neural Network
             with Hard Example Mining},
  journal = {Remote Sensing},
  volume  = {9},
  year    = {2017},
  number  = {11},
  article number = {1198},
  url     = {http://www.mdpi.com/2072-4292/9/11/1198},
  issn    = {2072-4292},
  doi     = {10.3390/rs9111198}
}
              

Training Deep Convolution Neural Network with Hard Example Mining for Airport Detection
Bowen Cai, Zhiguo Jiang, Haopeng Zhang, Yuan Yao and Jie Huang
IEEE International Conference on Geoscience and Remote Sensing Symposium (IGARSS), 2017
PDF   Abstract   BibTeX  

The geometrical characteristic and low-level manually designed features are usually used to detect airports in optical remote sensing images. But it is insufficient to describe airport in low resolution and illumination environment. This paper presents a hard example mining algorithm to train the end-to-end deep convolutional neural network for airport detection in complex situation. Compared with conventional airport detection methods which design specific low-level manually designed features for high-resolution remote sensing images, an end-to-end network can mine the general characteristic among the training samples and learn high-level features in multi-scale and multi-view remote sensing images. Meanwhile, an automatic hard example mining principle is introduced to make training more efficiently and accurately. The proposed method is validated on a multi-scale and multi-view dataset collected from Google Earth. The experimental results demonstrate that the proposed method is robust and efficient, and superior to the state-of-the-art airport detection models.

@inproceedings{caiIGARSS17,
  author    = {Bowen Cai and Zhiguo Jiang and Haopeng Zhang and Yuan Yao and Jie Huang},
  booktitle = {2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS)},
  title     = {Training Deep Donvolution Neural Network
               with Hard Example Mining for Airport Detection},
  year      = {2017},
  volume    = {},
  number    = {},
  pages     = {862—-865},
  doi       = {10.1109/IGARSS.2017.8127089},
  issn      = {},
  month     = {July}
}
              

Content-based Histopathological Image Retrieval for Whole Slide Image Database Using Binary Codes
Yushan Zheng, Zhiguo Jiang, Yibing Ma, Haopeng Zhang, Fengying Xie, Huaqiang Shi and Yu Zhao
SPIE Medical Imaging, 2017
PDF   Abstract   BibTeX  

Content-based image retrieval (CBIR) has been widely researched for medical images. In application of histo- pathological images, there are two issues that need to be carefully considered. The one is that the digital slide is stored in a spatially continuous image with a size of more than 10K x 10K pixels. The other is that the size of query image varies in a large range according to different diagnostic conditions. It is a challenging work to retrieve the eligible regions for the query image from the database that consists of whole slide images (WSIs). In this paper, we proposed a CBIR framework for the WSI database and size-scalable query images. Each WSI in the database is encoded and stored in a matrix of binary codes. When retrieving, the query image is first encoded into a set of binary codes and analyzed to pre-choose a set of regions from database using hashing method. Then a multi-binary-code-based similarity measurement based on hamming distance is designed to rank proposal regions. Finally, the top relevant regions and their locations in the WSIs along with the diagnostic information are returned to assist pathologists in diagnoses. The effectiveness of the proposed framework is evaluated in a fine-annotated WSIs database of epithelial breast tumors. The experimental results show that proposed framework is both effective and efficiency for content-based whole slide image retrieval.

@inproceedings{zhengSPIEME17,
  title     = {Content-based Histopathological Image Retrieval
               for Whole Slide Image Database Using Binary Codes},
  author    = {Yushan Zheng and Zhiguo Jiang and Yibing Ma and Haopeng Zhang
               and Fengying Xie and Huaqiang Shi and Yu Zhao},
  booktitle = {SPIE Medical Imaging},
  doi       = {doi.org/10.1117/12.2253988},
  pages     = {1014013},
  year      = {2017}
}
              

Semi-supervised Conditional Random Field for hyperspectral remote sensing image classification
Junfeng Wu, Zhiguo Jiang, Haopeng Zhang, Bowen Cai and Quanmao Wei
IEEE International Conference on Geoscience and Remote Sensing Symposium (IGARSS),2016
Abstract   BibTeX  

Conditional Random Field(CRF) has been successfully applied to the hyperspectral image classification. However, it suffers from the availability of large amount of labeled pixels, which is labor- and time-consuming to obtain in practice. In this paper, a semi-supervised CRF(ssCRF) is proposed for hyperspectral image classification with limited labeled pixels. Laplacian Support Vector Machine(LapSVM), after extended into the composite kernel type, is defined as the association potential. And the Potts model is utilized as the interaction potential. The ssCRF is evaluated on the two benchmarks and the results show the effectiveness of ssCRF.

@inproceedings{wuIGARSS2016,
  title     = {Semi-supervised Conditional Random Field for 
               hyperspectral remote sensing image classification},
  author    = {Junfeng Wu and Zhiguo Jiang and Haopeng Zhang 
               and Bowen Cai and Quanmao Wei},
  booktitle = {2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS)},
  doi       = {10.1109/IGARSS.2016.7729675},
  pages     = {2614-2617},
  year      = {2016}
}
              

Multi-scale orderless cross-regions-pooling of deep attributes for image retrieval
Jianwei Luo, Zhiguo Jiang and Jianguo Li
Electronics Letters,2016
Abstract   BibTeX  

How to represent an image is an essential problem of the image retrieval task. To build a powerful image representation, a novel method named cross-regions-pooling (CRP) combining two key ingredients is proposed: (i) region proposals detected by objectness detection technique; (ii) deep attributes (DA), i.e. the outputs of the softmax layer of off-the-shelf convolutional neural network pre-trained on a large-scale dataset. The ultimate representation of an image is the aggregation (e.g. max-pooling) of DA extracted from all the regions. In addition, a multi-scale orderless pooling strategy considering layout of contexts of an image is proposed to integrate with CRP to improve the image representation. Experimental results on standard benchmarks demonstrate superiority of the proposed method over state-of-the-arts.

@article{Luo2016Multi,
  title={Multi-scale orderless cross-regions-pooling of deep attributes for image retrieval},
  author={Luo, Jianwei and Jiang, Zhiguo and Li, Jianguo},
  journal={Electronics Letters},
  volume={52},
  number={4},
  pages={276-277},
  year={2016},
}

              

Breast Histopathological Image Retrieval Based on Latent Dirichlet Allocation
Yibing Ma, Zhiguo Jiang, Haopeng Zhang, Fengying Xie, Yushan Zheng, Huaqiang Shi and Yu Zhao
IEEE Journal of Biomedical and Health Informatics, 2016
Abstract   BibTeX  

In the field of pathology, whole slide image (WSI) has become the major carrier of visual and diagnostic information. Content-based image retrieval among WSIs can aid the diagnosis of an unknown pathological image by finding its similar regions in WSIs with diagnostic information. However, the huge size and complex content of WSI pose several challenges for retrieval. In this paper, we propose an unsupervised, accurate, and fast retrieval method for a breast histopathological image. Specifically, the method presents a local statistical feature of nuclei for morphology and distribution of nuclei, and employs the Gabor feature to describe the texture information. The latent Dirichlet allocation model is utilized for high-level semantic mining. Locality-sensitive hashing is used to speed up the search. Experiments on a WSI database with more than 8000 images from 15 types of breast histopathology demonstrate that our method achieves about 0.9 retrieval precision as well as promising efficiency. Based on the proposed framework, we are developing a search engine for an online digital slide browsing and retrieval platform, which can be applied in computer-aided diagnosis, pathology education, and WSI archiving and management.

@article{maJBHI2016, 
  author  = {Yibing Ma and Zhiguo Jiang and Haopeng Zhang and Fengying Xie 
             and Yushan Zheng and Huaqiang Shi and Yu Zhao and Jun Shi}, 
  journal = {IEEE Journal of Biomedical and Health Informatics}, 
  title   = {Breast Histopathological Image Retrieval Based on Latent Dirichlet Allocation}, 
  year    = {2017}, 
  volume  = {21}, 
  number  = {4}, 
  pages   = {1114-1123}, 
  doi     = {10.1109/JBHI.2016.2611615}, 
  ISSN    = {2168-2194}, 
  month   = {July}
}    
              

Pose Estimation of Space Objects Based on Hybrid Feature Matching of Contour Points
Xin Zhang, Haopeng Zhang, Quanmao Wei and Zhiguo Jiang
Image and Graphics Technologies and Applications (IGTA), 2016
PDF   Abstract   BibTeX  

This paper presents an improved pose estimation algorithm for vision-based space objects. The major weakness of most existing methods is limited convergence radius. In most cases they ignore the influence of translation, only focusing on rotation parameters. To breakthrough these limits, we utilizes hybrid local image features to explicitly establish 2D-3D correspondences between the input image and 3D model of space objects, and then estimate rotation and translation parameters based on the correspondences. Experiments with simulated models are carried out, and the results show that our algorithm can successfully estimate the pose of space objects with large convergence radius and high accuracy.

@inproceedings{zhangIGTA16,
  author    = {Xin Zhang and Haopeng Zhang and Quanmao Wei and Zhiguo Jiang},
  editor    = {Tieniu Tan and Guoping Wang and Shengjin Wang and Yue Liu
               and Xiaoru Yuan and Ran He and Sheng Li},
  title     = {Pose Estimation of Space Objects Based on Hybrid Feature Matching
               of Contour Points},
  booktitle = {Advances in Image and Graphics Technologies},
  year      = {2016},
  publisher = {Springer Singapore},
  address   = {Singapore},
  pages     = {184—-191},
  isbn      = {978-981-10-2260-9}
}
              

Sequential-image-based Space Object 3D Reconstruction
Haopeng Zhang, Quanmao Wei, Wei Zhang, Junfeng Wu and Zhiguo Jiang
Journal of BUAA, 2016 [In Chinese]
PDF   Abstract   BibTeX  

Space object 3D reconstruction is of important significance for both space situational awareness and theoretical study. A new structure from motion method was proposed to avoid the reconstruction error caused by the symmetrical structure and similar texture of space targets. In this method, new images were added sequentially for reconstruction using the imaging time as a priori knowledge. In addition, image simulation of space target and ground imaging simulation experiment were carried out for the lack of space target image data. And experiments on the simulated space target images, in which the motion analysis results are accurate and robust to noise, and the recovery 3D point cloud can express the structural information of the target to a certain extent, have demonstrated the effectiveness of the approach proposed, and the boundary conditions of multi-frame-image for 3D reconstruction are acquired as well.

@article{zhangJBUAA16,
  language = {Chinse},
  author   = {Haopeng Zhang and Quanmao Wei and Wei Zhang and Junfeng Wu and Zhiguo Jiang},
  title    = {Sequential-image-based Space Object 3D Reconstruction},
  journal  = {Journal of Beijing University of Aeronautics and Astronautics},
  year     = {2016},
  volume   = {42},
  number   = {2},
  pages    = {273--279},
  issn     = {10015965},
  URL      = {http://dx.doi.org/10.13700/j.bh.1001-5965.2015.0117}
}
              

High-resolution Optical Satellite Image Simulation of Ship Target in Large Sea Scenes
Yuan Yao, Zhiguo Jiang and Haopeng Zhang
IEEE International Conference on Geoscience and Remote Sensing Symposium (IGARSS), 2016
PDF   Abstract   BibTeX  

Ship target detection in optical remote sensing images has attracted more and more attention in the field of remote sensing. The ship target detection technology of optical remote sensing images is vulnerable to many factors, while the real data are difficult to contain various elements. In order to obtain the various situations in the large sea scenes, we develop a simulation system for high-resolution optical remote sensing image of ship targets. The simulated images with different sea states, cloud conditions, target types and imaging conditions can support the evaluation and comparison of ship detection algorithms as well as other tasks in remote sensing image analysis.

@inproceedings{yaoIGARSS16,
  author    = {Yuan Yao and Zhiguo Jiang and Haopeng Zhang},
  booktitle = {2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS)},
  title     = {High-resolution Optical Satellite Image Simulation of Ship Target
               in Large Sea Scenes},
  year      = {2016},
  month     = {July},
  volume    = {},
  number    = {},
  pages     = {1241—-1244},
  doi       = {10.1109/IGARSS.2016.7729314}
}
              

Satellite recognition and pose estimation using homeomorphic manifold analysis
Haopeng Zhang, Zhiguo Jiang, Elgammal A
IEEE Transactions on Aerospace and Electronic Systems, 2015
Abstract   BibTeX  

We propose a novel monocular vision-based framework for both satellite recognition and pose estimation, using homeomorphic manifold analysis. We use a unified conceptual manifold to represent continuous pose variation of all satellites in the visual input space, learn nonlinear function mapping from conceptual manifold representation to visual inputs, and decompose discrete category variation in the mapping coefficient space. Experimental results on a simulated image data set show the effectiveness and robustness of our approach.

@article{HaopengTAES2015,
  title   = {Satellite recognition and pose estimation using homeomorphic manifold analysis},
  author  = {Haopeng Zhang, Zhiguo Jiang, Elgammal A},
  journal = {IEEE Transactions on Aerospace and Electronic Systems},
  pages   = {785 - 792},
  year    = {2015},
  issn    = {0018-9251},
  url     = {https://ieeexplore.ieee.org/document/7073533/},
}      
              

Factorization of View-Object Manifolds for Joint Object Recognition and Pose Estimation
Haopeng Zhang, El-Gaaly T, Elgammal A, Zhiguo Jiang
Computer Vision and Image Understanding, 2015
Abstract   BibTeX  

Due to large variations in shape, appearance, and viewing conditions, object recognition is a key precursory challenge in the fields of object manipulation and robotic/AI visual reasoning in general. Recognizing object categories, particular instances of objects and viewpoints/poses of objects are three critical subproblems robots must solve in order to accurately grasp/manipulate objects and reason about their environments. Multi-view images of the same object lie on intrinsic low-dimensional manifolds in descriptor spaces (e.g. visual/depth descriptor spaces). These object manifolds share the same topology despite being geometrically different. Each object manifold can be represented as a deformed version of a unified manifold. The object manifolds can thus be parameterized by its homeomorphic mapping/reconstruction from the unified manifold. In this work, we develop a novel framework to jointly solve the three challenging recognition sub-problems, by explicitly modeling the deformations of object manifolds and factorizing it in a view-invariant space for recognition. We perform extensive experiments on several challenging datasets and achieve state-of-the-art results.

@article{HaopengCVIU2015,
  title   = {Factorization of View-Object Manifolds for Joint Object Recognition and Pose Estimation},
  author  = {Haopeng Zhang, El-Gaaly T, Elgammal A, Zhiguo Jiang},
  journal = {Computer Vision and Image Understanding},
  volume  = {139},
  pages   = {89 - 103},
  year    = {2015}
  url     = {https://www.sciencedirect.com/science/article/pii/S1077314215000715?via%3Dihub},
}      
              

Vision-based pose estimation for space objects by Gaussian process regression
Haopeng Zhang, Zhiguo Jiang, Yuan Yao, et al
IEEE Aerospace Conference, 2015
Abstract   BibTeX  

We address the problem of vision-based pose estimation for space objects, which is to estimate the relative pose of a target spacecraft using imaging sensors. We develop a novel monocular vision-based method by employing Gaussian process regression (GPR) to solve pose estimation for space objects. GPR is a powerful regression model for predicting continuous quantities, and can easily obtain and express uncertainty. Assuming that the regression function mapping from the image (or feature) of the target spacecraft to its pose follows a Gaussian process (GP) properly parameterized by a mean function and a covariance function, the predictive equations can be easily obtained by a maximum-likelihood approach when training data are given. The mean value of the predicted output (i.e. the estimated pose) and its variance (which indicates the uncertainty) can be computed via these explicit formulations. Besides, we also introduce a manifold constraint to the output of GPR model to improve its performance for spacecraft pose estimation. We performed extensive experiments on a simulated image dataset that contains satellite images of 1D and 2D pose variation, as well as images with noises and different lighting conditions. Experimental results validate the effectiveness and robustness of our approach. Our model can not only estimate the pose angles of space objects but also provide the uncertainty of the estimated values which may be used to choose convincing results in applications.

@article{HaopengAC2015,
title   = {Vision-based pose estimation for space objects by Gaussian process regression},
author  = {Haopeng Zhang, Zhiguo Jiang, Yuan Yao, et al},
journal = {Aerospace Conference, 2015 IEEE},
pages   = {1 - 9},
year    = {2015},
doi     = {10.1109/AERO.2015.7118908},
issn    = {1095-323X}
url     = {https://ieeexplore.ieee.org/document/7118908/},
}      
            

Composite kernels conditional random fields for remote-sensing image classification
Junfeng Wu，Zhiguo Jiang, Jianwei Luo and Haopeng Zhang
Electronic Letters, 2014
Abstract   BibTeX  

The problem of classifying a remote-sensing image by specifically labelling each pixel in the image is addressed. A novel method, named composite kernels conditional random field (CKCRF), which embeds multiple kernels into a classical CRFs model is proposed. Rather than manually selecting kernel-like KCRF, CKCRFs chooses the appropriate kernel by training. Moreover, a genetic programming-based decision-level fusion framework is proposed to tackle the problem of feature selection. It can select the appropriate features suitable to each category. Evaluations show that CKCRFs outperform CRFs and KCRFs, and CKCRFs with the fusion scheme is better than that without the fusion step.

  @inproceedings{wuElectron Lett2014,
    title     = {Composite kernels conditional random fields for remote-sensing image classification},
    author    = {Junfeng Wu and Zhiguo Jiang and Jianwei Luo and Haopeng Zhang},
    booktitle = {2014 ELECTRONICS LETTERS(Electron Lett)},
    doi       = {10.1049/el.2014.1964},
    pages     = {1589-1590},
    year      = {2014}
  }
                

Learning Semantic Binary Codes by Encoding Attributes for Image Retrieval
Jianwei Luo and Zhiguo Jiang
International Conference on Pattern Recognition ,2014
Abstract   BibTeX  

This paper addresses the problem of learning semantic compact binary codes for efficient retrieval in large-scale image collections. Our contributions are three-fold. Firstly, we introduce semantic codes, of which each bit corresponds to an attribute that describes a property of an object (e.g. dogs have furry). Secondly, we propose to use matrix factorization (MF) to learn the semantic codes by encoding attributes. Unlike traditional PCA-based encoding methods which quantize data into orthogonal bases, MF assumes no constraints on bases, and this scheme is coincided with that attributes are correlated. Finally, to augment semantic codes, MF is extended to encode extra non-semantic codes to preserve similarity in origin data space. Evaluations on a-Pascal dataset show that our method is comparable to the state-of-the-art when using Euclidean distance as ground truth, and even outperforms state-of-the-art when using class label as ground truth. Furthermore, in experiments, our method can retrieve images that share the same semantic properties with the query image, which can be used to other vision tasks, e.g. re-training classifiers.

@inproceedings{Luo2014Learning,
  title={Learning Semantic Binary Codes by Encoding Attributes for Image Retrieval},
  author={Luo, Jianwei and Jiang, Zhiguo},
  booktitle={International Conference on Pattern Recognition},
  pages={279-284},
  year={2014},
}
            

Multi-view space object recognition and pose estimation based on kernel regression
Haopeng Zhang, Zhiguo Jiang
Chinese Journal of Aeronautics, 2014
Abstract   BibTeX  

We address the problem of vision-based pose estimation for space objects, which is to estimate the relative pose of a target spacecraft using imaging sensors. We develop a novel monocular vision-based method by employing Gaussian process regression (GPR) to solve pose estimation for space objects. GPR is a powerful regression model for predicting continuous quantities, and can easily obtain and express uncertainty. Assuming that the regression function mapping from the image (or feature) of the target spacecraft to its pose follows a Gaussian process (GP) properly parameterized by a mean function and a covariance function, the predictive equations can be easily obtained by a maximum-likelihood approach when training data are given. The mean value of the predicted output (i.e. the estimated pose) and its variance (which indicates the uncertainty) can be computed via these explicit formulations. Besides, we also introduce a manifold constraint to the output of GPR model to improve its performance for spacecraft pose estimation. We performed extensive experiments on a simulated image dataset that contains satellite images of 1D and 2D pose variation, as well as images with noises and different lighting conditions. Experimental results validate the effectiveness and robustness of our approach. Our model can not only estimate the pose angles of space objects but also provide the uncertainty of the estimated values which may be used to choose convincing results in applications.

    @article{HaopengCJA2014,
    title   = {Multi-view space object recognition and pose estimation based on kernel regression},
    author  = {Haopeng Zhang, Zhiguo Jiang},
    journal = {Chinese Journal of Aeronautics},
    pages   = {1233 - 1241},
    year    = {2014},
    volume  = {27},
    url     = {https://www.sciencedirect.com/science/article/pii/S1000936114000533},
    }      
                

Manifold representation of multi-view images
Haopeng Zhang, Zhiguo Jiang
Journal of Computational Information Systems,2014
Abstract   BibTeX  

Abstract Images of the same object lie on a low-dimensional manifold (view manifold) in the visual space. View manifolds can be used to represent viewpoint variation of multi-view images in the embedding space, and can be very helpful to multi-view object detection, classification, and viewpoint estimation. In this paper, we introduce a conceptual manifold as a common representation of all view manifolds. In order to evaluate the performance of the conceptual manifold representation, we learn a generative model that can map from the manifold representation to visual inputs for the tasks of arbitrary view image synthesis and viewpoint estimation. We did experiments on COIL-20 dataset, and compared with popular manifold learning methods. Experimental results show that our conceptual manifold representation can effectively describe the viewpoint variation of multi-view images with strong robustness, and outperform the view manifolds learned by popular manifold learning methods.

    @article{HaopengJCIS2014,
    title   = {Manifold representation of multi-view images},
    author  = {Haopeng Zhang, Zhiguo Jiang},
    journal = {Journal of Computational Information Systems},
    pages   = {4867 - 4876},
    year    = {2014},
    doi     = {10.12733/jcis10624},
    url     = {https://www.researchgate.net/publication/288287272_Manifold_representation_of_multi-view_images},
    }      
                

Constrained kernel regression for pose estimation
Haopeng Zhang, Zhiguo Jiang
Electronic letters, 2014
Abstract   BibTeX  

A constrained kernel regression model is proposed to solve the problem of one-dimensional (1D) pose estimation. Unlike the traditional kernel regression model, a circular constraint is applied to the output of the regression function, i.e. using 2D coordinates on a unit circle as output instead of 1D pose angles from 0 to 360°. The experimental results show that with this constraint, the performance of kernel regression on the 1D pose estimation can be improved significantly, and the constrained kernel regression model can run in real-time.

    @article{HaopengEL2014,
    title   = {Constrained kernel regression for pose estimation},
    author  = {Haopeng Zhang, Zhiguo Jiang},
    journal = {Electronics Letters},
    pages   = {77 - 79},
    year    = {2014},
    issn    = {0013-5194},
    doi     = { 10.1049/el.2013.2071},
    url     = {https://ieeexplore.ieee.org/document/6729319/?arnumber=6729319&tag=1},
    }      
                

Retrieval of Pathology Image for Breast Cancer Using PLSA Model Based on Texture and Pahological Features
Yushan Zheng, Zhiguo Jiang, Jun Shi and Yibing Ma
IEEE International Conference on Image Processing (ICIP), 2014
PDF   Abstract   BibTeX  

Content-based image retrieval (CBIR) for digital pathology slides is of clinical use for breast cancer aided diagnosis. One of the largest challenges in CBIR is feature extraction. In this paper, we propose a novel pathology image retrieval method for breast cancer, which aims to characterize the pathology image content through texture and pathological features and further discover the latent high-level semantics. Specifically, the proposed method utilizes block Gabor features to describe the texture structure, and simultaneously designs nucleus-based pathological features to describe morphological characteristics of nuclei. Based on these two kinds of local feature descriptors, two codebooks are built to learn the probabilistic latent semantic analysis (pLSA) models. Consequently, each image is represented by the topics of pLSA models which can reveal the semantic concepts. Experimental results on the digital pathology image database for breast cancer demonstrate the feasibility and effectiveness of our method.

@article{zhengICIP14,
  title     = {Retrieval of Pathology Image for Breast Cancer Using PLSA Model
               Based on Texture and Pathological Features},
  author    = {Yushan Zheng and Zhiguo Jiang and Jun Shi and Yibing Ma},
  booktitle = {2016 IEEE International Conference on Image Processing (ICIP)},
  pages     = {2304--2308},
  year      = {2014}
}
              

Pathology Image Retrieval by Block LBP Based PLSA Model with Low-Rank and Sparse Matrix Decomposition
Yushan Zheng, Zhiguo Jiang, Jun Shi and Yibing Ma
Image and Graphics Technologies and Applications (IGTA), 2014
PDF   Abstract   BibTeX  

Content-based image retrieval (CBIR) is widely used in Computer Aided Diagnosis (CAD) systems which can aid pathologist to make reasonable decision by querying the slides with diagnostic information from the digital pathology slide database. In this paper, we propose a novel pathology image retrieval method for breast cancer. It firstly applies block Local Binary Pattern (LBP) features to describe the spatial texture property of pathology image, and then use them to construct the probabilistic latent semantic analysis (pLSA) model which generally takes advantage of visual words to mine the topic-level representation of image and thus reveals the high-level semantics. Different from conventional pLSA model, we employ low-rank and sparse matrix composition for describing the correlated and specific characteristics of visual words. Therefore, the more discriminative topic-level representation corresponding to each pathology image can be obtained. Experimental results on the digital pathology image database for breast cancer demonstrate the feasibility and effectiveness of our method.

@inproceedings{zhengIGTA14,
  title     = {Pathology Image Retrieval by Block LBP Based PLSA Model
               with Low-Rank and Sparse Matrix Decomposition},
  author    = {Yushan Zheng and Zhiguo Jiang and Jun Shi and Yibing Ma},
  booktitle = {Advances in Image and Graphics Technologies},
  pages     = {327—-335},
  year      = {2014}
}
              

Regularized least square discriminant projection and feature selection
Jun Shi, Zhiguo Jiang, Danpei Zhao, Hao Feng, Chao Gao
Journal of Electronic Imaging, 2014
Abstract   BibTeX  

Conventional graph embedding framework uses the Euclidean distance to determine the similarities of neighbor samples, which causes the graph structure to be sensitive to outliers and lack physical interpretation. Moreover, the graph construction suffers from the difficulty of neighbor parameter selection. Although sparse representation (SR) based graph embedding methods can automatically select the neighbor parameter, the computational cost of SR is expensive. On the other hand, most discriminant projection methods fail to perform feature selection. In this paper, we present a novel joint discriminant analysis and feature selection method that employs regularized least square for graph construction and l 2; 1-norm minimization on projection matrix for feature selection. Specifically, our method first uses the regularized least square coefficients to measure the intraclass and interclass similarities from the viewpoint of reconstruction. Based on this graph structure, we formulate an object function with scatter difference criterion for learning the discriminant projections, which can avoid the small sample size problem. Simultaneously, the l2; 1-norm minimization on projection matrix is applied to gain row-sparsity for selecting useful features. Experiments on two face databases (ORL and AR) and COIL-20 object database demonstrate that our method not only achieves better classification performance, but also has lower computational cost than SR. (C) 2014 SPIE and IS&T

@article{shiJEI2014,
  author  = {Jun Shi and Zhiguo Jiang and Danpei Zhao and Hao Feng and Chao Gao},
  title   = {Regularized least square discriminant projection and feature selection},
  journal = {Journal of Electronic Imaging},
  volume  = {23},
  pages   = {23 - 23 - 15},
  year    = {2014},
  doi     = {10.1117/1.JEI.23.1.013003},
  URL     = {https://doi.org/10.1117/1.JEI.23.1.013003},
}             
              

Adaptive Graph Embedding Discriminant Projections
Jun Shi, Zhiguo Jiang and Hao Feng
Neural Processing Letters, 2014
Abstract   BibTeX  

Graph embedding based learning method plays an increasingly significant role on dimensionality reduction (DR). However, the selection to neighbor parameters of graph is intractable. In this paper, we present a novel DR method called adaptive graph embedding discriminant projections (AGEDP). Compared with most existing DR methods based on graph embedding, such as marginal Fisher analysis which usually predefines the intraclass and interclass neighbor parameters, AGEDP applies all the homogeneous samples for constructing the intrinsic graph, and simultaneously selects heterogeneous samples within the neighborhood generated by the farthest homogeneous sample for constructing the penalty graph. Therefore, AGEDP not only greatly enhances the intraclass compactness and interclass separability, but also adaptively performs neighbor parameter selection which considers the fact that local manifold structure of each sample is generally different. Experiments on AR and COIL-20 datasets demonstrate the effectiveness of the proposed method for face recognition and object categorization, and especially under the interference of occlusion, noise and poses, it is superior to other graph embedding based methods with three different classifiers: nearest neighbor classifier, sparse representation classifier and linear regression classifier.

@Article{ShiNPL2014,
  author  = {Jun Shi and Zhiguo Jiang and Hao Feng},
  title   = {Adaptive Graph Embedding Discriminant Projections},
  journal = {Neural Processing Letters},
  year    = {2014},
  volume  = {40},
  number  = {3},
  pages   = {211--226},
  doi     = {10.1007/s11063-013-9323-8},
}     
              

Shadow boundaries identification in single natural images via multiple kernels learning
Junfeng Wu,Zhiguo Jiang,Junli Yang and Jianwei Luo
International Conference on Image and Graphics (ICIG),2013
Abstract   BibTeX  

The identification of shadow and shading boundaries is a key step towards reducing the imaging effects that are caused by direct illumination of the light source in the scene. Discriminating shadow boundaries from images of natural scenes has been widely applied in the field of computer vision such as object recognition, intelligent monitoring and image understanding. In this paper, we propose a method to identify shadow boundaries based on multiple kernel learning. We first extract all possible candidate boundaries and then analyze their properties. Unlike the previous proposed methods which simply combine features as a vector, we choose the optimal kernel function for every feature and learn the correct weights of different features from training database. At last, we link shadow boundaries fragments together to get longer and complete shadow boundaries. The experiment results show that the method we propose works well in shadow boundaries identification.

  @inproceedings{wuICIG2013,
    title     = {Shadow boundaries identification in single natural images via multiple kernels learning},
    author    = {Junfeng Wu and Zhiguo Jiang and Junli Yang and Jianwei Luo },
    booktitle = {2013 Image and Graphics (ICIG)},
    doi       = {10.1109/ICIG.2013.75},
    pages     = {348-352},
    year      = {2013}
  }
                

Joint Object and Pose Recognition Using Homeomorphic Manifold Analysis
Haopeng Zhang, Tarek El-Gaaly, Ahmed Elgammal, Zhiguo Jiang
AAAI Conference on Artificial Intelligence, 2013
Abstract   BibTeX  

Object recognition is a key precursory challenge in the fields of object manipulation and robotic/AI visual reasoning in general. Recognizing object categories, particular instances of objects and viewpoints/poses of objects are three critical subproblems robots must solve in order to accurately grasp/manipulate objects and reason about their environments. Multi-view images of the same object lie on intrinsic low-dimensional manifolds in descriptor spaces (e.g. visual/ depth descriptor spaces). These object manifolds share the same topology despite being geometrically different. Each object manifold can be represented as a deformed version of a unified manifold. The object manifolds can thus be parametrized by its homeomorphic mapping/reconstruction from the unified manifold. In this work, we construct a manifold descriptor from this mapping between homeomorphic manifolds and use it to jointly solve the three challenging recognition sub-problems. We extensively experiment on a challenging multi-modal (i.e. RGBD) dataset and other object pose datasets and achieve state-of-the-art results.

    @article{HaopengAAAI2013,
    title      = {Joint Object and Pose Recognition Using Homeomorphic Manifold Analysis},
    author     = {Haopeng Zhang, Tarek El-Gaaly, Ahmed Elgammal, Zhiguo Jiang},
    conference = {The 27th AAAI Conference on Artificial Intelligence},
    pages      = {1012-1019},
    year       = {2013},
    url        = {https://dl.acm.org/citation.cfm?id=2891601},
    }      
                

Vision-based pose estimation for cooperative space objects
Haopeng Zhang, higuo Jiang, Ahmed Elgammal
Acta Astronautica, 2013
Abstract   BibTeX  

Imaging sensors are widely used in aerospace recently. In this paper, a vision-based approach for estimating the pose of cooperative space objects is proposed. We learn generative model for each space object based on homeomorphic manifold analysis. Conceptual manifold is used to represent pose variation of captured images of the object in visual space, and nonlinear functions mapping between conceptual manifold representation and visual inputs are learned. Given such learned model, we estimate the pose of a new image by minimizing a reconstruction error via a traversal procedure along the conceptual manifold. Experimental results on the simulated image dataset show that our approach is effective for 1D and 2D pose estimation.

    @article{HaopengAA2013,
    title      = {Vision-based pose estimation for cooperative space objects},
    author     = {Haopeng Zhang, higuo Jiang, Ahmed Elgammal},
    Journal    = {Acta Astronautica},
    pages      = {115-122},
    year       = {2013},
    url        = {https://www.sciencedirect.com/science/article/pii/S0094576513001781},
    }      
                

Unsupervised texture segmentation based on latent topic assignment
Hao Feng, Zhiguo Jiang and Jun Shi
Journal of Electronic Imaging, 2013
Abstract   BibTeX  

We present an effective solution for unsupervised texture segmentation by taking advantage of the latent Dirichlet allocation (LDA) model. LDA is a generative topic model that is capable of hierarchically organizing discrete data including texts and images. We propose a new texture model by connecting texture primitives to the topic of LDA. The model is able to extract the characteristic features of a texture primitive and group them into a topic based on their frequencies of co-occurrence. Here, the feature descriptor is the connection of Haar-like features of multiple sizes. The segments of an image are finally obtained by identifying the homogeneous regions in the corresponding topic assignment map. The evaluation results for synthetic texture mosaics, remote sensing images, and natural scene images are illustrated.

@article{fengJEI2013,
  author  = {Hao  Feng and Zhiguo Jiang and Jun  Shi},
  title   = {Unsupervised texture segmentation based on latent topic assignment},
  journal = {Journal of Electronic Imaging},
  volume  = {22},
  number  = {1},
  pages   = {13-26},
  year    = {2013},
  doi     = {10.1117/1.JEI.22.1.013026},
  URL     = {https://doi.org/10.1117/1.JEI.22.1.013026},
}
              

Pathological Image Retrieval for Breast Cancer with pLSA Model
Jun Shi, Yibing Ma, Zhiguo Jiang, Hao Feng, Jin Chen and Yu Zhao
International Conference on Image and Graphics (ICIG), 2013
Abstract   BibTeX  

Pathological image retrieval contributes to computer-aided diagnosis for breast cancer due to the fact that the retrieval results generally contain detailed diagnostic information (e.g. abnormal regions and diagnostic opinion from other doctors) which can offer some reference and assistance to the doctor during diagnosis process. In this paper, we present a novel pathological image retrieval approach based on probabilistic latent semantic analysis (pLSA) model. The method respectively utilizes SIFT features after visual saliency detection, and block Gabor features for the construction of two semantic codebooks, which not only can characterize the salient local invariant features and texture information under different scales and orientations in the pathological images, but also consider the high-level semantic features. Furthermore, we apply pLSA model to discover the latent topics in each codebook. Finally each pathological image is represented by the combination of topics from these two codebooks. The proposed method is evaluated on the pathological image database for breast cancer, which includes 5 categories (mucinous cystadenocarcinoma, invasive lobular carcinoma, basal-like carcinoma, invasive breast cancer and low-grade adenosquamous carcinoma) and 110 cases for each category. Experimental results demonstrate the feasibility and effectiveness of our method.

@inproceedings{shiICIG2013, 
  author    = {Jun Shi and Yibing Ma and Zhiguo Jiang and Hao Feng and Jin Chen and Yu Zhao}, 
  booktitle = {2013 Seventh International Conference on Image and Graphics}, 
  title     = {Pathological Image Retrieval for Breast Cancer with pLSA Model}, 
  year      = {2013}, 
  pages     = {634-638}, 
  doi       = {10.1109/ICIG.2013.131},
  month     = {July}
}
              

PLSA-based pathological image retrieval for breast cancer with color deconvolution
Yibing Ma, Jun Shi, Zhiguo Jiang and Hao Feng
International Symposium on Multispectral Image Processing and Pattern Recognition (MIPPR), 2013
Abstract   BibTeX  

Digital pathological image retrieval plays an important role in computer-aided diagnosis for breast cancer. The retrieval results of an unknown pathological image, which are generally previous cases with diagnostic information, can provide doctors with assistance and reference. In this paper, we develop a novel pathological image retrieval method for breast cancer, which is based on stain component and probabilistic latent semantic analysis (pLSA) model. Specifically, the method firstly utilizes color deconvolution to gain the representation of different stain components for cell nuclei and cytoplasm, and then block Gabor features are conducted on cell nuclei, which is used to construct the codebook. Furthermore, the connection between the words of the codebook and the latent topics among images are modeled by pLSA. Therefore, each image can be represented by the topics and also the high-level semantic concepts of image can be described. Experiments on the pathological image database for breast cancer demonstrate the effectiveness of our method.

@proceeding{maMIPPR2013,
  author  = {Yibing Ma and Jun Shi and Zhiguo Jiang and Hao Feng},
  title   = {PLSA-based pathological image retrieval for breast cancer with color deconvolution},
  journal = {Proc.SPIE},
  volume  = {8920},
  pages   = {8920-7},
  year    = {2013},
  doi     = {10.1117/12.2032054},
  URL     = {https://doi.org/10.1117/12.2032054},
}          
              

Sparse coding-based topic model for remote sensing image segmentation
Jun Shi, Zhiguo Jiang, Hao Feng and Yibing Ma
IEEE International Conference on Geoscience and Remote Sensing Symposium (IGARSS), 2013
Abstract   BibTeX  

Land cover segmentation can be viewed as topic assignment that the pixels are grouped into homogeneous regions according to different semantic topics in topic model. In this paper, we propose a novel topic model based on sparse coding for segmenting different kinds of land covers. Different from conventional topic models which generally assume each local feature descriptor is related to only one visual word of the codebook, our method utilizes sparse coding to characterize the potential correlation between the descriptor and multiple words. Therefore each descriptor can be represented by a small set of words. Furthermore, in this paper probabilistic Latent Semantic Analysis (pLSA) is applied to learn the latent relation among word, topic and document due to its simplicity and low computational cost. Experimental results on remote sensing image segmentation demonstrate the excellent superiority of our method over k-means clustering and conventional pLSA model.

@inproceedings{shiIGARSS2013, 
  author    = {Jun Shi and Zhiguo Jiang and Hao Feng and Yibing Ma}, 
  booktitle = {2013 IEEE International Geoscience and Remote Sensing Symposium - IGARSS}, 
  title     = {Sparse coding-based topic model for remote sensing image segmentation}, 
  year      = {2013}, 
  pages     = {4122-4125}, 
  doi       = {10.1109/IGARSS.2013.6723740}, 
  ISSN      = {2153-6996}, 
  month     = {July}
}
              

Space object, high-resolution, optical imaging simulation of space-based systems
Haopeng Zhang; Wei Zhang; Zhiguo Jiang
SPIE Defense, Security, and Sensing, 2012
Abstract   BibTeX  

Acquiring optical images of space objects is one of the most important goals of space-based optical surveillance systems. However, it's actually difficult to obtain enough high resolution optical images for space object recognition, attitude measurement and situational awareness. To solve this problem, the imaging model of space-based optical camera and the imaging characteristics of space objects are analyzed in this paper, and a novel method of image simulation is proposed. The high resolution images of space objects simulated by our method are visually similar to the actual imaging results and may provide data support for further research on space technology.

    @article{HaopengSPIE2013,
    title      = {Space object, high-resolution, optical imaging simulation of space-based systems},
    author     = {Haopeng Zhang; Wei Zhang; Zhiguo Jiang},
    booktitle  = {SPIE Defense, Security, and Sensing},
    pages      = {7},
    year       = {2012},
    url        = {https://www.spiedigitallibrary.org/conference-proceedings-of-spie/8385/1/Space-object-high-resolution-optical-imaging-simulation-of-space-based/10.1117/12.918368.full},
    }      
                

SIFT-based Elastic sparse coding for image retrieval
Jun Shi, Zhiguo Jiang, Hao Feng and Liguo Zhang
IEEE International Conference on Image Processing (ICIP), 2012
Abstract   BibTeX  

Bag-of-features (BoF) model based on SIFT generally assumes each descriptor is related to only one visual word of the codebook. Therefore, the potential correlation between the descriptor and other visual words is ignored. On the other hand, sparse coding through l1-norm regularization fails to generate optimal sparse representations since l1-norm regularization randomly selected one variable from a group of highly correlated variables. In this study we propose a novel bag-of-features model for image retrieval called SIFT-based Elastic sparse coding. The method utilizes a large number of SIFT descriptors to construct the codebook. The Elastic Net regression framework, which combines both l1-norm and l2-norm penalties, is then used to obtain the sparse-coefficient vector corresponding to the SIFT descriptor. Finally each image can be represented by a unified sparse-coefficient vector. Experimental results on Coil20 dataset demonstrate the consistent superiority of the proposed method over the state-of-the-art algorithms including original SIFT matching, conventional BoF strategy and BoF model based on l1-norm sparse coding.

@inproceedings{shi2012ICIP, 
  author    = {Jun Shi and Zhiguo Jiang and Hao Feng and Liguo Zhang}, 
  booktitle = {2012 19th IEEE International Conference on Image Processing}, 
  title     = {SIFT-based Elastic sparse coding for image retrieval}, 
  year      = {2012},
  pages     = {2437-2440}, 
  doi       = {10.1109/ICIP.2012.6467390}, 
  ISSN      = {1522-4880}, 
  month     = {Sep}
}
              

Image segmentation based on pixel feature manifold
Haopeng Zhang; Zhiguo Jiang
International Symposium on Multispectral Image Processing and Pattern Recognition (MIPPR), 2011
Abstract   BibTeX  

Image segmentation is an important problem in pattern recognition, computer vision and other related area, which is still a research focus. In this paper, we consider the segmentation as pixel classification scheme and introduce a manifold way to address this problem. Some local features, such as Haar, LBP and SIFT, are used to represent each pixel in the image together with the basic property of the pixel. We put these pixel features on a manifold called pixel feature manifold (PFM) obtained via manifold learning methods and classify pixels with k-NN classifier in the pixel embedding space. Experimental results on MSRC image dataset show that our PFM method can effectively segment images.

@inproceedings{Zhang2011Image,
  title={Image Segmentation Based on Pixel Feature Manifold},
  author={Zhang, Haopeng and Jiang, Zhiguo and Zhang, Wei and Zhao, Danpei},
  booktitle={SPIE MIPPR},
  pages={800307},
  year={2011},
}    
              

Full-viewpoint 3D space object recognition based on kernel locality preserving projections
Gang Meng, Zhiguo Jiang, Zhengyi Liu, Haopeng Zhang and Danpei Zhao
Chinese Journal of Aeronautics, 2010
Abstract   BibTeX  

Space object recognition plays an important role in spatial exploitation and surveillance,followed by two main problems:lacking of data and drastic changes in viewpoints.In this article,firstly,we build a three-dimensional (3D) satellites dataset named BUAA Satellite Image Dataset (BUAA-SID 1.0) to supply data for 3D space object research.Then,based on the dataset,we propose to recognize full-viewpoint 3D space objects based on kemel locality preserving projections (KLPP).To obtain more accurate and separable description of the objects,firstly,we build feature vectors employing moment invariants,Fourier descriptors,region covariance and histogram of oriented gradients.Then,we map the features into kernel space followed by dimensionality reduction using KLPP to obtain the submanifold of the features.At last,k-nearest neighbor (kNN) is used to accomplish the classification.Experimental results show that the proposed approach is more appropriate for space object recognition mainly considering changes of viewpoints.Encouraging recognition rate could be obtained based on images in BUAA-SID 1.0,and the highest recognition result could achieve 95.87%.

@article{Meng2010CJA,
  title = {Full-viewpoint 3D Space Object Recognition Based on Kernel Locality Preserving Projections},
  author = {Meng, Gang and Jiang, Zhiguo and Liu, Zhengyi and Zhang, Haopeng and Zhao, Danpei},
  journal = {Chinese Journal of Aeronautics},
  volume = {23},
  number = {5},
  pages = {563 - 572},
  year = {2010},
  doi = {doi.org/10.1016/S1000-9361(09)60255-7},
}    
              

The usage of color invariance in SURF
Gang Meng; Zhiguo Jiang; Danpei Zhao
International Symposium on Multispectral Image Processing and Pattern Recognition (MIPPR), 2009
Abstract   BibTeX  

Space object recognition plays an important role in spatial exploitation and surveillance,followed by two main problems:lacking of data and drastic changes in viewpoints.In this article,firstly,we build a three-dimensional (3D) satellites dataset named BUAA Satellite Image Dataset (BUAA-SID 1.0) to supply data for 3D space object research.Then,based on the dataset,we propose to recognize full-viewpoint 3D space objects based on kemel locality preserving projections (KLPP).To obtain more accurate and separable description of the objects,firstly,we build feature vectors employing moment invariants,Fourier descriptors,region covariance and histogram of oriented gradients.Then,we map the features into kernel space followed by dimensionality reduction using KLPP to obtain the submanifold of the features.At last,k-nearest neighbor (kNN) is used to accomplish the classification.Experimental results show that the proposed approach is more appropriate for space object recognition mainly considering changes of viewpoints.Encouraging recognition rate could be obtained based on images in BUAA-SID 1.0,and the highest recognition result could achieve 95.87%.

@inproceedings{MengMIPPR2009,
  title      = {The usage of color invariance in SURF},
  author     = {Gang Meng; Zhiguo Jiang; Danpei Zhao},
  booktitle  = {Sixth International Symposium on Multispectral Image Processing and Pattern Recognition},
  pages      = {7},
  year       = {2009},
  doi        = {10.1117/12.833509},
}      
              

Maneuvering Target Tracking in Cluttered Background Based on Color Invariance and Support Vector Machine
Gang Meng ; Zhiguo Jiang ; Danpei Zhao ; Yue Gao
International Conference on Image and Graphics (ICIG), 2009
Abstract   BibTeX  

Maneuvering targets tracking in cluttered environment is a challenging problem in computer vision because of the difficulty of distinguishing the target from the background. In this paper, we treat tracking as a binary classification problem and employ support vector machine to suppress the background. In order to enhance the robustness against illumination changes, we propose to combine color invariance with traditional RGB values to train the SVM. First, we use expectation maximization algorithm to extract the target from the environment; then, RGB and color invariance values are used to train SVM. In the incoming frames, pixels in regions of interest are classified by SVM and the confidence map is produced, which will afterward be used by traditional tracking approach to track the target, in this paper, we employ particle filter. Experimental results on challenging sequences validate the effectiveness of the proposed method in cluttered background target tracking.

  @inproceedings{MengIG2009,
  title      = {Maneuvering Target Tracking in Cluttered Background Based on Color Invariance and Support Vector Machine},
  author     = {Gang Meng ;  Zhiguo Jiang ;  Danpei Zhao ;  Yue Gao},
  booktitle  = {International Conference on Image and Graphics},
  pages      = {510-515},
  year       = {2009},
  doi        = {10.1109/ICIG.2009.155},
  url        = {https://ieeexplore.ieee.org/document/5437928/figures},
  }      
              

Real-time illumination robust maneuvering target tracking based on color invariance
Gang Meng; Zhiguo Jiang; Danpei Zhao; Keren Ye
Image and Signal Processing, 2009
Abstract   BibTeX  

Illumination change is an important factor that affects the accuracy of tracking. In order to enhance the robustness against illumination changes in directions and intensities of the conventional tracking approaches, this paper proposes to apply color invariance theory to maneuvering target tracking. To depress the influence of the background, at the beginning of the process, we use expectation maximization algorithm based on Gaussian mixture model to extract the objects from the environment. Then, instead of using the gray space or RGB space, frames are transformed to color invariant space based on Kubelka-Munk model, followed by the implementation of conventional tracking approach such as mean shift or particle filtering to track the targets. Experimental results show that mean shift and improved particle filtering with color invariance can achieve 64.1 and 16.0 frames per second correspondingly, which is real-time, and the proposed approach improves the robustness of maneuvering targets tracking greatly.

  @article{MengISP2009,
  title      = {Real-time illumination robust maneuvering target tracking based on color invariance},
  author     = {Gang Meng; Zhiguo Jiang; Danpei Zhao; Keren Ye},
  journal    = {Image and Signal Processing},
  pages      = {1-5},
  year       = {2009},
  doi        = { 10.1109/CISP.2009.5301491},
  url        = {https://ieeexplore.ieee.org/document/5301491/},
  }