Features for Image Features for Image Retrieval vorgelegt von: Thomas Deselaers Matrikelnummer...

download Features for Image Features for Image Retrieval vorgelegt von: Thomas Deselaers Matrikelnummer 218894

of 121

  • date post

    28-Sep-2020
  • Category

    Documents

  • view

    0
  • download

    0

Embed Size (px)

Transcript of Features for Image Features for Image Retrieval vorgelegt von: Thomas Deselaers Matrikelnummer...

  • Diplomarbeit im Fach Informatik Rheinisch-Westfälische Technische Hochschule Aachen

    Lehrstuhl für Informatik VI Prof. Dr.-Ing. H. Ney

    Features for Image Retrieval

    vorgelegt von: Thomas Deselaers

    Matrikelnummer 218894

    Gutachter: Prof. Dr.-Ing. H. Ney

    Prof. Dr. T. Seidl

    Betreuer: Dipl.-Inform. D. Keysers

  • Hiermit versichere ich, dass ich die vorliegende Diplomarbeit selbständig verfasst und keine anderen als die angegebenen Hilfsmittel verwendet habe. Alle Textauszüge und Grafiken, die sinngemäß oder wörtlich aus veröffentlichten Schriften entnommen wurden, sind durch Refe- renzen gekennzeichnet.

    Aachen, im Dezember 2003

    Thomas Deselaers

  • Acknowledgements

    The present work originates from my work as student researcher at the Chair of Computer Science VI of the RWTH Aachen University of Technology, where I have been a member of the image recognition group since July 2001. I would like to thank Prof. Dr.-Ing. Hermann Ney for the interesting possibilities at this department and the chance to attend the conference “25th Pattern Recognition Symposium” in Magdeburg.

    Furthermore, I would like to thank Daniel Keysers for the helpful suggestions, discussions and assistances this work received. His ideas and suggestions were a great help.

    I would like to thank Prof. Dr. Thomas Seidl as well, who kindly accepted to attend the work.

    Also, I would like to thank the other student members of the image recognition group at the Chair of Computer Science VI – Christian Gollan, Tobias Kölsch, Philippe Dreuw, and Ilja Bezrukov – for many discussions, helpful tips and comments.

    Also I would like to thank Diego Biurrun, Stefan Jacobs, and Arne Mauser for proof reading the manuscript.

    And of course special thanks go to my parents for all the possibilities they gave me and are still giving to me and to my girl friend Daniela for supporting me while doing this work.

  • Contents

    1 Introduction 1

    2 Basic principles 3 2.1 Basic Principles for Image Retrieval . . . . . . . . . . . . . . . . . . . . . . . 3 2.2 Basic Principles for Image Clustering . . . . . . . . . . . . . . . . . . . . . . . 4

    2.2.1 k-Means Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2.2 LBG-Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

    2.3 Basic Principles for Classification . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.4 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

    3 State of the Art in Content-Based Image Retrieval 11 3.1 Related Work in Content-Based Image Retrieval . . . . . . . . . . . . . . . . 11 3.2 Related Work in Clustering of Images . . . . . . . . . . . . . . . . . . . . . . 13 3.3 Related Work in Object-Recognition and Classification of Images . . . . . . . 14

    4 Features for Content-Based Image Retrieval 17 4.1 Image Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 4.2 Color Histograms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 4.3 Invariant Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 4.4 Invariant Features by Integration . . . . . . . . . . . . . . . . . . . . . . . . . 20

    4.4.1 Invariant Feature Histograms . . . . . . . . . . . . . . . . . . . . . . . 23 4.4.2 Invariant Feature Vectors . . . . . . . . . . . . . . . . . . . . . . . . . 24 4.4.3 Invariant Fourier Mellin Features . . . . . . . . . . . . . . . . . . . . . 24

    4.5 Gabor Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 4.6 Tamura Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 4.7 Global Texture Descriptor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 4.8 Local Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 4.9 Histograms of Local Features . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 4.10 Region-Based Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 4.11 PCA Transformed Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 4.12 Correlation of Different Features . . . . . . . . . . . . . . . . . . . . . . . . . 33

    5 Comparing Features 35 5.1 Histogram Comparison Measures . . . . . . . . . . . . . . . . . . . . . . . . . 35

    5.1.1 Bin-by-Bin Comparison Measures . . . . . . . . . . . . . . . . . . . . . 35 5.1.2 Cross-Bin Comparison Measures . . . . . . . . . . . . . . . . . . . . . 37

    5.2 Comparing Images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

    i

  • 5.2.1 Euclidean Distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 5.2.2 Tangent Distance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 5.2.3 Image Distortion Model . . . . . . . . . . . . . . . . . . . . . . . . . . 41

    5.3 Comparing Images Based on Local Features . . . . . . . . . . . . . . . . . . . 42 5.3.1 Direct Transfer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 5.3.2 Local Feature Image Distortion Model . . . . . . . . . . . . . . . . . . 42

    5.4 Comparing Region-Based Descriptions of Images . . . . . . . . . . . . . . . . 42 5.4.1 Integrated Region Matching . . . . . . . . . . . . . . . . . . . . . . . . 43 5.4.2 Quantized Hungarian Region Matching . . . . . . . . . . . . . . . . . 43

    5.5 Other Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

    6 Applications 45 6.1 Content-Based Image Retrieval . . . . . . . . . . . . . . . . . . . . . . . . . . 45 6.2 Grouping of Visually Similar Images . . . . . . . . . . . . . . . . . . . . . . . 46 6.3 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

    7 Databases 49 7.1 Corel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 7.2 WANG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 7.3 Corel Subset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 7.4 IRMA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 7.5 CalTech Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 7.6 UW Database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 7.7 ZuBuD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 7.8 MPEG-7 Test Set . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 7.9 Google . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 7.10 COIL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

    8 Results 57 8.1 Performance Evaluation for Content-Based Image Retrieval . . . . . . . . . . 57 8.2 Results for Content-Based Image Retrieval . . . . . . . . . . . . . . . . . . . . 59

    8.2.1 Comparison of Different Distance Functions . . . . . . . . . . . . . . . 60 8.2.2 Comparison of Different Features . . . . . . . . . . . . . . . . . . . . . 65 8.2.3 Image Retrieval Using Different Databases . . . . . . . . . . . . . . . . 68

    8.3 Performance Evaluation for Clustering Visually Similar Images . . . . . . . . 76 8.4 Results for Clustering Visually Similar Images . . . . . . . . . . . . . . . . . . 76

    8.4.1 Clustering the Google Images . . . . . . . . . . . . . . . . . . . . . . . 77 8.4.2 Clustering the WANG & COIL Images . . . . . . . . . . . . . . . . . . 77

    9 Conclusion and Perspectives 81

    A Software Documentation 85 A.1 FIRE Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 A.2 Clustering Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 A.3 Feature Extraction Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

    B Additional Tables 95

    ii

  • List of Tables

    2.1 Symbols used in this work. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

    4.1 Monomials used for invariant feature vectors. . . . . . . . . . . . . . . . . . . 24

    8.1 Error rates on WANG and IRMA-1617 using different comparison measures. . 61 8.2 Error rates using time warp distance on IRMA-1617. . . . . . . . . . . . . . . 61 8.3 Error rates for the IRMA-1617 database using different comparison measures. 64 8.4 Error rates using region based features for the WANG database. . . . . . . . 64 8.5 Comparison of error rates on IRMA-1617 for local features and IDM. . . . . . 65 8.6 Error rates using different features for the WANG database. . . . . . . . . . . 66 8.7 Error rates using different features for the IRMA-1617 database. . . . . . . . 66 8.8 Error rates using partially rotation invariant feature histograms. . . . . . . . 68 8.9 Results for different training situations on the IRMA-3879 database. . . . . . 70 8.10 Error rates obtained on the IRMA-1617 database. . . . . . . . . . . . . . . . 71 8.11 Results for different training situations for the WANG and UW databases. . . 72 8.12 Error rates for the WANG and Corel subset database. . . . . . . . . . . . . . 73 8.13 Error rates for WANG subsets. . . . . . . . . . . . . . . . . . . . . . . . . . . 73 8.14 R̃ank for the MPEG-7 databases. . . . . . . . . . . . . . . . . . . . . . . . . . 74 8.15 Error rates for ZuBuD using different methods. . . . . . . . . . . . . . . . . . 75 8.16 Equal error rates on the CalTech database. . . . . . . . . . . . . . . . . . . . 76 8.17 Results for clustering WANG. . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 8.18 Results for clustering COIL. . . . . . . . . . . . . . . . . . . . . . . . . .