The Art and Science of HDR Imaging

John J. McCann, Consultant, McCann Imaging, USA John McCann received a B.A. degree in Biology from Harvard University in 1964. He worked in, and later managed, the Vision Research Laboratory at Polaroid from 1961 to 1996. He has studied human color vision, digital image processing, large format instant photography and the reproduction of fine art. His 120 publications have studied Retinex theory, color from rod/Lcone interactions at low light levels, appearance with scattered light, and HDR imaging. He has been a Fellow of the Society of Imaging Science and Technology (IS&T) since 1983. He is a past President of IS&T and the Artists Foundation, Boston. In 1996 he received the SID Certificate of Commendation. He is the IS&T/OSA 2002 Edwin H. Land Medalist, and IS&T 2005 Honorary Member, and is a 2008 Fellow of the Optical Society of America. He is currently consulting and continuing his research on color vision. Alessandro Rizzi, Universita degli Studi di Milano, Italy Professor Alessandro Rizzi holds a degree in Computer Science at University of Milano and received a PhD in Information Engineering at University of Brescia (Italy). He taught Information Systems and Computer Graphics at University of Brescia and at Politecnico di Milano. He is currently an assistant professor teaching Multimedia and Human-Computer Interaction, and senior research fellow at the Department of Information Technologies at University of Milano. Since 1990 he has researched in the field of digital imaging and vision. His main research topic is the use of color information in digital images with particular attention to color perception mechanisms. He is the coordinator of the Italian Color Group Conference Chair of Color Conference at IS&T/SPIE Electronic Imaging, and a principle organizer of European Marie Curie Project CREATE.

About the Authors xix Preface xxi Series Preface xxiii Acknowledgements xxv Section A HISTORY OF HDR IMAGING 1 1 HDR Imaging 3 1.1 Topics 3 1.2 Introduction 3 1.3 Replicas and Reproductions 4 1.4 A Choice of Metaphors for HDR Reproduction 5 1.5 Reproduction of Scene Dynamic Range 7 1.6 HDR Disciplines 8 1.7 Outline of the Text 10 1.8 Summary 11 1.9 References 12 2 HDR Tools and Defi nitions 13 2.1 Topics 13 2.2 Introduction 13 2.3 Pixels 14 2.4 Dynamic Ranges 14 2.5 Measuring Light 17 2.6 Measuring Color Spaces 18 2.7 Image Reproduction 21 2.8 Contrast 24 2.9 Digital Imaging 25 2.10 Summary 25 2.11 References 26 3 HDR in Natural Scenes 27 3.1 Topics 27 3.2 Appearance in HDR and Color Constancy 27 3.3 Summary 30 3.4 References 31 4 HDR in Painting 33 4.1 Topics 33 4.2 Introduction 33 4.3 Ancient Painting 33 4.4 Perspective 35 4.5 Chiaroscuro 37 4.6 Gerritt van Honthorst (Gherardo delle Notti) 37 4.7 Rembrandt van Vijn 40 4.8 John Constable 40 4.9 John Martin 40 4.10 Impressionism 41 4.11 Photorealism 43 4.12 Summary 43 4.13 References 44 5 HDR in Film Photography 45 5.1 Topics 45 5.2 Introduction 45 5.3 Multiple Exposures in the 1850s 46 5.4 HP Robinson 47 5.5 Hurter and Driffi eld-Scientifi c Calibration of AgX Film Sensitivity 48 5.6 Sheppard and Mees 50 5.7 19th Century Professional Amateur Photography 50 5.8 20th Century Corporate Photography 50 5.9 20th Century Control of Dynamic Range 51 5.10 Other Silver-Halide Stories 56 5.11 Summary 56 5.12 References 57 6 The Ansel Adams Zone System 59 6.1 Topics 59 6.2 Introduction 59 6.3 Compressing the HDR World into the LDR Print 59 6.4 Visualization 60 6.5 Scene Capture 61 6.6 Performing the Score 65 6.7 Moonrise, Hernandez 66 6.8 Apparent vs. Physical Contrast 66 6.9 Summary 66 6.10 References 68 7 Electronic HDR Image Processing: Analog and Digital 69 7.1 Topics 69 7.2 Introduction 69 7.3 Human Spatial Vision 69 7.4 Electronic HDR Image Processing 70 7.5 Summary 74 7.6 References 75 8 HDR and the World of Computer Graphics 77 8.1 Topics 77 8.2 Introduction 77 8.3 Early Years: the 60s 78 8.4 Early Digital Image Synthesis: the 70s 78 8.5 The Turning Point: the 80s 79 8.6 Computational Photorealism: from the 90s 80 8.7 Summary 80 8.8 References 81 9 Review of HDR History 83 9.1 Topics 83 9.2 Summary of Disciplines 83 9.3 Review 84 9.4 Summary 87 9.5 References 87 Section B MEASURED DYNAMIC RANGES 89 10 Actual Dynamic Ranges 91 10.1 Topics 91 10.2 Introduction 91 10.3 Dynamic Range of Light Sensors 92 10.4 Bits per Pixel 93 10.5 Dynamic Range of Display Devices 94 10.6 Interactions of Pixels in Images 95 10.7 Summary 96 10.8 References 96 11 Limits of HDR Scene Capture 99 11.1 Topics 99 11.2 Introduction 99 11.3 HDR Test Targets 99 11.4 Camera Veiling Glare Limits 101 11.5 Glare in Film Cameras 107 11.6 Review 111 11.7 Summary 111 11.8 References 112 12 Limits of HDR in Humans 113 12.1 Topics 113 12.2 Introduction 113 12.3 Visual Appearance of HDR Displays 113 12.4 von Honthorst s Painting and the 4scaleBlack HDR Target 116 12.5 HDR Displays and Black and White Mondrian 116 12.6 HDR and Tone Scale Maps 117 12.7 HDR Displays and Contrast 117 12.8 Summary 117 12.9 References 118 13 Why Does HDR Improve Images? 119 13.1 Topics 119 13.2 Introduction 119 13.3 Why are HDR Images Better? 120 13.4 Are Multiple Exposures Necessary? 120 13.5 Summary 121 13.6 References 121 Section C SEPARATING GLARE AND CONTRAST 123 14 Two Counteracting Mechanisms: Glare and Contrast 125 14.1 Topics 125 14.2 Introduction 125 14.3 Two Spatial Mechanisms 126 14.4 Calculated Retinal Image 126 14.5 Measuring the Range of HDR Appearances 131 14.6 Calculating the Retinal Image 131 14.7 Visualizing the Retinal Image 131 14.8 HDR and Uniform Color Space 132 14.9 Summary 132 14.10 References 132 15 Measuring the Range of HDR Appearances 135 15.1 Topics 135 15.2 Introduction 135 15.3 Design of Appearance Scale Target 136 15.4 Magnitude Estimation Experiments 138 15.5 Scene Dependent Tone Scale 141 15.6 Glare and Contrast 142 15.7 Summary 143 15.8 References 143 16 Calculating the Retinal Image 145 16.1 Topics 145 16.2 Introduction 145 16.3 Converting Scene Luminance to Retinal Contrast 146 16.4 Calculating Retinal Radiance 146 16.5 Changes in the Retinal Image from Glare 149 16.6 Appearance and Retinal Image 149 16.7 Scene Content and Psychometric Functions 151 16.8 Summary 151 16.9 References 152 17 Visualizing HDR Images 153 17.1 Topics 153 17.2 Introduction 153 17.3 Calculated Retinal Image Contrast 154 17.4 Retinal Image Contrast 155 17.5 Summary 159 17.6 References 159 18 HDR and Uniform Color Spaces 161 18.1 Topics 161 18.2 Introduction 161 18.3 Uniform Color Spaces Psychophysics 161 18.4 Color Vision Physiology 164 18.5 Accurate Transformations from CMF to UCS 165 18.6 Summary 167 18.7 References 168 19 Glare: A Major Part of Vision Theory 169 19.1 Topics 169 19.2 Introduction 169 19.3 Glare: Distorts Lightness below Middle Gray, More or Less 169 19.4 Pixel-based HDR Image Processing 170 19.5 Summary 171 19.6 References 171 Section D SCENE CONTENT CONTROLS APPEARANCE 173 20 Scene Dependent Appearance of Quanta Catch 175 20.1 Topics 175 20.2 Introduction 175 20.3 Models of Vision A Choice of Paradigms 175 20.4 Illumination, Constancy and Surround 176 20.5 Maximum s Enclosure and Distance 176 20.6 Size of Maxima 177 20.7 Assimilation 177 20.8 Maxima and Contrast with Maxima 177 21 Illumination, Constancy and Surround 179 21.1 Topics 179 21.2 Introduction 179 21.3 Hipparchus of Nicea 180 21.4 Flat-2-D Transparent Displays 182 21.5 A Simple Two-Step Physical Description 183 21.6 Complex 3-D Scenes 185 21.7 Local Maxima 189 21.8 Review 190 21.9 Summary 190 21.10 References 191 22 Maximum s Enclosure and Separation 193 22.1 Topics 193 22.2 Introduction 193 22.3 Experimental Design 194 22.4 Lightness Matches Light Gray on Black 194 22.5 Lightness Matches Dark Gray on Black 195 22.6 Dark Gray on Black: Varying White s Position 197 22.7 Review 198 22.8 Summary 199 22.9 References 200 23 Maxima Size and Distribution 201 23.1 Topics 201 23.2 Introduction 201 23.3 Experimental Procedure 202 23.4 Controls 202 23.5 Dispersion of White ( Snow ) 202 23.6 Sides and Corners 203 23.7 Lines 204 23.8 Equivalent Backgrounds 205 23.9 Equivalent Backgrounds and Models of Vision 207 23.10 Summary 207 23.11 References 208 24 From Contrast to Assimilation 209 24.1 Topics 209 24.2 Introduction 209 24.3 Segmented Surrounds 210 24.4 Checkerboard Variants 215 24.5 Summary 216 24.6 References 216 25 Maxima and Contrast with Maxima 217 25.1 Topics 217 25.2 Merger of Aperture and Object Modes 217 25.3 Infl uence of the Maxima 218 25.4 Summary 219 Section E COLOR HDR 221 26 HDR, Constancy and Spatial Content 223 26.1 Topics 223 26.2 Introduction 223 26.3 Red and White Projections 224 26.4 Color Mondrians 225 26.5 Constancy s On/Off Switch 225 26.6 Color of 3-D Mondrians LDR/HDR Illumination 226 26.7 Color Constancy is HDR 226 26.8 References 226 27 Color Mondrians 227 27.1 Topics 227 27.2 Introduction 227 27.3 Color Mondrians 229 27.4 The Signature of Color Constancy 237 27.5 Search for Evidence of Adaptation Averages 240 27.6 Transparency in Mondrians 243 27.7 Color Assimilation 243 27.8 Summary 244 27.9 References 245 28 Constancy s On/Off Switch 247 28.1 Topics 247 28.2 Introduction 247 28.3 Maximov s Shoe Boxes 247 28.4 New Maxima Restores Constancy 250 28.5 Independent L, M, S Spatial Processing 251 28.6 Model Predictions 253 28.7 Center-Surround Target Results 253 28.8 Summary 255 28.9 References 256 29 HDR and 3-D Mondrians 257 29.1 Topics 257 29.2 Color Constancy and Appearance 257 29.3 Color Constancy Models 258 29.4 Measuring Changes in Appearance from Changes in Illumination 259 29.5 Magnitude Estimation Appearance Measurements 262 29.6 Watercolor Rendition Measurements of Appearance 263 29.7 Review of 3-D Mondrian Psychophysical Measurements 266 29.8 Color Constancy Models 268 29.9 Conclusions 270 29.10 References 271 30 Color Constancy is HDR 273 30.1 Topics 273 30.2 Introduction 273 30.3 Rod Receptors and HDR 274 30.4 Assembling Appearance: Color Constancy, Rod Vision and HDR 279 30.5 Summary 280 30.6 References 280 Section F HDR IMAGE PROCESSING 283 31 HDR Pixel and Spatial Algorithms 285 31.1 Topics 285 31.2 Introduction HDR Image Processing Algorithms 285 31.3 One Pixel Tone Scale Curves 286 31.4 Some of the Pixels Local Processing 288 31.5 All of the Pixels 289 31.6 All Pixels and Scene Dependent The Retinex Extended Family 289 31.7 Retinex Algorithms 290 31.8 ACE Algorithms 290 31.9 Analytical, Computational and Variational Algorithms 290 31.10 Techniques for Analyzing HDR Algorithms 290 31.11 The HDR Story 291 31.12 References 291 32 Retinex Algorithms 293 32.1 Topics 293 32.2 Introduction 293 32.3 How to Calculate Lightness Using Ratio-Products 297 32.4 A Variety of Processing Networks 301 32.5 Image Content 302 32.6 Real Images 1975 307 32.7 The Extended Family of Retinex Models 319 32.8 Algorithm s Goal 334 32.9 References 337 33 ACE Algorithms 341 33.1 Topics 341 33.2 Introduction 341 33.3 ACE Algorithm 341 33.4 Retinex and ACE 344 33.5 ACE Characteristics 345 33.6 RACE 349 33.7 Other Vision-based Models 350 33.8 Summary 350 33.9 References 351 34 Analytical, Computational and Variational Algorithms 353 34.1 Topics 353 34.2 Introduction 353 34.3 Math in the Framework of the Human Visual System 354 34.4 Analytical Retinex Formulas 354 34.5 Computational Retinex in Wavelets 354 34.6 Retinex and the Variational Techniques 355 34.7 Summary 356 34.8 References 357 35 Evaluation of HDR Algorithms 359 35.1 Topics 359 35.2 Introduction 359 35.3 Quantitative Approaches to Algorithm Evaluation 360 35.4 Lightness Test Targets 361 35.5 Ratio Metric 362 35.6 Quantitative Evaluation of 3-D Mondrians 367 35.7 Locality Test Targets 369 35.8 Summary 370 35.9 Lessons From Quantitative Studies of HDR in Cameras 371 35.10 References 371 36 The HDR Story 373 36.1 Topic 373 36.2 Straightforward Technology Stories 373 36.3 The HDR Story is Defi ned by Limits 373 36.4 HDR Works Well 374 36.5 References 375 Glossary 377 Author Index 385 Subject Index 387