Perception of colour gradients in real and computer simulated scenes effects on depth

Can you tell the difference between real filmed footage of an event, and a computer-rendered counterpart? Despite tremendous progress in animation and graphics, the answer is most likely yes. We still have a long way to go in generating high quality realistic rendered worlds, that have a wide variety of applications, from gaming, through medical and industrial simulators, to architect-designed walk-throughs that give us a feel for how a new building could look. Improving the naturalness and realism of such virtual environments is a key challenge for those involved in computer graphics and rendering, particularly when there is a demand for interactive, real-time applications: we want to walk around in that simulated new building, not just view static photograph-like scenes. One of the reasons that our progress is slow, is that the extraordinary visual capabilities of most humans, though apparently effortless, hide a complex web of visual processing that is not yet fully understood. If we do not yet understand what enhances realism for the human visual system, it is not surprising that progress is slow in developing technology to improve the realism of simulations. The aim of this work will be to elucidate some of the basic perceptual processes that underlie how subtle changes in colour and lightness enhance the realism of our perception of a three-dimensional scene. This human behavioural research underpins the development of graphics and rendering technologies that will deliver enhanced realism for virtual environments.One of the reasons why this problem is so hard is that the real world contains very complex patterns of light and colour that somehow translate into our perceptions of whether something is green, light, dark, near or far away. For example, my coffee cup on the table in front of me contains white, bright specular highlights that contribute to it looking glossy. There are also reflections of the stripy table mat on which it sits, yet I know these to be reflections, not part of the pattern on the mug. There is an attached shadow, cast by light from the window, and I know that is not part of my mug. And there are changes in lightness and colour across the surface, yet I see the mug as being a single colour, made from a single material. In this project we will study how humans distinguish between depth, light source, and material properties. For most real scenes, this is a very difficult computational problem because specific local patches that appear, for example, darker than those around them, can have the value they do for a variety of reasons. For example, the side of your grey filing cabinet may have a very different lightness than the front, because it slopes away from you in depth, and is at a different angle with the light source, compared with the front of the cabinet (and it could actually be a different colour). How does the human visual system achieve a coherent perception of a solid, non-deforming world, despite changes in view-point and lighting? This is a huge question in both human and computer vision research, with direct implications for computer graphics and virtual environment development, but one that is still poorly understood. We will start by exploring very simple visual scenes containing isolated objects, and study how colour and luminance information can influence depth perception. This data will be used to create models that predict when the luminance/colour information will be most useful. In other experiments we will use real objects and realistically computer-rendered scenes that preserve the relationship between objects and light source that would occur in the real world. Our work will give us immediate information about basic visual mechanisms of depth and scene perception that directly informs the fields of computer graphics and image processing, giving guidelines for when realistic luminance and colour gradients are required for a rendered scene to look realistic.

This project set out to explore interactions between different visual cues, for the perception of depth and shape. In this project we have studied how humans distinguish between depth, light source, and material properties, and how varying these various parameters impacts on depth and shape perception. In one strand of the project, we studied how colour and luminance information can influence depth perception. We found that realistic colour/luminance gradients can be used to perceive shape, and that although the visual information is often ambiguous, observers can quickly learn the relationship between location, lighting and shape, so that the cue becomes reliable (Harding, Harris & Bloj, 2012, PLoS ONE 7(4): e35950. doi:10.1371/journal.pone.0035950). We explored the effects of colour variation on shape-from-shading information. Contrary to previous findings, we discovered that colour does not always enhance perceived depth, as would be expected if the visual system exploited information about the statistics of the natural environment (Clery, Bloj, Harris, 2013, Journal of Vision, in press). Instead, the effects of colour on shape from shading are likely to be due to process in very early vision, where the separate colour and luminance process channels first interact.
In the other main strand of the project, we used optimal integration models to demonstrate that shading information can be used as a reliable cue to depth, when combined with binocular disparity, and that a classic model, the Maximum Likelihood Estimator, provides an excellent description of how the brain combines the visual information (Lovell, Bloj, Harris, 2012, Journal of Vision. 12, 1, p. 1-18. doi: 10.1167/12.1.1). We have also, for the first time, explored how these two visual cues are combined over time. Our research has demonstrated that shape from shading can be a ‘quick and dirty’ cue to shape at very short viewing durations (where disparity is not useful). At longer durations, binocular disparity becomes the more reliable source of visual information (paper in preparation).