The foundation for practically every modern digital image capture method starts with the Bayer sensor concept. In this blog we’re going to cover what exactly that means.
To do so, let’s go back to the beginning. In order to create a digital image, be it a still or video (they work on the exact same principles) light needs to be captured and processed. For traditional film this was done using a frame of film containing a layer of emulsion made up of a silver halide crystal substance which chemically reacts to light resulting in a film negative.
To do so, let’s go back to the beginning. In order to create a digital image, be it a still or video (they work on the exact same principles) light needs to be captured and processed. For traditional film this was done using a frame of film containing a layer of emulsion made up of a silver halide crystal substance which chemically reacts to light resulting in a film negative.
Digital sensors replicate this exact same process. In essence, photons of light hit the sensor which creates an electronic signal that can be processed by the cameras on board processor into a tangible image.
Let's dive into the details of this a little more, a sensor is made of individual ‘cavities’ known as photosites. It is these photosites that are reactive to light. You could think of these as the equivalent to pixels in video, they are the individual discrete squares that make up the sensors face. This is great but we face one issue. These photosites can determine the intensity of light but currently can’t distinguish differences in colour without modification resulting in considerable downsides. This is because the colour of light is determined by it's wavelength, a attribute which cannot be differentiated by a sensor by nature.
Let's dive into the details of this a little more, a sensor is made of individual ‘cavities’ known as photosites. It is these photosites that are reactive to light. You could think of these as the equivalent to pixels in video, they are the individual discrete squares that make up the sensors face. This is great but we face one issue. These photosites can determine the intensity of light but currently can’t distinguish differences in colour without modification resulting in considerable downsides. This is because the colour of light is determined by it's wavelength, a attribute which cannot be differentiated by a sensor by nature.
![Picture](/uploads/2/4/5/7/24574909/7097649.gif?380)
Imagine this is a side view of a cross-section of a sensor. Each slot is a photosite and each coloured circle represents a photon of light of a certain wavelength (giving it it’s colour). Photons of all wavelengths get through to the photosites but the photosite can’t distinguish which colour is which - they all appear the same.
This is where the Bayer sensor design comes in. It is the most successful of a number of solutions currently designed to acquire colour data in digital imaging. The system comprises of two key elements; the Bayer pattern, an intelligently designed pattern of colour-specific photosites, and a Colour Filter Array to determine which colour light can pass through to each photosite. Let's break it down:
Colour Filter Array
You can think of the two elements of the bayer sensor as a theoretical principle and a phyical piece hardware, the array being the later. At a basic level this is fairly simple. Similar to before, we have individual photosites which accept light in much the same way, only this time there is a filter applied to each photosite allowing only certain wavelengths of light to pass through.
![Picture](/uploads/2/4/5/7/24574909/6322499.gif?375)
With the colour filter array applied only light of a certain wavelength is allowed to pass into each photosite, meaning the sensor knows which colour is which and how much of it there is.
As you’ll notice, this technique means a large percentage of light (50-70%) is left behind resulting in a darker image. Although sensor technology is constantly improving, this is the major drawback of the colour array filter technique but there is currently no better alternative.
Bayer Pattern
You may have noticed in the example above that there are 2 photosites reactive to green whilst only one each for red and blue, this is no accident. The Bayer pattern is a specific arrangement of colour filters which make up the array. It gives us two green photosites per every red and blue one. This may seem bizarre at first but the reason behind it is that the human eye is actually more sensitive to green light (who's wavelengths fall in the center of the visible spectrum) than any other. We can use this knowledge to our advantage by capturing more data for green light allowing us to create a perceived higher quality image. We can represent the Bayer pattern using a 4x4 grid:
![Picture](/uploads/2/4/5/7/24574909/2550014.gif?164)
Imagine this 4x4 grid is a tiny cross section of the face of a sensor. Every individual colour square represents a colour-specific photosite. Now, if each photosite can only give us a single colour value (of only red, green or blue, yet alone any other colour/shade) the system needs to take data from multiple photosites at once to generate a full colour pixel. This process is known as 'Bayer Demosaicing'.
This could be done by treating each 2x2 grid as one whole pixel like Example A below, but we can actually improve on this greatly by using each photosite for multiple pixels at once, such as Example B, however this is only at a basic level. In practice manufacturers have complex algorithms to squeeze as much data out of the pattern as possible.
This could be done by treating each 2x2 grid as one whole pixel like Example A below, but we can actually improve on this greatly by using each photosite for multiple pixels at once, such as Example B, however this is only at a basic level. In practice manufacturers have complex algorithms to squeeze as much data out of the pattern as possible.
In Example A, each square represents a whole pixel in the most basic demosaic method. Example B shows how photosites can be used multiple times to gather information for multiple pixels in order to be more efficient and resourceful, imagine the same 2x2 grid is drawn around each letter to create overlapping pixels.
Of course this is really only a basic overview of the digital sensor, but it gives you a good idea of how colour images are generally achieved without the need for multiple sensors (research 3 chip sensor camera for more).