javascript - White balance (Color Suppression) Formula?

Question

I need help with a little bit of color math.(RGBA) I'm trying to reduce the amount of green on a character, without actually affecting the green background. Green screen for visual effects.

So here is what I have:

I have a guy on green screen.

Using the expression: g>(r+b)/2 ? (r+b)/2:g Expression says if the green color value is greater than the sum of red and blue divided by 2, then it green will be set to that sum. Running that expression will give me the second image. You can see that the green is removed perfectly from the guy, which I want, but it also takes the green from the background and makes it gray, which I don't want.

This image here is how much green is removed. I get this by subtracting the first image from the second. So its the difference between the two.

Can someone help me figure out how I can only remove the green from the guy and not the background via an expression/formula ? I have tried to clamp the green, meaning, write an expression that says: for any green pixel thats below a certain value that not equal to the background green, turn it black. This kinda works, but its very harsh on the fine edges like that hair.

Below is the desired result which i achieved by doing a lot of color correction and grading. I would love to get this result via expression.

and here is the final result. it looks great. This image is the green gone from the character, but not the background. I got this by adding the image above to the second image.I did a split screen so you can see the difference. mostly visible in the jacket and hair.

Answer 1

I am no expert in the matter but:

I do not think your approach work as you intend (from physics side).

The green background is shining on your character which is adding light to the character surface. So you need to remove that (unwanted light). You are turning all greenish pixels to grayish instead and that is WRONG in my opinion. What if your guy has green T-shirt ?

I would approach this like this:

Firstly I hope you got seamless lighting condition on your Scene set otherwise this will never work without proper 3D illumination/scattering analysis which is a huge task and you would need for that also complete 3D model of your scene.

1. First detect the amount of light scattered to objects position.

   Simply place white paper sheet to approximate position of your person. take the picture. Then you find the image of the sheet compute its average color and compute how much green color is added to white.

   c[i]=c0[i]+c0[i]*m[i]
   

   • c pixel color (average color of sheet)
   • c0 object color (real object color ... white but should be in scale with c which can be a bit tricky without access to scene stage set)
   • m scale (this is what we need)
   • i channel

   This will lead to system of 3 linear equations so compute m from it. This is invariant on background color so it will work for any color ...

   I do not have an image of white sheete from your set so I use the Teeth instead. The problem is that there is too much red-ish color illuminated from the skin/flesh around so I use only blue channel to compensate. This is not accurate but it will do (unless the background illuminate also to another channels)... so for the extracted teeth image average color:

   <- this is the extracted teeth image I used)

   float m[green]=float(avg[green]-avg[blue])/float(avg[blue]);
   

   • c0 is blue channel of average color
   • c is green channel of average color

   All the other channels of m are zero (for this case). Do not forget that m[] is float !!!.

2. Now just remove the illumination from image

   Following the same equation compute c0 so:

   c0[i]=c[i]/(1.0+m[i])
   

   • c uncorrected image pixel color
   • c0 corrected image pixel color
   • m scale (computed in step #1)
   • i channel (R,G,B)

   Do each channel each pixel... This is the original image c:

   

   This is the corrected image c0:

   

   This is the substraction of c-c0:

   

   As you can see some green is taken away also from background but that is alright as the background is illuminated too !!! Also the wrist is more green then it should be but I think that is because it is too far from the teeth I calibrated for and also might be too close to wall ... as this is highly dependent on the object position too (if lighting is not seamless enough) not to mention it is in shade ...