malleus, eu estava lendo no help do vray, no link abaixo e a descrição pro canal "vray shadow" diz que é a luz difusa bloqueada por outros objetos. Depois vou fazer um teste pra ver como funciona, mas acho que é uma mascara invertida. ;)
http://www.spot3d.com/vray/help/150R1/render_elements.htm
EDIT:
Opa! Descobri o canal que gera as sombras, é o VrayMatteShadow, mas não tem nada sobre ele no help, desconfio que este canal só tenha na ultima versão.
EDIT2:
Esse assunto me deixou mais interessado em composição, pois hoje em dia é o que há de melhor nos renders tanto que é muito usado em cinema, então encontrei um tópico na Chaos que tem uma pequena aulinha do Richard Rosenman (www.richardrosenman.com):
************
I never use render elements. I do each layer seperately and I have never run into problems.
If I need to render a character, I usually make the ground a matte object (-1 in the VRay object settings), and it renders the character without the shadow in the alpha and with the correct GI solution affecting it's feet.
I then render the BG without the char in the scene.
And finally, I make the character "not visible to camera", turn the ground back to a non-matte object, add a white VRay material and render it out for the shadow. Since the char is not visible to camera, it will cast it's shadow on the ground but not render.
You then comp the three layers together, using multiply for the shadow RGB layer.
Sometimes you need a shadow on something more complex than a simple ground plane. If this is the case then you need to derive the shadow from the BG. You render the same pass as above, and then remove the character and re-render the floor plane. In the comp, you do a difference comp with the two shadow layers and you are left with just the shadow. (Since one layer has the bg and shadow, and the other has just the bg, a difference of the two will result in just the shadow).
There are simpler ways than this to derive a shadow pass but this is guaranteed to work in every single scenario. Difference passes are extremely powerful and I often use them for a lot more than just shadows - ie. deriving a radiance pass, ambient occ pass, etc.
Hope this helps. Oh, we always use TGA with premultiplied alphas and comp in either Combustion or AE.
-Richard
************