Deferred lighting with toon shading (custom lighting)

Node-based Shader Editor
Post Reply
Liraco
Posts: 7
Joined: Mon Sep 25, 2017 10:11 pm

Deferred lighting with toon shading (custom lighting)

Post by Liraco »

Is there a way to have toon shading with deferred lightning?

Me and my team have worked on a nice toon shader but noticed in debugging post-processing effects that it has buggy behaviors in deferred lighting mode. You can see anything with the toon shader disappears when viewing normals (in the example, there's one's using a custom ASE toon shader, one made with Shader Forge, and then the example toon shader you provide). AO is messed up (too dark) and depth of field is also wrong (you can see the primitives with red tinge but the heads are completely gray despite being at the same plane as the primitives).

I hope there's a workaround to get custom lighting and toon shading to work with deferred. Thanks!

BTW, this is using ASE v1.3.4 and Unity 2017.1
Attachments
Head should be somewhat focused.
Head should be somewhat focused.
deferred-focus.jpg (165.03 KiB) Viewed 6557 times
AO should be like in the primitives and wall.
AO should be like in the primitives and wall.
deferred-ao.jpg (162.04 KiB) Viewed 6557 times
Toon shaders disappear in "normals" view.
Toon shaders disappear in "normals" view.
deferred-normals.jpg (155.64 KiB) Viewed 6557 times
User avatar
Ricardo Teixeira
Posts: 954
Joined: Fri Aug 09, 2013 2:26 pm

Re: Deferred lighting with toon shading (custom lighting)

Post by Ricardo Teixeira »

Liraco wrote:Is there a way to have toon shading with deferred lightning?

Me and my team have worked on a nice toon shader but noticed in debugging post-processing effects that it has buggy behaviors in deferred lighting mode. You can see anything with the toon shader disappears when viewing normals (in the example, there's one's using a custom ASE toon shader, one made with Shader Forge, and then the example toon shader you provide). AO is messed up (too dark) and depth of field is also wrong (you can see the primitives with red tinge but the heads are completely gray despite being at the same plane as the primitives).

I hope there's a workaround to get custom lighting and toon shading to work with deferred. Thanks!

BTW, this is using ASE v1.3.4 and Unity 2017.1
Greetings!

Apologies for the delay, it's best to get in touch directly at [email protected] for dedicated support. Unfortunately, there's currently no workaround as it would involve changing the actual deferred lighting shader, which is global (the same for every light and material).

However, AO and DoF should work as expected. From the screenshot it seems that the shader is not writing to depth, or perhaps depth+normal.

Would it be possible to test a sample on our side?

Thanks!
Sales & Customer Relations at Amplify Creations
Learn more about our offering: Amplify Creations Products
Amplify Shader Editor won the Asset Store Best Tool Award - Thank you for your support!
Liraco
Posts: 7
Joined: Mon Sep 25, 2017 10:11 pm

Re: Deferred lighting with toon shading (custom lighting)

Post by Liraco »

Ricardo Teixeira wrote:However, AO and DoF should work as expected. From the screenshot it seems that the shader is not writing to depth, or perhaps depth+normal.

Would it be possible to test a sample on our side?
That's the thing though, your own included toon shader example is behaving in the same "wrong" manner (not writing to depth) as the custom shaders we wrote. Which is why I was curious as to there being a workaround. You've confirmed what I was thinking though (that it'll take a rewriting of the global shader.

Should I just send an email directly then (though it doesn't seem there's a fix)?
User avatar
Ricardo Teixeira
Posts: 954
Joined: Fri Aug 09, 2013 2:26 pm

Re: Deferred lighting with toon shading (custom lighting)

Post by Ricardo Teixeira »

Liraco wrote:
Ricardo Teixeira wrote:However, AO and DoF should work as expected. From the screenshot it seems that the shader is not writing to depth, or perhaps depth+normal.

Would it be possible to test a sample on our side?
That's the thing though, your own included toon shader example is behaving in the same "wrong" manner (not writing to depth) as the custom shaders we wrote. Which is why I was curious as to there being a workaround. You've confirmed what I was thinking though (that it'll take a rewriting of the global shader.

Should I just send an email directly then (though it doesn't seem there's a fix)?
Apologies, I was referring to the actual image effect shader. Without adjustments the AO effect will definitely be displayed wrong, so to speak, setting it to Force Forward Compatibility should resolve your immediate AO issues.

Image

We would be happy to examine the shader, we were unable to replicate the DoF issue.

Image
Image

Thanks!
Sales & Customer Relations at Amplify Creations
Learn more about our offering: Amplify Creations Products
Amplify Shader Editor won the Asset Store Best Tool Award - Thank you for your support!
Liraco
Posts: 7
Joined: Mon Sep 25, 2017 10:11 pm

Re: Deferred lighting with toon shading (custom lighting)

Post by Liraco »

Forcing Forward is something we want to avoid.

I found an answer and managed to get it working by having a custom Deferred Shader:
https://www.gamasutra.com/blogs/DavidLe ... ity_56.php
Article says Unity 5.6 but this is in Unity 2017.1.0

AO works fine now, and as you can see in the screenshot, normals work too. Hopefully this'll help someone with similar problems.
Attachments
After using the custom deferred shader.
After using the custom deferred shader.
CustomDeferredShader.jpg (114.18 KiB) Viewed 6513 times
User avatar
Ricardo Teixeira
Posts: 954
Joined: Fri Aug 09, 2013 2:26 pm

Re: Deferred lighting with toon shading (custom lighting)

Post by Ricardo Teixeira »

Liraco wrote:Forcing Forward is something we want to avoid.

I found an answer and managed to get it working by having a custom Deferred Shader:
https://www.gamasutra.com/blogs/DavidLe ... ity_56.php
Article says Unity 5.6 but this is in Unity 2017.1.0

AO works fine now, and as you can see in the screenshot, normals work too. Hopefully this'll help someone with similar problems.
That's awesome, thank you for sharing!

We'll make sure that it's added to the wiki!
Sales & Customer Relations at Amplify Creations
Learn more about our offering: Amplify Creations Products
Amplify Shader Editor won the Asset Store Best Tool Award - Thank you for your support!
Post Reply