Bug report with Tessellation

[dink]

Hi all!

I recently made an environment in Untiy using ASE. I loved the plug-in very much and it was incredibly useful and easy to use when putting my scene together.

However, I did run into a couple of issues when I tried to add tessellation to my materials, which ended up forcing me to not use tessellation for my scene. I'm not sure if these are bugs, or if this is common knowledge and not considered an issue, or if I'm just using the feature wrong. The video, although a little long, showcases the issues visually. But I'll give a quick explanation of what seems to be going on here:

1: If Tessellation is enabled when the rendering path is Deferred, the shader will err and goes purple/pink. The only way I could get Tessellation to be enabled was to be in the Forward rendering path. I don't know if this is intentional or if this is common knowledge that, perhaps Unity only can do Tessellation in Forward, but it seems odd to me.

2: Any texture with its own UV coordinate that is plugged into Local Vertex Offset will multiply or add that tiling value on top of the UV coordinate values on all other textures in the same UV set.
So for example, in my scene, and you can see this in the video, I have most of my textures on UV set 2 tiling 45 times on U and V, and I have another texture sample also on UV set 2 that is set to tile at 9 on U and V. But when I plug my height map into Local Vertex Offset, which has the same UV coordinate node as most of my other textures (tiling at 45), this number 45 gets added on top of the 45 that's already being done, so it tiles most of my textures 90 times, and tiles my other texture sample 54 times because it's being added on top of 9.

Here's the video where I demonstrate the issues. Forgive my sluggishness in the video, it was quite late when I recorded! Was just before I went to bed.
https://youtu.be/GlAJjCrxcjk

You can also see the environment here at the Unity forums: https://forum.unity.com/threads/environment-the-stakeout.508519/

Or at my website here: http://www.alexjamerson.com/

Has anyone else run into these issues before?

Thanks in advanced!!
-Alex

[Amplify_Borba]

Hello Alex, thank you for getting in touch and for providing such detailed information!

We've recently released a new ASE build on our website, v1.4.0.005 – 13th December 2017, which might solve some of the issues you're having.

I recommend adjusting your UV use as UV Set 2 is used by Realtime GI Data, you will have to set it to 4 in ASE in order to use the fourth channel, which might solve some of the tessellation issues you're having.

0ZVI7es.png (8.74 KiB) Viewed 3436 times

Regarding the Deferred rendering path, it might be an Unity related issue, we're going to investigate this and will get back in touch as soon as we have any developments.

Would it be possible for you to export your shader, material and textures via Unity and send it over to [email protected] so that we can debug it on our side? Thanks!

[dink]

Hey Borba! Thanks so much for the reply.

So based on your suggestions, I went in and tried some of these changes and I've come out the other side with an operable displacement, but with some caveats.

First, it seems like the labeling in the Shader Editor for the UVs is one number off from that figure - which is technically right. The Shader Editor doesn't label the first UV parameter as 0, it labels it as 1. So it's UV1-4, which means UV2 is for the baked lightmap and UV3 is for the realtime GI. This isn't a bug or a problem, I just wanted to clarify this language for the sake of communication. This is actually why I was using UV2 for all my tiling materials, because my tiling UV map doesn't have any overlap and you shouldn't have overlap for a baked lightmap.



Second, in the past I've had issues with using all 4 channels in Unity, so I have always been adverse to using them. But since you suggested it, I went ahead and went for it. And it seems like if I keep all my tiling textures on UV2, as they were before, but I make my Displacement on UV3, this solves the issue and my displacement tiling works correctly. Of course the problem with this is that we're now using 2 UV maps even though it's the same coordinate space, and one should be able to use a single UV map for this instead of 2 identical UV maps. But this is because of the tiling override I demonstrated in the video.

I tried to put everything on UV4, but the same tiling issue occurs. This seems like the best thing to do is to keep Displacement textures on their own UV channel to prevent this tiling override issue.

So now that this issue has been resolved for now, I think the next thing to tackle for me is the issue of Displacement cracking.



There's got to be a way to solve displacement cracking in a procedural way. If there's any way to get a mask of the UV seams procedurally, then I think I have a method.

[dink]

Oh! And yes I could see about packing up the files to send to you guys.

[dink]

Good news and bad news!! And a new video!

Bad news: I got the odd shader error when using Tessellation again. This time I was not in Deferred rendering mode, which is what seemed to cause the issue last time. I was able to quickly fix it, as you can see in the video link below.

Good news: In my last post I talked about the problem with Displacement cracking and my simple theory to fix it. I tested it using a bespoke mask I painted and it works great. I'm adding a Feature Request/Suggestion here. You can also see that in the video.

https://youtu.be/RagVtpbn_fM

[Amplify_Borba]

First, it seems like the labeling in the Shader Editor for the UVs is one number off from that figure - which is technically right. The Shader Editor doesn't label the first UV parameter as 0, it labels it as 1. So it's UV1-4, which means UV2 is for the baked lightmap and UV3 is for the realtime GI. This isn't a bug or a problem, I just wanted to clarify this language for the sake of communication. This is actually why I was using UV2 for all my tiling materials, because my tiling UV map doesn't have any overlap and you shouldn't have overlap for a baked lightmap.

Indeed, this may cause some confusion given the image we shared (older reference). To our knowledge, the order is maintained but this could always change with future Unity lighting related updated.

Second, in the past I've had issues with using all 4 channels in Unity, so I have always been adverse to using them. But since you suggested it, I went ahead and went for it. And it seems like if I keep all my tiling textures on UV2, as they were before, but I make my Displacement on UV3, this solves the issue and my displacement tiling works correctly. Of course the problem with this is that we're now using 2 UV maps even though it's the same coordinate space, and one should be able to use a single UV map for this instead of 2 identical UV maps. But this is because of the tiling override I demonstrated in the video.
I tried to put everything on UV4, but the same tiling issue occurs. This seems like the best thing to do is to keep Displacement textures on their own UV channel to prevent this tiling override issue.

We would need to examine it for this one. Lighting use or even togging the "Generate Lightmap UV's" option in your model could mess up your existing layout.

There's got to be a way to solve displacement cracking in a procedural way. If there's any way to get a mask of the UV seams procedurally, then I think I have a method.

The cracking, so to speak, is caused by a specific Unity limitation. This is present when using ASE because we pull directly from what Unity provides, sort of speak. There are some workaround, hopefully we can provide our own implementation in future version.

Here's an example of a working solution: https://www.assetstore.unity3d.com/en/#!/content/52518

Bad news: I got the odd shader error when using Tessellation again. This time I was not in Deferred rendering mode, which is what seemed to cause the issue last time. I was able to quickly fix it, as you can see in the video link below.

Are you using the latest ASE version available via our website? We could really use a sample for further examination.

Good news: In my last post I talked about the problem with Displacement cracking and my simple theory to fix it. I tested it using a bespoke mask I painted and it works great. I'm adding a Feature Request/Suggestion here. You can also see that in the video.

Cool temporary solution, and thank you for providing the additional details!

[dink]

We would need to examine it for this one. Lighting use or even togging the "Generate Lightmap UV's" option in your model could mess up your existing layout.

I sent a unity package to the Support e-mail you provided. Let me know if there's anything else I can do to help.

The cracking, so to speak, is caused by a specific Unity limitation. This is present when using ASE because we pull directly from what Unity provides, sort of speak. There are some workaround, hopefully we can provide our own implementation in future version.

Here's an example of a working solution: https://www.assetstore.unity3d.com/en/#!/content/52518

I've seen that plugin, but I have not tried it. I do see it says it includes the full source code, so I suppose one could buy it and reverse engineer it for their own shaders. I certainly wouldn't want to lose the versatility of working in ASE in order to have crack-free tessellation. I would prefer to have both!

Are you using the latest ASE version available via our website? We could really use a sample for further examination.

I am. Yet the issue seems to persist. EDIT: I take that back, I see there's a slightly newer version on your direct website (as you mentioned) and I only had the latest version from the asset store. Downloading now to give it a shot.

Good news: In my last post I talked about the problem with Displacement cracking and my simple theory to fix it. I tested it using a bespoke mask I painted and it works great. I'm adding a Feature Request/Suggestion here. You can also see that in the video.

Cool temporary solution, and thank you for providing the additional details!

If there's any way to get a mask from the UV borders procedurally in ASE, this 'workaround' would be a bit better than how I'm doing it right now. I just don't know if there's a way to do it.

Either way, I don't have time to look for a solution at the moment as I've moved on from my last environment and am working on my next couple of projects.

Thanks for your help, Borba. I look forward to hearing more, and if there's anything I can do to help, I can certainly make time.

[Amplify_Borba]

I sent a unity package to the Support e-mail you provided. Let me know if there's anything else I can do to help.

Thanks for taking the time to provide us a sample!
Regarding the tessellation issues, upon taking a quick look at the package I noticed that you have GPU Instancing on for wood_paneling02_mat, you must turn it off for your materials as follows:

1. Open shader, disable tessellation and compile
2. Select Material(s) using the shader and disable GPU instancing
3. Open shader, enable tessellation and compile

image (9).png (9.65 KiB) Viewed 3381 times

In regards to tiling and masking, we can't know for sure if the project is set up as the sample but the model shared is set to Static and the UV Coordinate nodes in your shader are set to 1 and 2. Channel 2 is used for baked lightmap data. Additionally, we are unable to fully test your shader as it seems to be missing a few Shader Functions.

If there's any way to get a mask from the UV borders procedurally in ASE, this 'workaround' would be a bit better than how I'm doing it right now. I just don't know if there's a way to do it.

Either way, I don't have time to look for a solution at the moment as I've moved on from my last environment and am working on my next couple of projects.

Thanks for your help, Borba. I look forward to hearing more, and if there's anything I can do to help, I can certainly make time.

Always glad to be of assistance, and thank you for your detailed reports and feedback, we will definitely look into this situation and hopefully we can implement an alternative tessellation solution at a later stage.