tfp.dev

I'm working on a level editor for my game engine and I need an outline effect to indicate the user's selection.

I think it looks cool, but how's it different?

Jump flood algorithm

I've used the typical "inverse hull" approach in the past, as well as Sobel filters-- but was never truly happy with the result.

These traditional approaches look decent-- but they have issues, depending on your taste.

• Inverse hull outlines have inconsistent thickness and artifacts on certain geometries.
• Sobel-based outlines have inconsistent thickness and are just too thin.

However, I recently ran into a massively popular Medium post by Ben Golus describing a GPU implementation of jump-flood algorithm for outlines. The results are really good.

Without getting into details, JFA supports arbitrary thickness with minimal performance cost and consistent width outlines no matter what geometry is outlined or what projection is used by the camera.

A quick integration..

There's a crate from @dataphract that implements JFA. I patched it up to be Bevy 0.12 compatible, and..

Nice, they look pretty good :)

To be honest, these outlines don't really take advantage of JFA since they're so thin, but this post isn't really about JFA-- you can find all that on Ben's post.

This post is about a humble UX nit I wanted to solve with the outlines in my editor.

Why are editor outlines special?

In a game, outlines are just meant to draw attention to an area of interest-- but in an editor, they should also clearly separate objects.

A typical JFA outline implementation won't separate individual objects.

If we take a look from the side, it's more apparent.

Notice how all three objects share one outline. For an editor, I need to communicate that three distinct objects are selected here.

This is actually a super common issue with outlining shaders.

Check out what Blender does in a similar case.

In both screenshots, I have the cuboid in the center selected, and the floor behind it is also selected.

Blender's outlines won't appear if the selected object is on top of another selected object, but my editor will produce outlines for both objects, and by design, the interior object's outline will be slightly thinner.

Let's quickly talk about how someone might achieve something like this.

Separating objects

First, let's get down to the reason why these outlines merge together.

Let's take a look at what happens in XCode's Metal debugger.

This is the mask texture that the JFA pass will outline. This mask texture has absolutely no information segmenting objects-- so no matter what, we just can't get the results we want.

Luckily, I am pretty familiar with this crate's source after upgrading it to Bevy 0.12, so let's add some info.

I'm just hacking the origin of the object in clip space into the color of each pixel shaded by the object for now.

I think some kind of object identity makes more sense, but for now-- that's why they show up as whacky colors.

From here, the general idea is:

1. Render each object, writing their clip-space origin into the frame.
2. Run a pass using a simple Sobel-like filter to produce a thin outline.
3. Using JFA, "outline the thin outline" to make it as thick as we want.
4. Composite the outlines onto the frame.

This gets us here, see how the two objects have a thin yellow line separating them now-- not just one blob like the last screenshot.

It looks great, but I also want to draw occluded outlines, like these purple lines I drew in:

I think rendering those would help make the editor easier to use.

Rendering occluded outlines

To render occluded outlines, I came up with a cheap trick-- just use additive blending when rendering the outline mask.

This ensures that each object will be outlined correctly-- even if it's occluded-- since each object blended on top of one another should produce a "unique-enough" color that the Sobel filter can pick up on.

I ended up tweaking the Sobel filter to look for any difference in adjacent pixels, and the result is pretty good.

Sidenote, the trick here really is cheap. It seems easy to have collisions in object identity, overflow, etc-- but it's good enough for now.

Open to any suggestions-- blending is the only way I can think to do it in one pass-- and I know blending hardware is usually still fixed function-- so not many options.

Now that we have these fast, thin outlines, we can use JFA to thicken them up as thick as we'd like.

Wrapping it up

Then we just composite everything on top, and here's an animation of the entire process running.

The final compositing pass also does some AA magic (which makes the outlines appear thinner) and fills the center of the selected object with a yellow tint.

And there you have it-- each item distinctly outlined, and visible through one another. Perfect for an editor!

Next steps

In the future, I'd like for the occluded outlines to show up as dashed lines. This is possible too, by sampling against the depth buffer during the compositing pass to check which lines are occluded.

Not crazy complicated work, but it's an interesting problem-- however minor-- that I have seen ignored in almost every 3D editor I've ever used. Wanted to solve it in my own editor.