Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Impeller] use varying interpolation to compute linear gradients. #148651

Open
jonahwilliams opened this issue May 19, 2024 · 2 comments
Open

[Impeller] use varying interpolation to compute linear gradients. #148651

jonahwilliams opened this issue May 19, 2024 · 2 comments
Labels
e: impeller Impeller rendering backend issues and features requests P3 Issues that are less important to the Flutter project team-engine Owned by Engine team triaged-engine Triaged by Engine team

Comments

@jonahwilliams
Copy link
Member

See #148496 for a motivating example.

Background

Impeller's current gradient shader is quite slow (I know I wrote it), due to accomidating an extremely flexible gradient API into a single shader. Flutter has approximately the same API as https://developer.mozilla.org/en-US/docs/Web/CSS/gradient/linear-gradient , which supports an arbitrary number of colors, color stops, potentially inset in the rendered geometry. Skia generates new gradient shaders at runtime based on the number of colors and tile mode, but this isn't a viable option with Impeller. We can do better than this anyway.

Overview

Use varying interpolation to compute the gradient colors for us. This requires us to potentially change the vertices associated with a draw so that there is sufficient information to perform the interpolation.

For example, suppose we are applying a linear gradient with 2 colors to a rectangle (red and blue), with the start and end at the top center and bottom center (respectively).

1 -------S-------2
|                |
|                |
3--------E-------4

It is apparent that assigning red to varyings for vertices 1, 2 and blue to vertices 3, 4 will lead to exactly the interpolation needed by the gradient. Thus the fragment stage can be simplified to computing the premultiplied color (our gradients interpolate in unpremul space) and applying the ordered dithering.

What if we have more than two colors?

When there are more than two colors, we must insert additional vertices for our interpolation. In the case of 3 colors (red, green, blue) and the same rectangle, we need to insert new vertices to assign the green color 2, at the correct location determined by the color stop value (if unspecified 0.5). This is equivalent to dividing the rectangle in half

1 -------S-------2
|        |       |
3--------*-------4
|        |       |
5--------E-------6

In this case we have red at vertices 1, 2, green at 3, 4, and blue at 5, 6. This process can be repeated for an arbitrary number of divisions, provided that S/E are located on the coverage rect. The division point can be computed by using the stop value to lerp start and end ( point_i = (1.0 - t_i) * start + t_i * end ).

What if the S/E are located elsewhere?

IF the start/end are horizontal, then I believe we could handle this by rotating the shape such that it was vertical, computing the triangles, and then rotating back. Likewise if the directions are reversed - these are all 90, 180, 270 degree rotations.

If The start and end are diagonal in some way, things get more interesting. We have a few possibilities here, but I haven't thought through all of the options.

What if the S/E are inside the rectangle

Then we need to handle the tile mode, which computes a value of t similar to a texture sampler. For example, t may clamp, or repeat, or be mirrored. I think the best option would be to expand the geometry and compute t for each stop. If its decal though, we can avoid this.

What if the S/E are outside the rectangle

Assuming the existing conditions are met, we treat the starting point on the rectangle as whatever the value of t would be.

What if the shape isn't a rectangle

With StC, all shapes can be rectangles so this isn't a problem. Draw the actual shape with the stencil buffer and then the coverage rect can be substituted for the geometry rect.

What about other gradient types?

These are less common. I think we could do concentric circles for the radial gradient. Conical/Sweep 🤷

@jonahwilliams jonahwilliams added e: impeller Impeller rendering backend issues and features requests team-engine Owned by Engine team labels May 19, 2024
@jonahwilliams
Copy link
Member Author

For conical gradients we can do triangle wedges divided from a covering circle. Two point conical still 🤷

@jonahwilliams jonahwilliams added P3 Issues that are less important to the Flutter project triaged-engine Triaged by Engine team labels May 20, 2024
@bdero
Copy link
Member

bdero commented May 28, 2024

If using varying interpolation for mapping gradient colors, the effect transform would need to be applied to the cover rectangle geometry in order to map the colors properly, which would involve handling some corner cases with fallback.

  • Tiny scale? Large volume of geometry.
  • Extreme perspective/tiny projective coordinates? Precision issues with adjusted geometry.

But I think this can be a great approach when limited to effect transforms that are translation+rotation only (probably the overwhelmingly common case).

For the initial pass of this, I would strongly recommend limiting scope for the initial implementation to:

  • Identity effect transforms. Writing an algorithm to rejigger the geometry for translation+rotation effect transforms could be done separately.
  • Doing a 2-draw stencil-then-cover approach (in order to support arbitrary path types). A specialized 1-draw solution could later be added for some rectangle geometry cases, if desired.

To implement this, you won't be able to use the ColorSourceContents::DrawGeometry as-is, because it forms a position-only "cover" rectangle from the local bounds of the original geometry. So another variation would be needed that allows the caller more control over producing the "cover" stage geometry.

We're already a little bit callback crazy, but one possibility would be to add an optional callback to ColorSourceContents::DrawGeometry that produces the cover geometry.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
e: impeller Impeller rendering backend issues and features requests P3 Issues that are less important to the Flutter project team-engine Owned by Engine team triaged-engine Triaged by Engine team
Projects
None yet
Development

No branches or pull requests

2 participants