Linear Algebra and BrushCue: A Peak Under the Hood
When I first encountered linear algebra and matrices in chool, the appear pointless. Unbenounced to me, this type of math powers swaths of computer graphics — not to mention, all of AI. While the matrices that underly programs such as BrushCue are hidden, learning how they will use will allow you to predict how these tools work and understanding how to best use these tools. Most of the application of linear algebra is just multiplying matrices together. In this post, we will demonstrate to how linear algebra powers colors and positions in BrushCue.
Colors
Linear algebra has filled many books. For the purposes here, we just need a tiny sliver of it, the matrix. A matrix just being a grid of numbers. We can multiply matrices against each other. Many color modifications that are common are expressed internally as a matrix multiplication against the RGB values.
There are a lot of additions and multiplications here! The key point is that this defines an output based on the input and a bunch of multiplications. The top row of the matrix controls the output red value, the second row defines the the green , the third the blue and the fourth the alpha. Each column defines how much from the other components combine in the output.
To examine what we can do, let's run through some examples. We will see the example using Sargent painting as well as one example color, a green sampled from the riverbank.
TODO example of toning down the blues TODO example of color blend TODO example of increase intensity TODO example of swap channels
You may be thinking. What's the point? If I wanted to scale the blue by 0.7 that is just one multiplication, you are doing 16, are you stupid? Well, who's to say. But, there is a point to doing all this work. Talking all of the matrices from above and multiplying them together in reverse order we get:
SHOW MATRIX
Using that matrix against our initial values, we get the following:
TODO show this example.
Taking this composite matrix and applying it against our original values gives the exact same result as applying each of the above steps one after another. This property is why matrices are so powerful and explains the ubiuity of their use in computer garaphics and AI. Our example image agove has dimensions 3000x2009. This means 6,027,000 times per output we need to apply our change. Expressing those steps as matrix multiplications, we can guarentee that the maximum number of multiplications is 16. Additionally, we often need to perform a color conversion as part of these edits, these color conversions contain matrix multiplciations themselves. Therefore, many modifications that use a linear transformation end up being completely free.
Almost all media created with BrushCue has a matrix multiplciation as part of the output — at least to convert between color types. Operators that directlyty use matrix multiplication include saturation adjust, brightness adjust and grayscale.
Location
Another area where matrices dominate is determing locations. In BrushCue, this is the Transform2 type. For any given drawing or shape in there is a corresponding transform that encodes it's position, rotation and scale.
Previously, we had r, g, b, a and multiplied it by a 4x4 marix. Now things are simplified, we only have x and y coordinates and this means that we just have a 2x2 matrix yielding the following formula:
TODO give formula here
In this example, we have a square over an orange background. To note, the coordinate system for BrushCue and many other editors is reversed in the y-direction from what you probably had in school — positive y going down. Let's start with the square at the top left.
TODO give the scale example
TODO give a rotation example.
To move the square down and to the right we... Wait, with our current approach there is no way to move something left, right, up or down. We need something else.
From our discussion of colors, you will remmeber that we never touched the alpha column in the last row. It remained 1. We could have modified the alpha, but we never did. Too bad we don't have this for position, we could add to the x and y doing this.
Well actually, this is exactly how we solve the issue of having no translation ability. We simply pop 1 into our coordinate and proceed with instead of 2x2 matrices, with 3x3 ones. Miracuolously, we ignore that 3rd element and the math ends up working out. As an example, let's now move our square down and to the right.
TODO TRANSLATION EXAMPLE
That works, just using the items in the rightmost column we can move our item without modifying the scale or rotation.
What if we wanted to rotate around the center? If we tried rotation now, we would get something like this:
ROTATION EXAMPLE
The origin of the canvas is the point of rotation. To rotate around the center, first we must move it to the center, then perform the rotation, and then move it back.
EXAMPLE DOING THESE THREE STEPS.
This techniques work for other points too. If we wanted to rotate around the bottom left corner, we could do that by moving that point to the center of the canvas first.
EXAMPLE DOING THESE STEPS.
Similar to colors, when we multiply all of the matrixes together in reverse order and apply it to our original we get the same end result as applying these steps one after another. This means that no matter how much you move your objects around, the GPU has to do the same amount of work.