Why we created GPUEffect

Here at Frontback’s, we love the Open Source community and we believe open sourced libraries are not only great because they are developed by amazing developers but also because they are tested by a large number of people who report bugs and contribute to them. That is why we trust many of them in our code and contribute back whenever we can.

Among those libraries was GPUImage developed by CyberAgent. We used it to apply filters on images both after the picture was taken and during the live preview. This library is great and takes a lot of the burden of using OpenGL off the developer’s shoulders.

Yet, it is very opinionated and tries to do too much for you. Therefore, to be able to use it across our application, we had to trick the library and we reached its limits.
For example, when using GPUImageFilterGroup, the library assumes each filter’s output will be the input of the next filter. This makes it very difficult to combine filters in the background that will not be used immediately.
Another example is how inputs are received, in the case of GPUImageTwoInputFilter, the second input has to be a bitmap even though, behind the scene, this bitmap is sent into an OpenGL texture. This had lead us to trick the library to be able to realize some effects which needed to combine some filters’ output with others.

That is why we decided to inspire ourselves from this amazing library and come up with our own that would be more flexible.

Here comes GPUEffect !

First of all, this library has been written in Kotlin because, I mean, obviously ! :) We believe this greatly improves the readability of the code both the Kotlin code and the GLSL code (using the """ notation) but also using operator overloading such as + and += .

Then, we tried to stay as close to OpenGL as possible but abstracting it at the same time. This has lead to interface such as Texture, Program and FrameBuffer. The idea was pretty simple : You should be able to use those as part of the GPUEffect and GPUMultiEffect classes or as completely independent classes in your code.

OpenGL is hard to master and understand, it’s even harder to debug. That’s why we decided to fail often and fail early. GPUEffect will try to detect inconsistencies (E.g.: a Program not loading correctly) and throw exceptions to help you detect errors at their sources and not end up with a black screen without any explanation as of why.

We also decided to stick with the naming convention introduced in iOS’ GPUImage (from which Android’s GPUImage is inspired) for shaders variables. It’s therefore not too hard to convert a shader from either iOS’ or Android’s GPUImage. We also provided some constants for those to make the code more robust (Check out GLSLProgram’s code).

Finally, we know there is room for improvements. For example, the default implementation of GPUMultiEffect assumes that if you pass it Effects without Input and FrameBuffer, it will chain them. It’s up to you to either override the onDraw method or ensure everything upfront. That’s the cost of flexibility.

What’s next ?

Head to the GitHub repository and look at the read me and the sample code. Try it for yourself and send us feedback :)