Self-hosted OpenGL ES Development on Ubuntu Touch

Orange Pi Development Boards

Blu wrote BQ Aquaris M10 Ubuntu Edition review – from a developer’s perspective – last year, and now is back with a new post explaining how to develop and deploy OpenGL ES applications directly on the Ubuntu Touch tablet.

Ever since I started using a BQ M10 for console apps development on the go I’ve been wanting to get something, well, flashier going on that tablet. Since I’m a graphics developer by trade and by heart, GLES was the next step on the Ubuntu Touch for me. This article is about writing, building and deploying GLES code on Ubuntu Touch itself, sans a desktop PC. Keep that in mind if some procedure seems unrefined or straight primitive to you – for one, I’m a primitive person, but some tools available on the desktop are, in my opinion, impractical on the Touch itself. That means no QtCreator today, nor Qt, for that matter.

The display of any contemporary Ubuntu Touch device is powered by Mir – a modern compositor/surface manager taking care of all (rectangular-ish) things eventually appearing on screen. We won’t be delving much into Mir beyond obtaining an EGL context (EGL being the binding layer between GLES and the native windowing system). But enough ado – let’s get to work.

Preparations for doing GLES on a Ubuntu Touch box:


The above, as of the time of this writing, should provide you with gcc/g++-4.9, make and gdb-7.9, among other things. The last package and its dependencies provide you with up-to-date Mir headers. Git comes out of the box, IIRC, but if it’s missing just apt-get it.

We need a primer to step on, so here’s my adaptation of Don Bright’s Mir/GLES adaptation of Joe Groff’s OpenGL tutorials, using Daniel van Vugt’s Mir/EGL examples (yes, that’s a quite a chain-work):


I’ve taken the liberty to expand on the work of those gentlemen by bringing the Mir integration up to date, handling Touch’s novelty Desktop Mode and throwing in my own dusty GLES sample code, for good measure.

To build and install the primer, just do:


That will provide you with an original police-car flashing-lights primer. An alternative primer featuring tangential-space bump-mapping can be built by passing arg ‘guest’ to the build script:


Both versions of the primer use a fundamentally identical interface — a resource-initialization procedure and a frame-drawing procedure, so it’s not much of an effort to use the respective routines from either primers in the framework of the host app hello.cpp, and thus get a running render loop.

A few words about the peculiarities of the GLES development for Ubuntu Touch. It took me some time to show anything on screen, despite the fact I had a valid draw context and a render loop soon after the primer was building successfully. The reason is Unity8 on the Touch will not simply let you run a window-painting app from the terminal – you would get your Mir and EGL contexts alright, but the target surface will never be composited to the screen of the device upon eglSwapBuffers() unless you take certain actions. You have two alternatives here:

  • Produce a valid Click package from your app and subsequently install that to the Apps pane (what our build script does), where you can launch from an icon, or…
  • Use a launcher app to start your window app (info courtesy of Daniel van Vugt):

Unfortunately the second (much quicker and convenient) approach is not currently usable due to a bug, so we’ll stick with the first. Any command-line args we’d want to pass to the app will need to be written to the app’s .desktop file, which can be found at the official app location after installation:


In that file, set the desired args on the ‘Exec’ line, like this:


Another peculiarity was that in Desktop Mode the app window does a classical ‘zoom to full size’ animation at start. Nothing extraordinary in that, if not for the fact that the Mir surface itself resizes along with the window. Now, a default viewport in a GLES context spans the geometry of the target surface at the time of its creation, which, in our case, is the start of the window-zoom animation, with its tiny surface geometry. One needs to wait for the zoom animation to finish, and then set the viewport geometry to the final geometry of the Mir surface, or live with a post-stamp-sized output in the lower left corner of the window, if the viewport is left unchanged.

Once we get past those teething hurdles we actually get quite a nicely behaving full-screen app on our hands – it composites smoothly with all other Ubuntu Touch desktop elements like the Launcher tab at the desktop’s left edge and the pull-down Indicator pane on right (see screenshot). Our app even does live output to the Scopes selector screen (i.e. the task-switching screen) — behold the miracles of modern-day screen compositors! ; )

Click for Original Size (1920×1080)

But hey, don’t just take my word for it – try out GLES coding on a Ubuntu Touch device – you have the basics covered:

  • App’s rendering loop and the entirety of the flashing-screen primer are in hello.cpp
  • Mir context creation and subsequent EGL context binding are in eglapp.cpp
  • Bump-mapping primer is entirely in app_sphere.cpp
  • Various helpers are spread across util_* TUs and hello.cpp
  • All files necessary for the generation of the Click package are in resource folder.

In conclusion, self-sustained development on the Ubuntu Touch is a perfectly viable scenario (take that, iOS!). Moreover, the GPU in the BQ M10 turned out to have a very nice modern GLES3 (3.1) stack – see excerpts from the app logs below. Actually, this is my first portable device with a GLES 3.1 stack, so I haven’t started using it properly yet — the GLES2 primer above doesn’t make use of the new functionality.

If I have to complain about something from the development of this primer, it’d be that I couldn’t use my arm64 code on the primer, since there are only armhf (32-bit) EGL/GLES libraries available for the Touch. So 64-bit code on the Ubuntu Touch remains in console land for now.

Excerpts from the primer logs:

egl version, vendor, extensions:

1.4 Android META-EGL
Android
EGL_KHR_get_all_proc_addresses EGL_ANDROID_presentation_time EGL_KHR_image EGL_KHR_image_base EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_cubemap_image EGL_KHR_gl_renderbuffer_image EGL_KHR_fence_sync EGL_KHR_create_context EGL_ANDROID_image_native_buffer EGL_KHR_wait_sync EGL_ANDROID_recordable EGL_HYBRIS_native_buffer2 EGL_HYBRIS_WL_acquire_native_buffer EGL_WL_bind_wayland_display

gl version, vendor, renderer, glsl version, extensions:

OpenGL ES 2.0 (OpenGL ES 3.1)
ARM
Mali-T720
OpenGL ES GLSL ES 3.10
GL_EXT_debug_marker GL_ARM_rgba8 GL_ARM_mali_shader_binary GL_OES_depth24 GL_OES_depth_texture GL_OES_depth_texture_cube_map GL_OES_packed_depth_stencil GL_OES_rgb8_rgba8 GL_EXT_read_format_bgra GL_OES_compressed_paletted_texture GL_OES_compressed_ETC1_RGB8_texture GL_OES_standard_derivatives GL_OES_EGL_image GL_OES_EGL_image_external GL_OES_EGL_sync GL_OES_texture_npot GL_OES_vertex_half_float GL_OES_required_internalformat GL_OES_vertex_array_object GL_OES_mapbuffer GL_EXT_texture_format_BGRA8888 GL_EXT_texture_rg GL_EXT_texture_type_2_10_10_10_REV GL_OES_fbo_render_mipmap GL_OES_element_index_uint GL_EXT_shadow_samplers GL_OES_texture_compression_astc GL_KHR_texture_compression_astc_ldr GL_KHR_texture_compression_astc_hdr GL_KHR_debug GL_EXT_occlusion_query_boolean GL_EXT_disjoint_timer_query GL_EXT_blend_minmax GL_EXT_discard_framebuffer GL_OES_get_program_binary GL_OES_texture_3D GL_EXT_texture_storage GL_EXT_multisampled_render_to_texture GL_OES_surfaceless_context GL_OES_texture_stencil8 GL_EXT_shader_pixel_local_storage GL_ARM_shader_framebuffer_fetch GL_ARM_shader_framebuffer_fetch_depth_stencil GL_ARM_mali_program_binary GL_EXT_sRGB GL_EXT_sRGB_write_control GL_EXT_texture_sRGB_decode GL_KHR_blend_equation_advanced GL_OES_texture_storage_multisample_2d_array GL_OES_shader_image_atomic

GL_MAX_TEXTURE_SIZE: 8192
GL_MAX_CUBE_MAP_TEXTURE_SIZE: 4096
GL_MAX_VIEWPORT_DIMS: 8192, 8192
GL_MAX_RENDERBUFFER_SIZE: 8192
GL_MAX_VERTEX_ATTRIBS: 16
GL_MAX_VERTEX_UNIFORM_VECTORS: 1024
GL_MAX_VARYING_VECTORS: 15
GL_MAX_FRAGMENT_UNIFORM_VECTORS: 1024
GL_MAX_COMBINED_TEXTURE_IMAGE_UNITS: 48
GL_MAX_VERTEX_TEXTURE_IMAGE_UNITS: 16
GL_MAX_TEXTURE_IMAGE_UNITS: 16

Support CNX Software - Donate via PayPal or become a Patron on Patreon

5
Leave a Reply

avatar
5 Comment threads
0 Thread replies
1 Followers
 
Most reacted comment
Hottest comment thread
2 Comment authors
bluArthur Frayn Recent comment authors
  Subscribe  
newest oldest most voted
Notify of
Arthur Frayn
Guest
Arthur Frayn

Some might ask “Is it worth investing money and time in Mir when Wayland is the way to go?”

Will GLES code written for Mir be fully compatible with Wayland?

blu
Guest
blu

@Arthur Frayn
The only difference is in the “glue” EGL surface creation – once an EGL surface & context are set up everything else is the same across all EGL-supported platforms. Now, I’m only vaguely familiar with Wayland (which is still more than the zero familiarity I had with Mir before starting this primer), but the code for surface setup is not that different between the two client-side APIs. BTW, there appears to be a Wayland wrapper .so in the same folder containing the hybris .so’s [1] on BQ M10 OTA-14, and by briefly looking at it it appears to have most entry points necessary for the task. I just didn’t want to step into ‘it might or might not work’ territory and went for the one proven-working solution on Ubuntu Touch. Then even that one took me long enough to get it going : )

[1] GLES goes over hybris on Ubuntu Touch, for needs to communicate with android .so’s — it’s a long story.

Arthur Frayn
Guest
Arthur Frayn

@blu
Thanks for the explanation.

It is probably critical to the future of Canonical (could Mir + Unity on tablets and phones finally be the path to profitability?) that Mir becomes the accepted replacement for Xorg, whereas the proponents of Wayland will be hoping that Wayland trounces Mir in the same way that systemd killed upstart which was created by Canonical . No doubt Redhat (a major backer of Wayland) would welcome that as another blow to one of their competitors.

blu
Guest
blu

@Arthur Frayn
The good ol’ politics aspect.. I’m for standards as much as the next guy, but standards alone do not ensure smooth productivity – implementations of those standards and long-term guidance for those standards matter just as much. As long as there’s a sufficient degree of functional equivalence between Mir’s and Waylan’s clinet sides, AND the implementations don’t do something stupid behind the scenes (like breaking things just because), I don’t mind. At the end of the day, if a couple of hundred lines of boilerplate code, written once and forgotten about for the next few years, can get me Mir functionality equivalence with Wayland, that’s perfectly acceptable from my POV. So I don’t have a fundamental issue with Mir not being Wayland, while on the tech side I’ve seen better surface managers/compositors, but I’ve also seen worse. One thing for which I do have an issue with Canonical, though, is their sticking with 32-bit userland on Ubuntu Touch – that does make me go through hoops again and again, or plain dissuades me from doing some things on their platform.

Arthur Frayn
Guest
Arthur Frayn

Blu explained “long-term guidance for those standards matter.”

The underlying issue is whether long-term guidance is financially feasible.

Almost 3 months later and the official announcement today is that Mir has now been declared dead, Ubuntu Touch has been declared dead, and Convergence has been declared dead. Ubuntu is to switch to the Red Hat model of Gnome desktop on Wayland server and all further development (from Canonical) has ceased or is to end very soon.

Economics (ie Ubuntu phones are not selling, so no profit to be made there) trumps all.