That article wasn't really accurate, most clients aren't using the X drawing commands anymore. The "vast majority of cases" has moved to using GL or Vulkan, which also doesn't serialize over the network.
The number of Vulkan or GL programs running in a desktop Linux is close to zero (or at most 1: the compositor). Browsers vary but on most setups they use practically nothing with GL/Vulkan (blame drivers). Gtk+ (even 3) still sends X drawing commands. Qt5 does not by default (it renders by itself, without any GL/Vulkan) but can still be configured to use X.
Qt has Qt Quick and QGraphicsScene which will use a GL backend.
GTK4 has a GL backend by default.
Web browsers and Electron use Skia which has GL and Vulkan backends.
Pretty much every video game is using GL and Vulkan already, or Proton which translates D3D to Vulkan.
Video players are using VA-API directly instead of XvMC.
The only thing in your list that doesn't have a GPU accelerated backend is GTK3, which GNOME is currently migrating away from to use GTK4. To my knowledge GTK3 also tries really hard to use Cairo on the client side for as many things as possible, and generally avoids using X drawing commands. I think it should be fairly obvious by now that graphics developers will prefer to use accelerated APIs whenever possible and don't care at all for "network transparency" if it means they have to use an outdated and mostly inadequate API.
I have absolutely no single program using Qt Quick on my desktop, nor I can actually remember the name of any single one. Which is funny since I actually was a Qt Quick developer in the past and can tell you a shitton of embedded platforms (i.e. cars) that use Qt Quick. Just no desktop programs.
I do not have any single Gtk4+ program on my desktop, and I use a quite up-to-date rolling distribution.
Skia having a GL backend does not mean it is used. Firefox does not use it on almost any Linux platform. It still blacklists even the FOSS drivers. Tested today with upstream v93 and using radeon opensource driver.
No idea why bring VA-API into this. It is practically the same thing as XvMC with a much more generic API. It is also not necessarily Vulkan or GL. Ironically this is also one area where intelligent remoting tools win (they can send the original compressed video stream down the wire. RDP does it), where a plain H264 stream will be forced to recompress, increasing latency at best.
Cairo is also backed by X rendering and this is the default.
Basically, if I ignore the compositor and games, I do not have _any single program_ on my system which uses GL or Vulkan for 2d graphics. Not surprising: in my experience, using GL for 2D graphics (i.e. arcs, lines, etc.) usually ends up in a big slowdown -- and a big increase in memory usage and crashes. It is mostly worth only for when you do pure texture manipulation like scaling, rotating, etc. i.e. final compositing or layering.
And, if, in addition to the above, I ignore the browser, and set the corresponding Qt flag, then _all programs_ in my system render using X rendering.
Easily tested because the performance difference is abysmal when using NX.
I would say that's probably something specific to your set up, and you may want to try some more apps, or ask those developers of your apps what their plans are. If you use Plasma, then you are using some Qt Quick software. Currently only the GNOME extensions panel is ported to GTK4, but more things are aimed for GNOME 42. In Firefox you need to enable webrender which is still beta but should be stabilized soon. I used VA-API as an example because that is another thing that only works locally and doesn't touch X. If you are using VA-API to output video to an X window then it explicitly won't send the original compressed video stream because the point of VA-API is to use hardware decompression. If you want to stream video then X and VA-API are both the wrong tool, you have to use something like gstreamer or phonon.
"Cairo is also backed by X rendering and this is the default."
This is incorrect, Cairo X rendering only happens if you use the Cairo Xlib surface, which GTK3 only uses in some circumstances.
Sure, maybe you're using a lot of applications don't use GL or Vulkan now. But if they are being actively developed, they are probably actively moving towards it. We can revise my original comment if you think it was wrong and want to make it more relevant to your situation: The "vast majority of cases" has moved to using GL or Vulkan or are taking steps to move towards it.
Well, it is a big difference. Someone was saying below that the vast majority of rendering was done "client-side" since 2000s, and it actually couldn't be farther from the truth. Even in 2021, the vast majority of of programs are still rendering server-side.
And, I have also been reading the claims of programs going to switch to OpenGL "any time soon" since the 2000s, with people backing out of the idea always due to "drivers" or the like. My experience trying to accelerate 2d programs with OpenGL has always been a disaster anyway. Maybe Vulkan is more suited to 2d rendering, but I would be surprised.
That is why I don't believe in that argument. Server-side rendering _is_ working as of today. Most programs do not use OpenGL at all. Wayland is breaking all of this; it's not that it was already broken.
> If you use Plasma, then you are using some Qt Quick software.
I was perusing the KDE source and the only program I could find is plasma-widgets. Not surprising: the only program in the entire KDE desktop that uses Qt Quick is a widgets program. It's basically a layering program.
> Currently only the GNOME extensions panel is ported to GTK4
I use latest released version of Gnome and it is not.
> Cairo X rendering only happens if you use the Cairo Xlib surface, which GTK3 only uses in some circumstances.
i.e. ALWAYS when using X as backend. It is the default. What else are they going to use, the PostScript backend?
"And, I have also been reading the claims of programs going to switch to OpenGL 'any time soon' since the 2000s, with people backing out of the idea always due to 'drivers' or the like."
I really don't know what else to tell you. Like I said, GTK4 and Qt are already using it. Chrome/Electron is using it, and Firefox will have it very soon. Wayland didn't break anything here, that and improvements in the drivers were just the missing pieces to finally complete the transition. Really, developers have been trying to ditch X for an extremely long time, you just said you've been hearing them talk about it for 20 years. Well as you know it takes a long time to rebuild everything.
"I was perusing the KDE source and the only program I could find is plasma-widgets"
I can think of several: KDE Connect, KSysGuard, System Settings, Kamoso, Kongress, there are more but I can't remember all of them! And yes, the entire shell also uses QML and Qt Quick.
"I use latest released version of Gnome and it is not."
"i.e. ALWAYS when using X as backend. It is the default. What else are they going to use, the PostScript backend? "
No, the image backend is probably what you would consider the default because it works everywhere and can be used for off-screen rendering. In my experience Cairo Xlib surfaces are actually pretty uncommon because client side operations are done so frequently.
> I really don't know what else to tell you. Like I said, GTK4 and Qt are already using it. Chrome/Electron is using it, and Firefox will have it very soon.
Well, then don't repeat exactly the same argument. The point is that most desktop software _as of this day_ does not use client-side rendering, and even less desktop software uses OpenGL for rendering. Most of them are using plain classic widgets. We find some exceptions, but this is hardly enough to claim that most desktop software uses OpenGL, not in 2020, much less in 2000s.
> I can think of several: KDE Connect, KSysGuard, System Settings, Kamoso, Kongress,
Kongress looks like a widget anyway. You can easily recognize these programs by how poorly they integrate with the rest of KDE, and I definitely do not see them with any frequency at all.
> This happened in GNOME 40 so your distro may be behind.
So, apparently, gnome-shell uses it through GJS, which is why I can't find any binary linking directly to gtk4. It's still literally one user.
> No, the image backend is probably what you would consider the default because it works everywhere and can be used for off-screen rendering.
The xlib backend is obviously also capable of off-screen rendering, otherwise all hell would break loose. And the xlib backend is still the default.
Why don't you just try? It's not hard to get into a situation where you don't have a working OpenGL environment and _all_ software still works. It's not hard to measure the bandwidth used for indirect X. Etc. Etc.
There isn't anything else for me to say, and I am not arguing with you or presenting an argument, this is just a casual conversation. Most of the software I know about uses OpenGL or Vulkan for rendering, and the server side ones are the exception. I'm repeating it because it didn't seem like you were understanding what I was saying, if you did understand it then you can disregard the previous comment. You can tell me that you don't use that software, which is fine for you, and I'm happy for you to use whatever suits you, but it's not really a meaningful discussion for us to have either. Please avoid making such comments or suggesting that I don't know what I'm talking about. Of course I can't know exactly what is going on on your computer, so if you want to explain it, then just tell me.
"I have not been able to find any QML whatsoever for the rest"
You won't find some of the QML from looking at the apps, bits of it are scattered in the KDE frameworks too. I don't think they integrate poorly, work has been done to make them match the Breeze skin.
"It's still literally one user."
Yeah that was kind of a dry run for GTK4 porting. As I said before, everything else is being ported, the next step was to get the support libraries ported over and then everything else can follow. https://gitlab.gnome.org/GNOME/Initiatives/-/issues/26
"The xlib backend is obviously also capable of off-screen rendering"
You usually don't want to do that, it introduces unnecessary round trips when you could just render it on the client side and then avoid that. That link to gdk surface is misleading, even on X11, GTK4 is using the GL renderer as default and is not going to call that or bother creating a cairo surface. Believe me, I've tried this in a situation without a working OpenGL environment and the performance is degraded.
Then why not use a wire protocol for Vulkan? SPIRV already exists. It only needs some wrapper code tacked on to work as a X extension and you have perfectly efficient server side rendering that potentially works even over the network without breaking anybodies backwards compatibility.
I'm very confused by this comment, SPIR-V is not a wire protocol for Vulkan. And you can't really serialize Vulkan over the network, that breaks things like vkMapMemory.
If you want something like a remote Vulkan, the thing to watch there would be WebGPU.
Yes I was referring to the "WebGPU Shading Language" which I abbreviated with WebGPU which should be obvious. The WebGPU API is the boiler plate you need to make WGSL over the wire work. My original proposal was an X extension in a similar fashion that makes SPIR-V over the wire work.
> The "vast majority of cases" has moved to using GL or Vulkan, which also doesn't serialize over the network.
IIRC, GLX did serialize over the network (though AFAIK limited to an older version of OpenGL; the vast majority of cases has moved to a newer version of OpenGL).
