The Fedora Mobility SIG is working to get Fedora Linux working on mobile device. This includes the PinePhone, a smartphone built with open hardware. Many distros are working to add support for the features that smartphone users expect, including video calls. I was able to use Firefox as a fully-featured Matrix client to make video calls.

On my test runs, to get a fully featured Matrix client for the PinePhone, I had the idea to simply use Firefox or Chromium to access a self-hosted instance of the element-web client. It’s basically the same idea that Schildichat and others are using in form of an electron application, which you can install on your PC. Electron apps ship a special version of the Chromium browser with some HTML and JavaScript files as a package, i.e. inside a Flatpak.

As you may know, Matrix is a chat protocol which can be used with various clients, even as a web application. As a web application you can use web features like WebRTC to open audio & video chats with end-to-end encryption (“E2EC” in Matrix terms ).

All apps failing?

As of this writing, no tested application on the normal Fedora aarch64 repo is capable of accessing the video cameras on the PinePhone, besides the specially-crafted Megapixels. Megapixels just takes pictures at the moment, so we had no working video solution.

Firefox makes a good Matrix webclient UI and I knew from earlier tests, that in a WebRTC session audio works fine. Firefox can easily be made a clickable “Matrix” icon on the desktop by creating a matching desktopfile:

[Desktop Entry]
Version=1.0
Name=Matrix Web
Comment=Directlink
GenericName=Matrix Web
Exec=env LIBVA_DRIVER_NAME=v4l2_request LIBVA_V4L2_REQUEST_VIDEO_PATH=/dev/video0 LIBVA_V4L2_REQUEST_MEDIA_PATH=/dev/media0 MOZ_ENABLE_WAYLAND=1 firefox-wayland %U https://HOMESERVERNAME/web/
Terminal=false
X-MultipleArgs=false
Type=Application
Icon=/home/pine/images/matrix-logo.png
Categories=Network;WebBrowser;
MimeType=text/html;
Keywords=web;browser;internet;
X-Desktop-File-Install-Version=0.26

Unfortunately, the preferred –headless argument crashes current Firefox, so we don’t get the so called kiosk-mode, where the webpage takes the entire available space and removes all controls from the view.

The given URL needs to point to the web client and you need to enable Javascript. You may notice the V4L2 environment vars: they are step one to enable GPU acceleration for Firefox. Step 2 has not yet happened. If you run NoScript in Firefox, make the necessary adjustments and you’re on track. I tested audio calls and they worked as expected, so I thought to myself: “why not” and hit the video call button.

*Kaboom*

It does not only sound like an accident, it was. To our very great surprise, the PinePhone cameras went on and presented a live YUYV image from the ov5640 camera chip. I have a theory how Firefox did it, but that needs further investigations.

Screenshot of a Matrix video call in Firefox on the PinePhone.

You can mark the 4th of March, 2021 at 11:07 CET as the time and date, that first PinePhone video call (that we know of) was made. \o/

“Houston, we have video!”

The other side of the call looked like this, and it was a surprise too, as we got full colors for the image. This is something Megapixels doesn’t do by default.

The other side of the video call from the desktop browser.

The image format YUYV is post processed on the camera chip itself, which gives it a more colorful debrayering in opposite to Megapixels’ preferred way, where post processing is done with dcraw after the image is taken. It also means post-processing is way faster done in hardware.

You may notice that the image is rotated by 90 degrees. This is the native cam image. Firefox simply opened the camera, but was unaware that it had to rotate the image. A minor setback, I’m sure, will be fixed in no time, now that the world knows about it.

Video showing the performance of the Matrix client on a PinePhone under heavy load.

This video shows the actual speed things happen on the Pinephone, when it is under heavy load. Hardware acceleration should improve it.

The first reaction on the global PinePhone chat on Matrix was amazing:

aluizioneto: 11:19

Marius: 11:11 
Breaking News .. Video Chat via Matrix from Pinephone WORKING 😆
Very nice!!! That would solve everything for me. Where is my PP CE?? 😃

A great moment for PinePhone users!

Power Consumption

But there is still much to be done here. Firefox on PinePhone does not have hardware acceleration in use yet, so it took all 4 CPU cores on the phone to do that call. It was still not enough processing power, as the audio part was interrupted and lagging behind. At the moment, Mr. Stransky is helping out to fully enable the Allwinner GPU for Firefox. We already have a working libva driver for the Allwinner system-on-a-chip, called “libva-request”.

Hardware acceleration

MPV is already using this lib to decode FullHD MP4 movies on my PinePhone with next to zero frame drops. The amount of power consumption has dropped significantly, too. It’s possible to watch a full-length movie without running out of juice. The PinePhone is using 3.6 Watts when all CPU cores are running, which results in approximately two hours until the battery is drained. With the help of the GPU this is no longer the case, as the power consumption got down to around 2 Watts, doubling the life time of a full battery charge.

When Firefox is using the GPU and we have a better performance, it will be possible to make a videocall and leave the house without a mobile power bank.

We have to thank so many contributors to made this happen, in special the Fedora SIG Mobility team members who are massively testing and updating packages for Pinephone and PineTab.

I hope, that we all can see us in a PinePhone driven smooth Video-conference soon.