Gstreamer Camera 2 Source for Android

I'm very happy to send a new camera source element patch to support Android Camera 2 API.  This implementation is a part of R&D work which is sponsored by Collabora, Ltd.

Not much later since the previous element, ahcsrc, has been merged into 'gst-plugins-bad', Android N announced their public native level APIs. This news brought me another R&D opportunity to implement same featured, but more stable and little bit faster element to handle Camera Devices on Android.

I named the new element as 'ahc2src' because the previous one is just 'ahcsrc' and the fundamental API is Android Camera 2 NDK API. That means JNI is no longer required to access Camera devices. To understand why 'ahc2src' is better, here's comparison chart.

	ahcsrc	ahc2src
Supported Android version	> Lollipop (android-19)	> Nougat (android-24)
API	Camera API	Camera 2 NDK API
Device control	Java API through JNI	Native
Supported Image Format	YV12, YUY2, NV16, NV21, RGB16	NV12 (YUV_420_888)
Memory Copy	Java to GstBuffer	None (AImageReader inside)

Regarding memory copy, I'm not sure whether it doesn't perform inside of Android. However, ahc2src itself just passes image pointers to source pad and manage the reference counts.

How to get this new element?

Until merged and released gstreamer images which are built for Android Nougat, my personal repository will be continuously maintained to provide 'ahc2src' element. If you want to build from scratch, it might be annoying because of some issues which come from Android NDK.

See the details in the below links;

• BIONIC's  dlopen and dlsym (glib patch, gstreamer issue)
• Cebero build failure
• ahc2src working history

Build from Cerbero

In my personal repository, the patches are ready to build. It will fetch 'gst-plugins-bad' from here.

  $ git clone https://gitlab.collabora.com/joykim/cerbero.git
  $ git checkout wip/ahc2

Note that 'wip/ahc2' branch is potentially or frequently overwritten by forced push.

  $ ./cebero-uninstalled -c config/cross-android-nougat-arm64.cbc bootstrap
  $ ./cebero-uninstalled -c config/cross-android-nougat-arm64.cbc package gstreamer-1.0

Then, the generated SDK image should be extracted to a proper path. (e.g /Users/justin/Library/Android/gstreamer/devel/arm64)

Gstreamer Android Camera Example Application

From the point of Android application view, there's not much different from when using 'ahcsrc'. In theory, it might be enough to replace 'ahcsrc' with 'ahc2src'.

  $ git clone https://gitlab.collabora.com/joykim/gst-android-camera.git
  $ git checkout wip/ahc2

Then, you should modify 'gradle.properties' to set a proper gstreamer SDK path. (e.g, gstAndroidRoot=/Users/justin/Library/Android/gstreamer/devel)

  $ gradle installDebug

If there's no error, the example application will be installed successfully.

Enjoy your Gstreamer with Android Camera 2 source!

Embedded Linux Conference & Open IoT Summit 2017

A couple of months ago, I was hesitating to go to somewhere in North America because it is so far, and above all, topics of the conference are little different from my official domain. However, my curiosity about new area upstaged many obstacles so I could attend this time.

It was held in Portland, Oregon. The city looks like full-grid which I've built in SimCity and very quite and not crowded place.

Gustavo Padovan from Collabora has talked Unifying Android and Mainline Kernel Graphic Stack, but unfortunately, my another interesting session was opened at the same time so I missed his session. I will check some other sessions including his session later when the recording is registered to Youtube.

As for myself, I tended to take the sessions in Open IoT Summit, especially regarding RTOS and OTA matters. A year ago, Zephyr was introduced, but now, it wants to be a LINUX for end-devices.

Another RTOS, RIOT-OS, was showed in this conference. RIOT-OS supports more architectures than Zephyr because it has developed since 2013. However, most of people in the session worried about its license. It is a free open source software under LGPL 2.1. However, due to the characteristic of industry who potentially uses RTOS, the license policy would be an unsolved problem. In the case of Zephyr, it is developing under Apache License so it is relatively less restricted for the industry.

The other outstanding topic was OTA. I could see practical demonstrations to update software with less threatened methods. The traditional strategy for updating firmware is to have doubled space as a preserved area so substantial amount of flash is wasted even after produced, but one session proved that ostree is already available for providing enterprise level service.

Most of topics were very informative so I eagerly look forward to the recordings which will be uploaded soon.

I'd like to thank my employer, Collabora,  for sponsoring my flight, accommodation, and even time.