Not much later since the previous element, ahcsrc, has been merged into 'gst-plugins-bad', Android N announced their public native level APIs. This news brought me another R&D opportunity to implement same featured, but more stable and little bit faster element to handle Camera Devices on Android.
I named the new element as 'ahc2src' because the previous one is just 'ahcsrc' and the fundamental API is Android Camera 2 NDK API. That means JNI is no longer required to access Camera devices. To understand why 'ahc2src' is better, here's comparison chart.
ahcsrc
|
ahc2src
|
|
| Supported Android version |
> Lollipop (android-19)
|
> Nougat (android-24)
|
| API |
Camera API
|
Camera 2 NDK API
|
| Device control |
Java API through JNI
|
Native
|
| Supported Image Format |
YV12, YUY2, NV16, NV21, RGB16
|
NV12 (YUV_420_888)
|
| Memory Copy |
Java to GstBuffer
|
None (AImageReader inside)
|
Regarding memory copy, I'm not sure whether it doesn't perform inside of Android. However, ahc2src itself just passes image pointers to source pad and manage the reference counts.
How to get this new element?
Until merged and released gstreamer images which are built for Android Nougat, my personal repository will be continuously maintained to provide 'ahc2src' element. If you want to build from scratch, it might be annoying because of some issues which come from Android NDK.
See the details in the below links;
Build from Cerbero
In my personal repository, the patches are ready to build. It will fetch 'gst-plugins-bad' from here.
Note that 'wip/ahc2' branch is potentially or frequently overwritten by forced push.
Then, the generated SDK image should be extracted to a proper path. (e.g /Users/justin/Library/Android/gstreamer/devel/arm64)
Gstreamer Android Camera Example Application
From the point of Android application view, there's not much different from when using 'ahcsrc'. In theory, it might be enough to replace 'ahcsrc' with 'ahc2src'.
Then, you should modify 'gradle.properties' to set a proper gstreamer SDK path. (e.g, gstAndroidRoot=/Users/justin/Library/Android/gstreamer/devel)
If there's no error, the example application will be installed successfully.
Enjoy your Gstreamer with Android Camera 2 source!
How to get this new element?
Until merged and released gstreamer images which are built for Android Nougat, my personal repository will be continuously maintained to provide 'ahc2src' element. If you want to build from scratch, it might be annoying because of some issues which come from Android NDK.
See the details in the below links;
- BIONIC's dlopen and dlsym (glib patch, gstreamer issue)
- Cebero build failure
- ahc2src working history
Build from Cerbero
In my personal repository, the patches are ready to build. It will fetch 'gst-plugins-bad' from here.
$ git clone https://gitlab.collabora.com/joykim/cerbero.git $ git checkout wip/ahc2
Note that 'wip/ahc2' branch is potentially or frequently overwritten by forced push.
$ ./cebero-uninstalled -c config/cross-android-nougat-arm64.cbc bootstrap $ ./cebero-uninstalled -c config/cross-android-nougat-arm64.cbc package gstreamer-1.0
Then, the generated SDK image should be extracted to a proper path. (e.g /Users/justin/Library/Android/gstreamer/devel/arm64)
Gstreamer Android Camera Example Application
From the point of Android application view, there's not much different from when using 'ahcsrc'. In theory, it might be enough to replace 'ahcsrc' with 'ahc2src'.
$ git clone https://gitlab.collabora.com/joykim/gst-android-camera.git $ git checkout wip/ahc2
Then, you should modify 'gradle.properties' to set a proper gstreamer SDK path. (e.g, gstAndroidRoot=/Users/justin/Library/Android/gstreamer/devel)
$ gradle installDebug
If there's no error, the example application will be installed successfully.
Enjoy your Gstreamer with Android Camera 2 source!
Justin Kim,
ReplyDeleteThank you for this great improvement for gstreamer. Camera2 API is something we can work with.
I am trying to capture 4k video from the camera and stream it as H265 video. Assuming that my android device has the HW ecnoder, what would be the right gstreamer pipeline for that purpose?
Thanks,
Yechiel
Unfortunately, Ahc2src is still in progress due to lack of interest from the community. Although it’s difficult to access Android camera 2 API via GStreamer source element, there are encoder and decoder elements in gst-plugins-bad. So IMHO, if you need to use Android camera api 2, capturing 4k video as I420 then streaming to pipeline by using appsrc which is connected to encoder element. It’s currently possible way but it’s a bit tricky.
ReplyDeleteHi Justin Kim,
DeleteThanks for your reply.
Do you know of any other decent way of doing it from an android app without a gstreamer pipeline? Maybe using standard android apis such as Camera, MediaCodec and rtp lib for instance
Hi Justin,
ReplyDeleteHope you are doing great !!.
I am trying to reach you on one of our requirement and need your help in below.
Requirement:
• Need gstreamer based android implementation to preview both Front and Rear cameras.
Git hub project and running on device:
• We have taken your GitHub project where camera preview is happening using gstreamer pipeline in jni. Below is the link for sample.
o https://github.com/justinjoy/gst-android-camera
• We have taken it, built for our target/mobile, tried to run it.
• We are able to make Rear camera preview successfully.
History on Issue:
• We have gone through your complete implementation and could not find configuration/implementation to control front or rear camera, also could not find giving cameraid.
• Seems its taking rear camera as default, We tried to change the preview from rear to front in app layer but it did not work.
Could you help us in changing rear camera preview to front camera preview, also previewing both front and rear simultaneously.
Thanks and Regards
Rajesh Chanda
It is a shame this never matured. android.camera.hardware is deprecated and many many vendors don't implement the API anymore.
ReplyDelete