Can you ever catch up?: April 2021

There is also Android AOSP - Supporting External USB Camera this page, which defines same thing posted here, with some more details.

We have a device with an USB HUB connected to it. The external cameras are very flaky. I have tried many different varieties of cameras, and it turns out that not all are directly supported.

Then I found this library called https://github.com/suethan/AndroidUSBCamera based on multiple other projects. I was able to get the camera working by including this project (libuvccamera) as a dependency and using it to build my app. 2/3 cameras worked well. We used mainly generic USB UVC cameras.

Now for our purposes, we needed the camera buffer callbacks to send video data over a video call. I am not sure how that works.

On android 8.0 however I have a completely working implementation. I am not sure of the details, as the modifications to the source were made by my Software Architect. But it works perfectly well. Just plug in the camera and we can use it. However there is quite some work involved.

I'm working on porting those changes over to the Android Pie, however, I am not sure whether it can be done. It looks doable, but without his guidance or support (he is not available) I am not sure I can get it.

In conclusion, I have to say, yes! I was able to successfully compile and build AOSP Pie on sdm660_64, but it doesn't work as an internal camera yet. I am working on it.

You have to add couple of Kernel level configs, and also then enable some other services. Follow the guide here External USB Cameras

Rest of the details are really not that clear and would love to have someone explain what they did. (without screwing their NDAs of course :-) )

Actually I stand corrected.

UVC Camera works fine in Android 9.0. Follow the steps and add the required things.

I was thrown a little off track because we were facing trouble with one particular camera. Now let me tell you how we got it working, and please know this is not a one fix works for all situation. It depends on your requirement and you'll need to lose some to gain some. All cameras may never work.

First things first, install YAWCAM on your PC. Then connect your camera, open yawcam and make sure your device is selected and you can see it's preview. (Note: Yawcam won't start if you have 0 webcams connected).

Next, look under Settings > Device (Device name) > Format Control > Color Space/ Compression. You can have some options or just one option of the following

YUV
YUV2
MJPEG
UVY etc.

This helps to find out what formats are supported on your device.

Now in your AOSP USB Camera HAL, if you check, you might notice that only MJPEG or YUV might be selected. Usually YUV is supported by all cameras. However, MJPEG is not. MJPEG also allows for better resolution. (So that's gonna be a trade off). If you are using Qualcomm HAL, and have enabled all the services and the camera is still not working, then check under ~/android/hardware/qcom/camera/usbcamcore/src/QualcommUsbCamera.cpp

I noticed that MJPEG was enabled by default, and the 4 cameras I tried, 3 of them supported MJPEG so it worked perfectly. However, we had another camera, which was more important for us than others, and turns out it only supported YUV. So I turned this setting off like so

#define JPEG_ON_USB_CAMERA 0

Notice in this piece how it chooses

#if JPEG_ON_USB_CAMERA
    rc = initUsbCamera(camHal, camHal->pictWidth, camHal->pictHeight,
                        V4L2_PIX_FMT_MJPEG);
#else
    rc = initUsbCamera(camHal, camHal->pictWidth, camHal->pictHeight,
                        V4L2_PIX_FMT_YUYV);

So that's it people, I hope I have solved all your issues with these damn cameras and the HALs.

Find update for YUV support here: for interface 3.4

diff --git a/camera/device/3.4/default/ExternalCameraDevice.cpp b/camera/device/3.4/default/ExternalCameraDevice.cpp
index ee7ffaa..853c0a5 100644
--- a/camera/device/3.4/default/ExternalCameraDevice.cpp
+++ b/camera/device/3.4/default/ExternalCameraDevice.cpp
@@ -38,8 +38,10 @@ namespace {
 // Other formats to consider in the future:
 // * V4L2_PIX_FMT_YVU420 (== YV12)
 // * V4L2_PIX_FMT_YVYU (YVYU: can be converted to YV12 or other YUV420_888 formats)
-const std::array<uint32_t, /*size*/1> kSupportedFourCCs {{
-    V4L2_PIX_FMT_MJPEG
+const std::array<uint32_t, /*size*/3> kSupportedFourCCs {{
+    V4L2_PIX_FMT_MJPEG,
+    V4L2_PIX_FMT_YUYV,
+    V4L2_PIX_FMT_YUV420
 }}; // double braces required in C++11
 
 constexpr int MAX_RETRY = 5; // Allow retry v4l2 open failures a few times.
@@ -766,6 +768,7 @@ void ExternalCameraDevice::trimSupportedFormats(
 
     // Remove formats that has aspect ratio not croppable from largest size
     std::vector<SupportedV4L2Format> out;
+    std::vector<SupportedV4L2Format> outFixed;
     for (const auto& fmt : sortedFmts) {
         float ar = ASPECT_RATIO(fmt);
         if (isAspectRatioClose(ar, maxSizeAr)) {
@@ -782,6 +785,14 @@ void ExternalCameraDevice::trimSupportedFormats(
         }
     }
     sortedFmts = out;
+    for (const auto& fmt : sortedFmts) {
+        if (fmt.fourcc == V4L2_PIX_FMT_MJPEG) {
+            outFixed.push_back(fmt);
+        }
+    }
+    if (outFixed.size() > 0) {
+        sortedFmts = outFixed;
+    }
 }
 
 std::vector<SupportedV4L2Format>
diff --git a/camera/device/3.4/default/ExternalCameraDeviceSession.cpp b/camera/device/3.4/default/ExternalCameraDeviceSession.cpp
index 1af3f39..37bf75f 100644
--- a/camera/device/3.4/default/ExternalCameraDeviceSession.cpp
+++ b/camera/device/3.4/default/ExternalCameraDeviceSession.cpp
@@ -1724,33 +1724,60 @@ bool ExternalCameraDeviceSession::OutputThread::threadLoop() {
         return false;
     };
 
-    if (req->frameIn->mFourcc != V4L2_PIX_FMT_MJPEG) {
-        return onDeviceError("%s: do not support V4L2 format %c%c%c%c", __FUNCTION__,
+    if (req->frameIn->mFourcc != V4L2_PIX_FMT_MJPEG && req->frameIn->mFourcc != V4L2_PIX_FMT_YUYV && req->frameIn->mFourcc != V4L2_PIX_FMT_YUV420) {
+        return onDeviceError("%s: do not support V4L2 format %c%c%c%c xxx", __FUNCTION__,
                 req->frameIn->mFourcc & 0xFF,
                 (req->frameIn->mFourcc >> 8) & 0xFF,
                 (req->frameIn->mFourcc >> 16) & 0xFF,
                 (req->frameIn->mFourcc >> 24) & 0xFF);
     }
+    ALOGV("%s: V4L2 format: %c%c%c%c", __FUNCTION__,
+                req->frameIn->mFourcc & 0xFF,
+                (req->frameIn->mFourcc >> 8) & 0xFF,
+                (req->frameIn->mFourcc >> 16) & 0xFF,
+                (req->frameIn->mFourcc >> 24) & 0xFF);
 
     std::unique_lock<std::mutex> lk(mBufferLock);
     // Convert input V4L2 frame to YU12 of the same size
     // TODO: see if we can save some computation by converting to YV12 here
     uint8_t* inData;
     size_t inDataSize;
+    int res = -1;
     req->frameIn->map(&inData, &inDataSize);
-    // TODO: in some special case maybe we can decode jpg directly to gralloc output?
-    ATRACE_BEGIN("MJPGtoI420");
-    int res = libyuv::MJPGToI420(
-            inData, inDataSize,
-            static_cast<uint8_t*>(mYu12FrameLayout.y),
-            mYu12FrameLayout.yStride,
-            static_cast<uint8_t*>(mYu12FrameLayout.cb),
-            mYu12FrameLayout.cStride,
-            static_cast<uint8_t*>(mYu12FrameLayout.cr),
-            mYu12FrameLayout.cStride,
-            mYu12Frame->mWidth, mYu12Frame->mHeight,
-            mYu12Frame->mWidth, mYu12Frame->mHeight);
-    ATRACE_END();
+    ALOGV("%s: Size: %zu, WxH: (%u, %u)", __FUNCTION__, inDataSize, mYu12Frame->mWidth, mYu12Frame->mHeight);
+    if (req->frameIn->mFourcc == V4L2_PIX_FMT_MJPEG) {
+        // TODO: in some special case maybe we can decode jpg directly to gralloc output?
+        ATRACE_BEGIN("MJPGtoI420");
+        res = libyuv::MJPGToI420(
+                inData, inDataSize,
+                static_cast<uint8_t*>(mYu12FrameLayout.y),
+                mYu12FrameLayout.yStride,
+                static_cast<uint8_t*>(mYu12FrameLayout.cb),
+                mYu12FrameLayout.cStride,
+                static_cast<uint8_t*>(mYu12FrameLayout.cr),
+                mYu12FrameLayout.cStride,
+                mYu12Frame->mWidth, mYu12Frame->mHeight,
+                mYu12Frame->mWidth, mYu12Frame->mHeight);
+        ATRACE_END();
+    } else if (req->frameIn->mFourcc == V4L2_PIX_FMT_YUYV && inDataSize != libyuv::kUnknownDataSize ) { //&& inDataSize != libyuv::kUnknownDataSize
+        ALOGV("%s: Converting to I420 | Size: %zu, WxH: (%ux%u), yStride: %u, cStride: %u", __FUNCTION__, inDataSize, mYu12Frame->mWidth, mYu12Frame->mHeight, mYu12FrameLayout.yStride, mYu12FrameLayout.cStride);
+        ATRACE_BEGIN("YUY2ToI420");
+        res = libyuv::YUY2ToI420(inData, mYu12Frame->mWidth * 2,
+                static_cast<uint8_t*>(mYu12FrameLayout.y),
+                mYu12FrameLayout.yStride,
+                static_cast<uint8_t*>(mYu12FrameLayout.cb),
+                mYu12FrameLayout.cStride,
+                static_cast<uint8_t*>(mYu12FrameLayout.cr),
+                mYu12FrameLayout.cStride,
+                mYu12Frame->mWidth, mYu12Frame->mHeight);
+        ATRACE_END();
+    } else{
+        ALOGV("%s: YUV420 format. No conversion", __FUNCTION__);
+        memcpy(static_cast<uint8_t*>(mYu12FrameLayout.y), inData, inDataSize/2);
+        memcpy(static_cast<uint8_t*>(mYu12FrameLayout.cb), (inData + inDataSize/2), inDataSize/4);
+        memcpy(static_cast<uint8_t*>(mYu12FrameLayout.cr), (inData + inDataSize/2 + inDataSize/4), inDataSize/4 );
+        res = 0;
+    }
 
     if (res != 0) {
         // For some webcam, the first few V4L2 frames might be malformed...
diff --git a/camera/device/3.4/default/include/ext_device_v3_4_impl/ExternalCameraDeviceSession.h b/camera/device/3.4/default/include/ext_device_v3_4_impl/ExternalCameraDeviceSession.h
index 0b94c11..c764809 100644
--- a/camera/device/3.4/default/include/ext_device_v3_4_impl/ExternalCameraDeviceSession.h
+++ b/camera/device/3.4/default/include/ext_device_v3_4_impl/ExternalCameraDeviceSession.h
@@ -108,7 +108,7 @@ struct ExternalCameraDeviceSession : public virtual RefBase {
 
     static const int kMaxProcessedStream = 2;
     static const int kMaxStallStream = 1;
-    static const uint32_t kMaxBytesPerPixel = 2;
+    static const uint32_t kMaxBytesPerPixel = 4;
 
 protected:

Can you ever catch up?

Thursday, April 1, 2021

Android 9.0 USB Camera Implementation

Contact Form

Social Media