Android Automotive

When you are in a phone call on your car (and you’re connected on bluetooth HFP), you still hear turns signal or anything else… isn’t it? That’s mandatory.

signals
All those pictos has his proper sound (his .wav file that should be played when he goes visible by the driver)

Just so you know, i’m NOT embedding a media player to play those sounds.
I’m just connecting a little “buzzer” based on Pulse Width Modulation that can play wav files.

I wanted to play those waves from Android directly but I think it’s not a good idea because of the case if the driver is on phone call (NO sounds will come out) and that’s NOT what I see in my car.

This is way off topic and [has | should have] nothing to do with a car radio, but it sounds like what you want is a sound card with multiple line input and a mixer.

And note that the image you attached are for a set of dashboard LEDs that typically do NOT have any audible feedback.

Yes, and it’s a bit hard to explain… Never mind… :smile:
Thanks :blush:

I’m trying to add the Camera HAL because this is not on the device and I would like to be able to connect a USB Camera to the hikey and get the stream.

So what I did is that I added the HAL package from /hardware/interfaces/camera like that in the hikey960_car.mk

PRODUCT_PACKAGES += android.hardware.camera.provider@2.4-impl \
	android.hardware.camera.provider@2.4-service

On reboot I’m getting this log :

06-20 14:43:42.361  4495  4495 I android.hardware.camera.provider@2.4-service: Camera provider Service is starting.
06-20 14:43:42.362  2275  2275 W hwservicemanager: getTransport: Cannot find entry android.hardware.camera.provider@2.4::ICameraProvider/legacy/0 in either framework or device manifest.
06-20 14:43:42.363  4495  4495 D vndksupport: Loading /vendor/lib/hw/android.hardware.camera.provider@2.4-impl.so from current namespace instead of sphal namespace.
06-20 14:43:42.376  4495  4495 E CamProvider@2.4-impl: Could not load camera HAL module: -2 (No such file or directory)
06-20 14:43:42.376  4495  4495 E CamProvider@2.4-impl: HIDL_FETCH_ICameraProvider: camera provider init failed!
06-20 14:43:42.377  4495  4495 E android.hardware.camera.provider@2.4-service: Could not get passthrough implementation for android.hardware.camera.provider@2.4::ICameraProvider/legacy/0.

However, I added to the manifest the HIDL like following :

<hal format="hidl">
<name>android.hardware.camera</name>
	<transport arch="32">passthrough</transport>
	<version>2.4</version>
        <interface>
            <name>ICameraProvider</name>
            <instance>legacy</instance>
         </interface>
 </hal>

Indeed, the lshal command does not list the ICameraProvider.
What I’m doing wrong ?

Ok, so here is the thing about cameras;

  1. I haven’t looked into that yet. Down the road, eventually, I intend to.
  2. For the time being, my source trees actually do provide access to and use of cameras. Specifically, USB cameras.
  3. HOWEVER, my setup does NOT use any kind of HAL for the cameras. I’m using them strictly for recording out the front/rear for security purposes. This is done from camd service using ffmpeg. The recorded videos can be reviewed through an Android application such as VLC.

Now what you’ve done, is you’ve only implemented a small part of the Android camera stack. Not nearly enough to actually operate! For one, the thing you implemented only supports legacy / passthrough HALs, and you have not implemented a legacy hal. So one thing you are going to want to add is this; modules/camera/3_4 - platform/hardware/libhardware - Git at Google

Now according to this; Camera  |  Android Open Source Project you will also need a camera device hal, so this; camera/device - platform/hardware/interfaces - Git at Google

HOWEVER, we are building for AUTOMOTIVE here, and these HALs are intended for things like PHONES and TABLETS. They are ALL unsuitable. Instead, you should actually use THIS: Vehicle Camera HAL  |  Android Open Source Project

Pay attention to the sample driver mentioned here;
https://source.android.com/devices/automotive/camera-hal#kernel-drivers

You will need to use the V4L2 sample driver from here; packages/services/Car/evs/sampleDriver

AND NOT the standard hardware interface driver from here; automotive/evs/1.0 - platform/hardware/interfaces - Git at Google (which is not a complete implementation)

1 Like

Pay attention to this; https://android.googlesource.com/platform/packages/services/Car/+/master/evs/sampleDriver/EvsEnumerator.cpp#39

What that is doing, is it is adding ALL of /dev/video*, in no particular order. You are going to have to modify that so that you can add specific cameras in a predefined order. But being USB devices, you can’t count on their order as far as /dev/video* goes. Fortunately, I’ve already addressed this issue; camd/camd.c · master · HiKey960-Car / android_device_linaro_hikey · GitLab – I have code in there that is able to identify specific cameras based on the physical USB topology.

1 Like

Many thanks ! I’ll start from that ! I’ll let you know if I find some difficulties.

But indeed, that will be very cool to integrate this Automotive HAL Camera. I’ll add it and let you know if I succeed !

One other thing I just found is that the EVS “app” responds to states determined by the Automotive HAL. For example, “reverse” will activate the rear view camera(s).

Probably property GEAR_SELECTION and/or CURRENT_GEAR = GEAR_REVERSE

1 Like

Hello @doitright.

Ok then I added these lines to the hikey960_car.mk :
PRODUCT_PACKAGES += android.hardware.automotive.evs@1.0-service \ android.hardware.automotive.evs@1.0-sample

I first start the service, then the sample app.

06-21 08:50:11.066  4431  4431 I ServiceManagement: Removing namespace from process name android.hardware.automotive.evs@1.0-service to evs@1.0-service.
06-21 08:50:11.068  4431  4431 D android.hardware.automotive.evs@1.0-service: EvsEnumeratorHw-Mock is ready.
06-21 08:50:11.361  2355  2355 E libEGL  : call to OpenGL ES API with no current context (logged once per thread)
06-21 08:50:24.202  2355  2355 I chatty  : uid=1000(system) composer@2.1-se identical 66 lines
06-21 08:50:24.344  2355  2355 E libEGL  : call to OpenGL ES API with no current context (logged once per thread)
06-21 08:50:24.513  4432  4432 I EvsSampleDriver: EVS Hardware Enumerator service is starting
06-21 08:50:24.513  4432  4432 D EvsSampleDriver: EvsEnumerator created
06-21 08:50:24.513  4432  4432 I EvsSampleDriver: Starting dev/video* enumeration
06-21 08:50:24.602  4432  4432 I EvsSampleDriver: Found 1 qualified video capture devices of 1 checked
06-21 08:50:24.603  4432  4432 I ServiceManagement: Removing namespace from process name android.hardware.automotive.evs@1.0-sample to evs@1.0-sample.
06-21 08:50:24.608  4432  4432 D EvsSampleDriver: EvsEnumeratorHw is ready.
06-21 08:50:25.362  2355  2355 E libEGL  : call to OpenGL ES API with no current context (logged once per thread)

So, it seems that he founds my USB Camera, isn’t it.?

“EVS app” means the android.hardware.automotive.evs@1.0-sample ? Or the evs_app located in packagse/services/Car/evs/app. Both are installed anyway. The evs_app on start
console# evs_app [ 140.753088] init: Received control message 'interface_start' for 'android.hardware.automotive.evs@1.0::IEvsEnumerator/default' from pid: 2278 (/system/bin/hwservicemanager) [ 140.768976] init: Could not find service hosting interface android.hardware.automotive.evs@1.0::IEvsEnumerator/default

Here is my manifest.xml
<hal format="hidl"> <name>android.hardware.automotive.evs</name> <transport>hwbinder</transport> <version>1.0</version> <interface> <name>IEvsEnumerator</name> <instance>default</instance> </interface> </hal>

For now, this is ok for me because I only have one camera device attached to the hikey in /dev/video0

The sample seems to be good, I see that he opens camera, and construct his own EGL/OpenGL surface but to trigger this you said that I need to set a state on the Automotive HAL ? I aleady work with those values from types.hal
enum VehicleGear: int32_t { GEAR_NEUTRAL = 0x0001, GEAR_REVERSE = 0x0002, GEAR_PARK = 0x0004, GEAR_DRIVE = 0x0008, }
But it seems that nothing is showing when I play with those values

Well… sounds at least like you’re on your way with it.

This one;
https://android.googlesource.com/platform/packages/services/Car/+/master/evs/app/

This is key;
https://android.googlesource.com/platform/packages/services/Car/+/master/evs/app/evs_app.cpp#139

And this;
https://android.googlesource.com/platform/packages/services/Car/+/master/evs/app/evs_app.cpp#143

And don’t forget the config – you need to tell it about your camera.
https://android.googlesource.com/platform/packages/services/Car/+/master/evs/app/config.json#16

And of course, include its module in the build;
LOCAL_MODULE:= evs_app

And probably this;
https://android.googlesource.com/platform/packages/services/Car/+/master/evs/app/Android.mk#76

Yes I’m on the right way. I think I got it soon but just I have some problems with the manifest.xml…

These are already added.

https://android.googlesource.com/platform/packages/services/Car/+/master/evs/app/config.json.readme#32

Your camera is appearing as /dev/video0? The default config file does NOT specify “reverse” as a function for /dev/video0…

Indeed, I just noticed that I “reverse” is pointing on /dev/video32… I will modify it to point on /dev/video0 then.

Just, doo you know how to add the android.hardware.automotive.evs to manifest correctly ?
It seems that he does not find the instance when I start evs_app.
could not find service hosting interface android.hardware.automotive.evs@1.0::IEvsEnumerator/default

lshal results for “evs” :
android.hardware.automotive.evs@1.0::IEvsEnumerator/EvsEnumeratorHw 0/1 3819 2278 android.hardware.automotive.evs@1.0::IEvsEnumerator/EvsEnumeratorHw-Mock 0/1 3802 2278

Indeed there is no default instance. There is EvsEnumeratorHw, EvsEnumeratorHw-Mock

#EDIT
just found something on the code there is a “–mock” flag to connect to EvsEnumeratorHw-Mock

#EDIT 2
Ok… Segmentation fault :frowning: on evs_app… evs_app --mock --test to bypass the HAL communication and just simulate a “reverse”

That sounds like progress. Segfault isn’t the end of the world. Just trace it out.

2 Likes

sp<IEvsCamera> pCamera = pEnum->openCamera(evsCameraId); if (pCamera.get() == nullptr) { ALOGE("Failed to allocate new EVS Camera interface for %s", evsCameraId); return nullptr; }

This part of code is failing. However, the HAL enumerates correctly the device.

06-22 08:26:02.344  4623  4623 I EvsApp  : EVS app starting
06-22 08:26:02.344  4623  4623 I EvsApp  : Acquiring EVS Enumerator
06-22 08:26:02.349  4623  4623 I EvsApp  : Acquiring EVS Display
06-22 08:26:02.349  4623  4623 W EvsApp  : Test mode selected, so not talking to Vehicle HAL
06-22 08:26:02.349  4623  4623 I EvsApp  : Constructing state controller
06-22 08:26:02.349  4623  4623 D EvsApp  : Requesting camera list
06-22 08:26:02.349  4623  4623 D EvsApp  : State controller ready
06-22 08:26:02.349  4623  4623 I EvsApp  : Entering running state
06-22 08:26:02.349  4623  4625 D EvsApp  : Starting EvsStateControl update loop
06-22 08:26:02.349  4623  4625 D EvsApp  : Switching to state 1.
06-22 08:26:02.349  4623  4625 D EvsApp  :   Current state 0 has 0 cameras
06-22 08:26:02.349  4623  4625 D EvsApp  :   Desired state 1 has 1 cameras
06-22 08:26:02.349  4623  4625 D EvsApp  : Starting camera stream
06-22 08:26:02.395  4623  4625 I EvsApp  : Intiialized EGL at 1.4
06-22 08:26:02.395  4623  4625 I EvsApp  : Dummy surface looks good!  :)
06-22 08:26:02.396  4623  4625 I EvsApp  : We made our context current!  :)
06-22 08:26:02.396  4623  4625 I EvsApp  : GL EXTENSIONS:
06-22 08:26:02.396  4623  4625 I EvsApp  :   GL_EXT_debug_marker GL_ARM_rgba8 GL_ARM_mali_shader_binary GL_OES_depth24 GL_OES_depth_texture GL_OES_depth_texture_cube_map GL_OES_packed_depth_stencil GL_OES_rgb8_rgba8 GL_EXT_read_format_bgra GL_OES_compressed_paletted_texture GL_OES_compressed_ETC1_RGB8_texture GL_OES_standard_derivatives GL_OES_EGL_image GL_OES_EGL_image_external GL_OES_EGL_image_external_essl3 GL_OES_EGL_sync GL_OES_texture_npot GL_OES_vertex_half_float GL_OES_required_i[ 2132.737823] logd: logdr: UID=2000 GID=2000 PID=4642 n tail=50 logMask=8 pid=4623 start=0ns timeout=0ns
nternalformat GL_OES_vertex_array_[ 2132.749627] logd: logdr: UID=2000 GID=2000 PID=4642 n tail=50 logMask=1 pid=4623 start=0ns timeout=0ns
object GL_OES_mapbuffer GL_EXT_texture_format_BGRA8888 GL_EXT_texture_rg GL_EXT_texture_type_2_10_10_10_REV GL_OES_fbo_render_mipmap GL_[ 2132.772217] logd: logdr: UID=2000 GID=2000 PID=4642 n tail=0 logMask=8 pid=4623 start=0ns timeout=0ns
OES_element_index[ 2132.782605] logd: logdr: UID=2000 GID=2000 PID=4642 n tail=0 logMask=1 pid=4623 start=0ns timeout=0ns
_uint GL_EXT_shadow_samplers GL_OES_texture_compres[ 2132.796923] init: Untracked pid 4642 exited with status 0
sion_astc GL_KHR_[ 2132.802969] init: Untracked pid 4644 exited with status 0
texture_compression_astc_ldr GL_KHR_texture_compression_astc_hdr GL_KHR_texture_compression_astc_sliced_3d GL_KHR_debug GL_EXT_occlusion_query_boolean GL_EXT_disjoint_timer_query GL_EXT_blend_minmax GL_EXT_discard_framebuffer GL_OES_get_program_binary GL_OES_texture_3D GL_EXT_texture_storage GL_EXT_mult
06-22 08:26:02.407  4623  4625 E EvsApp  : Failed to allocate new EVS Camera interface for /dev/video0
06-22 08:26:02.407  4623  4625 E EvsApp  : Failed to set up video texture for /dev/video0 (reverse,park)
06-22 08:26:02.407  4623  4625 D EvsApp  : Arming the display
06-22 08:26:02.407  4623  4625 I EvsApp  : Activated state 1.`

The answer may be obvious, but did you check if your camera device has permissions and selinux context that allows it to be opened?

I can’t tell what it’s doing from those logs. You will need to drill into it deeper and find out why.

1 Like

I think so. I found the problem.
Problem was on the EvsEnumerator of the HAL. He does not emplace_back my device on his list.

Now he founds but evs_app crashes on other stuff. Take a look.
https://gist.github.com/AhmedX6/418678675cb9334266dd614e243f2ec1
Did you already use LOCAL_SANITIZE ? I would like to add it on the EVS driver sample & the evs_app to get exactly what’s wrong. The crash comes from the EVS Sample Driver and that makes evs_app crash after.

Thank you

https://gist.github.com/AhmedX6/418678675cb9334266dd614e243f2ec1#file-gistfile1-txt-L119

SEGV_ACCERR

I think that essentially means that the CPU itself is blocking access to some resource. I’m afraid that I won’t be able to help you much at this point.

The crash comes from gralloc.

Oh okay did not see that one. Will check. Thanks

@doitright
I identified the problem
https://android.googlesource.com/platform/packages/services/Car/+/master/evs/sampleDriver/EvsV4lCamera.cpp#499
This function transfers the video image into the output buffer, and makes any needed format conversion along the way.
mFillBufferFromVideo is a function’s pointer: that can point in one of these function.
https://android.googlesource.com/platform/packages/services/Car/+/master/evs/sampleDriver/bufferCopy.cpp

depending of the pixel format of the driver :
https://android.googlesource.com/platform/packages/services/Car/+/master/evs/sampleDriver/EvsV4lCamera.cpp#166

When I comment the call there is no crash anymore but of course nothing is displayed.

Right, commenting stuff out usually doesn’t make those things work :wink:

Might be useful to seek assistance for this on the bug tracker. Seems to be some issue with gralloc/surfaceflinger.

That’s obvious, it was just to be sure that bug is here and nothing else :smiley:

I will do that.