I’m currently working on enabling this camera on 4.14 with the upstream ov5640 driver. I mainly had to fill the DTB with camera pieces (cci subnodes, iommu entry). For now I get some timeouts communicating with the sensor via I2C/CCI because I use the wrong I2C bus. Actually we need the qcom-i2c-cci driver which is not present in release/qcomlt-4.14. Stay tuned 
1 Like
Any status? I’ve been attempting to bring this camera up in 17.09 but haven’t been having much luck. I’ve gotten as far as initializing the camera, but not much further. (i.e. Cannot stream video through gstreamer, nor capture stills.)
What I’ve done so far was to remove the qcom_cci hack from the out-of-tree driver on D3-Engineering’s github, and bring in support for the v4l2-ctrls for link_freq and pixel_clock.
If anyone wants to take a look - https://gist.github.com/RobGries/b7beb724574f6da2ff660604980f6311
Hi @Rob_Gries,
So I just took the 4.14 kernel branch from Qcom tree and did the below modifications to get OV5640 camera sensor working:
-
Enabling the relevant front/rear node and modifying the compatible string and endpoint name for OV5640.
-
Adding the stable 96MHz pixel clock (this is just a hack to get the driver working with camss and not the proper implementation yet).
With the help of the above modifications I can use gstreamer to capture images. I will share the patch soon and if possible will try to get this into Qcom tree in a clean manner.
Hope this helps!
Thanks,
Mani
Can you share a gist of the source? or better yet a patch?
How did you do this? More specifically, what value did you use?
Below is the diff which I used to get OV5640 working in 4.14 branch of Qcom kernel.
diff --git a/arch/arm64/boot/dts/qcom/apq8016-sbc.dtsi b/arch/arm64/boot/dts/qcom/apq8016-sbc.dtsi
index c99fc9a..a7293b4 100644
--- a/arch/arm64/boot/dts/qcom/apq8016-sbc.dtsi
+++ b/arch/arm64/boot/dts/qcom/apq8016-sbc.dtsi
@@ -361,9 +361,9 @@
cci@1b0c000 {
status = "ok";
- camera_rear@3b {
- compatible = "ovti,ov5645";
- reg = <0x3b>;
+ camera_rear@78 {
+ compatible = "ovti,ov5640";
+ reg = <0x78>;
enable-gpios = <&msmgpio 34 GPIO_ACTIVE_HIGH>;
reset-gpios = <&msmgpio 35 GPIO_ACTIVE_LOW>;
@@ -378,10 +378,8 @@
vdda-supply = <&camera_vdda_2v8>;
vddd-supply = <&camera_vddd_1v5>;
- status = "disabled";
-
port {
- ov5645_ep: endpoint {
+ ov5640_ep: endpoint {
clock-lanes = <1>;
data-lanes = <0 2>;
remote-endpoint = <&csiphy0_ep>;
@@ -428,8 +426,7 @@
csiphy0_ep: endpoint {
clock-lanes = <1>;
data-lanes = <0 2>;
- remote-endpoint = <&ov5645_ep>;
- status = "disabled";
+ remote-endpoint = <&ov5640_ep>;
};
};
port@1 {
diff --git a/drivers/media/i2c/ov5640.c b/drivers/media/i2c/ov5640.c
index 39a2269..56e143a 100644
--- a/drivers/media/i2c/ov5640.c
+++ b/drivers/media/i2c/ov5640.c
@@ -162,6 +162,7 @@ struct ov5640_ctrls {
struct v4l2_ctrl *auto_gain;
struct v4l2_ctrl *gain;
};
+ struct v4l2_ctrl *pixel_clock;
struct v4l2_ctrl *brightness;
struct v4l2_ctrl *saturation;
struct v4l2_ctrl *contrast;
@@ -2009,6 +2010,9 @@ static int ov5640_init_controls(struct ov5640_dev *sensor)
ctrls->gain = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_GAIN,
0, 1023, 1, 0);
+ /* Pixel Clock (default of 96MHz) */
+ ctrls->pixel_clock = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_PIXEL_RATE,
+ 1, INT_MAX, 1, 96000000);
ctrls->saturation = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_SATURATION,
0, 255, 1, 64);
ctrls->hue = v4l2_ctrl_new_std(hdl, ops, V4L2_CID_HUE,
As you can see that I have used 96MHz as the default Pixel clock. According to the datasheet, this value suits for most of the formats except QVGA.
After flashing the boot image, below scripts can be used to capture image using Gstreamer.
sudo media-ctl -d /dev/media0 -l '"msm_csiphy0":1->"msm_csid0":0[0],"msm_csid0":1->"msm_ispif0":0[0],"msm_ispif0":1->"msm_vfe0_rdi0":0[0],"msm_ispif0":1->"msm_vfe0_pix":0[0]'
sudo media-ctl -d /dev/media0 -l '"msm_csiphy1":1->"msm_csid1":0[0],"msm_csid1":1->"msm_ispif1":0[0],"msm_ispif1":1->"msm_vfe0_rdi1":0[0],"msm_ispif1":1->"msm_vfe0_pix":0[0]'
# Connect CSI0 to ISP0 output
sudo media-ctl -d /dev/media0 -l '"msm_csiphy0":1->"msm_csid0":0[1],"msm_csid0":1->"msm_ispif0":0[1],"msm_ispif0":1->"msm_vfe0_rdi0":0[1]'
# Set resolution to 1920x1080
sudo media-ctl -d /dev/media0 -V '"ov5640 4-0078":0[fmt:UYVY8_2X8/1920x1080 field:none],"msm_csiphy0":0[fmt:UYVY8_2X8/1920x1080 field:none],"msm_csid0":0[fmt:UYVY8_2X8/1920x1080 field:none],"msm_ispif0":0[fmt:UYVY8_2X8/1920x1080 field:none],"msm_vfe0_rdi0":0[fmt:UYVY8_2X8/1920x1080 field:none]
# Capture image
gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=1 ! 'video/x-raw,format=UYVY,width=1920,height=1080' ! jpegenc ! filesink location=image01.jpg
Hope this helps!
-Mani
1 Like
Hey Mani,
That worked to grab a JPEG perfectly! A few differences though, I applied your patch manually to the out-of-tree driver that I posted earlier, and I’m on the 17.09 kernel. However, now I’m having trouble setting the pix interface to do the format conversion to NV12 to allow h.264 encoding via the venc.
So far I’ve tried these pipelines:
media-ctl -d /dev/media0 -l '"msm_csiphy0":1->"msm_csid0":0[0],"msm_csid0":1->"msm_ispif0":0[0],"msm_ispif0":1->"msm_vfe0_rdi0":0[0],"msm_ispif0":1->"msm_vfe0_pix":0[0]'
sudo media-ctl -d /dev/media0 -V '"ov5640 2-0078":0[fmt:UYVY2X8/1280x960 field:none],"msm_csiphy0":0[fmt:UYVY2X8/1280x960 field:none],"msm_csid0":0[fmt:UYVY2X8/1280x960 field:none],"msm_ispif0":0[fmt:UYVY2X8/1280x960 field:none],"msm_vfe0_pix":0[fmt:UYVY2X8/1280x960 field:none],"msm_vfe0_pix":1[fmt:UYVY1_5X8/1280x960 field:none]'
Using Gstreamer-1.0 I tried to send encoded video using the following pipeline:
gst-launch-1.0 -e v4l2src device=/dev/video0 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1 ! v4l2h264enc extra-controls="controls,h264_profile=4,video_bitrate=2000000;" ! h264parse ! rtph264pay pt=127 config-interval=4 ! udpsink host=<local-ip-address> port=2600
This resulted an error: https://gist.github.com/RobGries/566ba578085ad33726af134905e7a01a
Any ideas? I’m going to try the 4.14 kernel with the in-tree driver for the ov5640. I’ll post back with how that goes.
Nevermind, I found out that my implementation of U-Boot wasn’t passing the correct root string to the kernel, this caused it to barf when trying to load the rest of the system from the rootfs.
Sorry for the confusion, now I have a working camera and no ‘green line’ issues. Thanks again, Mani!
1 Like
You have probably seen and fixed this already but to be complete - you want to activate the media links, so the argument in the square brackets must be 1, not 0. Also link with the pix, not with the rdi. So:
media-ctl -d /dev/media0 -l ‘“msm_csiphy0”:1->“msm_csid0”:0[1],“msm_csid0”:1->“msm_ispif0”:0[1],“msm_ispif0”:1->“msm_vfe0_pix”:0[1]’
Hi @Rob_Gries,
By apply patch from Mani manually, did you mean replace the in-tree driver (/drivers/media/i2c/ov5640.c) with the modified D3Engineering driver from your gist and then apply patch from Mani?
Thanks,
Nuntipat
No, I am now using Mani’s patch on the 4.14 kernel.
I suggest that you download the 18.01 release and kernel and apply the patch directly to the ov5640 driver already included in that kernel. No need to mess with D3-Engineering’s driver anymore, the mainline driver works great.
Thank you for the feedback. The downside is that the ov5640 driver in the kernel doesn’t seem to support autofocus. Could you please confirm? By the way, I believe it should be possible to integrate the autofocus firmware from D3Engineering’s driver into the in-tree driver. I will look into this tomorrow and will let you know.
Yes, you are right! There is no autofocus support enabled in the upstream driver. But it should be easy to add with the help of D3’s driver.
Hi, Could you please suggest the command to setup pipeline to capture image in full resolution (2592x1944)? Currently, I’m able to capture single 1920 x 1080 image using the script above and also display a real time preview using glimagesink. Thank you.
Hello, same question here.
I applied Mani’s patch to 18.01 release with the upstream ov5640 driver. I use only one camera on CSI0.
I could grap JPEG at 1920x1080 but couldn’t change the resolution to 2592x1944.
Also, I was not able to stream with h264 encoder.
Considering I upgraded to 18.01 to enjoy the “extra-properties” of the encoder, I am a bit stuck right now.
If anyone succeeded, any help would be appreciated, thank you.
Also,
Please check venus (the hw video encoder/decoder) is correctly intitialized (dmesg | grep venus). You should have 6 video devices in /dev, 4 for the camera subsystem and 2 for the video encoder/decoder susbsystem. Due to recent changes it’s possible that the driver expect firmwares to be in /lib/firmware/qcom/venus-1.8/, if this is the case move all venus.* file from /lib/firmware to /lib/firmware/qcom/venus-1.8.
You also need to follow the gstreamer rebuild procedure available in the release note: https://releases.linaro.org/96boards/dragonboard410c/linaro/debian/18.01/.
I’m able to stream to the h264 encoder, but only from the raw interface for now (which requires extra encoding to nv12):
media-ctl -d /dev/media0 -l '"msm_csiphy0":1->"msm_csid0":0[1],"msm_csid0":1->"msm_ispif0":0[1],"msm_ispif0":1->"msm_vfe0_rdi0":0[1]'
media-ctl -d /dev/media0 -V '"ov5640 4-0078":0[fmt:UYVY8_2X8/1920x1080 field:none],"msm_csiphy0":0[fmt:UYVY8_2X8/1920x1080 field:none],"msm_csid0":0[fmt:UYVY8_2X8/1920x1080 field:none],"msm_ispif0":0[fmt:UYVY8_2X8/1920x1080 field:none],"msm_vfe0_rdi0":0[fmt:UYVY8_2X8/1920x1080 field:none]'
gst-launch-1.0 -e v4l2src device=/dev/video0 ! video/x-raw,format=UYVY,width=1920,height=1080 ! videoconvert ! v4l2h264enc extra-controls="controls,h264_profile=4,h264_level=10,video_bitrate=2500000" ! h264parse ! mp4mux ! filesink location=video.mp4
The following pipeline exercises full hardware encoding/decoding on the camera stream (pretty useless):
camera->h264-encode->h264-decode->display
gst-launch-1.0 -e v4l2src device=/dev/video0 ! video/x-raw,format=UYVY,width=1920,height=1080 ! videoconvert ! v4l2h264enc extra-controls="controls,h264_profile=4,video_bitrate=2000000" ! h264parse ! v4l2video4dec capture-io-mode=dmabuf ! glimagesinkAny hints at getting a full (2592x1944) image?
So far I’ve tried the following with the 4.14 kernel -
Results in pipeline not negotiated from Gstreamer:
media-ctl -d /dev/media0 -l '"msm_csiphy0":1->"msm_csid0":0[1],"msm_csid0":1->"msm_ispif0":0[1],"msm_ispif0":1->"msm_vfe0_pix":0[1]'
media-ctl -v -d /dev/media0 -V '"ov5640 2-0078":0[fmt:UYVY2X8/1920x1280 field:none],"msm_csiphy0":0[fmt:UYVY2X8/2592x1944 field:none],"msm_csid0":0[fmt:UYVY2X8/2592x1944 field:none],"msm_ispif0":0[fmt:UYVY2X8/2592x1944 field:none],"msm_vfe0_pix":0[fmt:UYVY2X8/2592x1944 field:none],"msm_vfe0_pix":1[fmt:UYVY1_5X8/2592x1944 field:none]'
GST command used:
gst-launch-1.0 v4l2src device=/dev/video3 num-buffers=1 ! 'video/x-raw,format=UYVY,width=2592,height=1944' ! jpegenc ! filesink location=image01.jpg
Results in gstreamer crashing with a stacktrace:
sudo media-ctl -V -d /dev/media0 -l '"msm_csiphy0":1->"msm_csid0":0[1],"msm_csid0":1->"msm_ispif0":0[1],"msm_ispif0":1->"msm_vfe0_rdi0":0[1]'
sudo media-ctl -v -d /dev/media0 -V '"ov5640 2-0078":0[fmt:UYVY2X8/2592x1944 field:none],"msm_csiphy0":0[fmt:UYVY2X8/2592x1944 field:none],"msm_csid0":0[fmt:UYVY2X8/2592x1944 field:none],"msm_ispif0":0[fmt:UYVY2X8/2592x1944 field:none],"msm_vfe0_rdi0":0[fmt:UYVY2X8/2592x1944 field:none]'
GST Command used:
gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=1 ! 'video/x-raw,format=UYVY,width=2592,height=1944' ! jpegenc ! filesink location=image01.jpg
Hi I am also using 4.14 kernel on dragon 410c board. But I am unable to generate the test pattern. v4l-subdev also not registered.
I checked both the files and both are same as yours, the modifications which you made were already there. Still it is not getting.
Please help me to solve this issue…
@Rob_Gries I know I am late to the party but I’m fiddling around with the Banana Pi M2 Zero for a project which does support the ov5640 with Armbian. I was wondering if there is a driver somewhere which supports autofocus. The official driver doesn’t and I found your gist which indiciates that you managed to make it work, but I’m not sure if this is something that can be used. Could you help me out here?
We are also facing the same issue on qcomlt-5.15 kernel 820 SoM. Any luck further on this?
Note:
On the same platform(820) with kernel 4.14, ov5640 sensor is working fine for us…
