Hi. We have a custom board with a snapdragon 820. On a MIPI camera port we have MIPI data coming in from a chip that converts HDMI data to MIPI. With the camera outputting 1080p60, 4 mipi data lanes, the image looks great. We then switch the camera to output 4kp24, 4 mipi data lanes and the image looks skewed. I have tried adjusting settle-cnt in the kernel code to get the image to look correct, but still no change. I’ve attached a picture of what I get. There should be 8 color bars. So each color should be 480 pixels, but I only get ~303 pixels per color bar. Can anyone help me understand why this would be? thanks, Kim
Could you please share which command you’re running for configuring the pipeline (in both cases).
Hi Loic,
In the case of the 1080p60 I send these two commands to configure the pipeline:
sudo media-ctl -d /dev/media0 -l ‘“msm_csiphy0”:1->“msm_csid0”:0[1],“msm_csid0”:1->“msm_ispif0”:0[1],“msm_ispif0”:1->“msm_vfe0_rdi0”:0[1]’
sudo media-ctl -d /dev/media0 -V ‘“lt6911 5-002b”:0[fmt:UYVY8_2X8/1920x1080],“msm_csiphy0”:0[fmt:UYVY8_2X8/1920x1080],“msm_csid0”:0[fmt:UYVY8_2X8/1920x1080],“msm_ispif0”:0[fmt:UYVY8_2X8/1920x1080],“msm_vfe0_rdi0”:0[fmt:UYVY8_2X8/1920x1080]’
for the 4k capture, I send these two commands:
media=sudo media-ctl -d /dev/media0 -l ‘“msm_csiphy0”:1->“msm_csid0”:0[1],“msm_csid0”:1->“msm_ispif0”:0[1],“msm_ispif0”:1->“msm_vfe0_rdi0”:0[1]’
media=sudo media-ctl -d /dev/media0 -V ‘“lt6911 5-002b”:0[fmt:UYVY8_2X8/3840x2160],“msm_csiphy0”:0[fmt:UYVY8_2X8/3840x2160],“msm_csid0”:0[fmt:UYVY8_2X8/3840x2160],“msm_ispif0”:0[fmt:UYVY8_2X8/3840x2160],“msm_vfe0_rdi0”:0[fmt:UYVY8_2X8/3840x2160]’
I do have a second camera connected and it’s media-ctl configuration is this (in both cases):
sudo media-ctl -d /dev/media0 -l ‘“msm_csiphy2”:1->“msm_csid1”:0[1],“msm_csid1”:1->“msm_ispif1”:0[1],“msm_ispif1”:1->“msm_vfe0_rdi1”:0[1]’
sudo media-ctl -d /dev/media0 -V ‘“adv7611 1-004c”:0[fmt:UYVY8_2X8/640x480],“msm_csiphy2”:0[fmt:UYVY8_2X8/640x480],“msm_csid1”:0[fmt:UYVY8_2X8/640x480],“msm_ispif1”:0[fmt:UYVY8_2X8/640x480],“msm_vfe0_rdi1”:0[fmt:UYVY8_2X8/640x480]’
thanks,
Kim
Hi Loic,
I captured the raw data in a buffer for multiple frames and I noticed something odd. The mipi packet header is always in the first 4 bytes of the captured image. Throughout the buffer, at close to, but not exactly the width of a line, I see the same packet header. The bytes are:
0x2b - Data ID YUV
0x1e00 - Word Count (3840*2)
0x3a - ECC
Can you explain why the packet header would show up in the capture buffer? I’m confused.
Thanks,
Kim
Interesting, do you have a way to progressively change resolution to check at which one you start seeing this wrong behavior? I also suggest to add some debug into camss CSI/VFE code to check if configured values/registers are coherent…
Hi Loic,
Yes, I am able to change the resolution to a point where I can at least see something that looks like the color bar pattern at 2496 width, 2160 height. In the image below, you can see there is a green line running diagonally through the image. This is the header packet and seems to always be captured.
Initially I thought this was a settle-cnt issue. I modified the settle-cnt through each of the three possible csi frequencies (100, 200 and 266MHz), but with no better results. Do you think this is a signal integrity issue with our custom board?
Thanks,
Kim
Well, don’t know, you should probably scope the signal.
Maybe it would be interesting to also test with 2 CSI lane instead of 4, this is the configuration I usually run.
Hi @Kimbo
Since this is a custom board, you really need to simulate the MIPI-CSI signal integrity, impedance and cross talk, there is info in on how to do this in one of the 410c design guides I wrote while I was at Qualcomm. Also make sure you didn’t swap lanes or invert polarity, MIPI is fussy about that. You can do some variants on lane swapping, but not all combinations of 4 lanes are possible.
It is REALLY difficult to check MIPI signals with a scope (but it can be done if you have lots of high speed 1GHz+ differential signal measurement experience). Remember you are probing at the middle of a differential transmission line and what you see will differ from what the Tx and Rx devices see. The best way to figure out what you should see so you can compare to what you do see is to simulate and compute what the scope will see. However if you had simulated and all was good, it would just work.
-L-
Hi Loic and ljking,
I finally got an image of the 4kp25, 4 mipi data lanes,by forcing the csi clock frequency to 266MHz by setting the camera driver V4L2_CID_PIXEL_RATE=0. Then I set a settle-cnt = 15 in the csiphy_lanes_enable function of camss-csiphy-3ph-1.0c!
Can someone explain this? I had filed another ticket on how to set V4L2_CID_PIXEL_RATE and the response made sense How to set V4L2_CID_LINK_FREQ and V4L2_CID_PIXEL_RATE? - #2 by Loic. So in my case V4L2_CID_PIXEL_RATE should be 3840 * 2160 * 25=207,360,000. But in the camss_csiphy.c, csiphy_set_clock_rates function, the frequency is determined by:
u64 min_rate = pixel_clock * bpp / (2 * num_lanes * 4); //207,360,000 * 8/32 = 51,840,000, which would select the 100MHz frequency. And then I would get the horrible image as shown above.
Another thing I tried was to set the V4L2_CID_PIXEL_RATE in my driver to a high value, 800,000,000, in order to force the csi clock to 266MHz (810M*8/32=202,500,000), but there is an overflow in the calculation above, so I had to cast to u64 for it to correctly set to 266MHz. And not in this file alone, the camss-vfe.c and camss-csid.c had to be modified as well.
What am I doing wrong in the calculation of V4L2_CID_PIXEL_RATE and why does a frequency of 266MHz work?
thanks,
Kim
In order to correctly receive the data, the csiphy and csid clock must be >= CSI2 link clock / 4
The CSI2 link clock here is estimated to (pixel_clock * bpp) / ( 2 * num_lanes)
(‘2’ is because CSI2 is DDR).
I think the math is correct here, but after looking a bit deeper in the driver I suspect the bpp (bit per pixel) to be wrongly reported, if you look into camss-csiphy.c or vfe you’ll see a csi_format table:
static const struct csiphy_format csiphy_formats_8x16[] = {
{ MEDIA_BUS_FMT_UYVY8_2X8, 8 },
{ MEDIA_BUS_FMT_VYUY8_2X8, 8 },
{ MEDIA_BUS_FMT_YUYV8_2X8, 8 },
{ MEDIA_BUS_FMT_YVYU8_2X8, 8 },
{ MEDIA_BUS_FMT_SBGGR8_1X8, 8 },
{ MEDIA_BUS_FMT_SGBRG8_1X8, 8 },
{ MEDIA_BUS_FMT_SGRBG8_1X8, 8 },
{ MEDIA_BUS_FMT_SRGGB8_1X8, 8 },
{ MEDIA_BUS_FMT_SBGGR10_1X10, 10 },
{ MEDIA_BUS_FMT_SGBRG10_1X10, 10 },
{ MEDIA_BUS_FMT_SGRBG10_1X10, 10 },
{ MEDIA_BUS_FMT_SRGGB10_1X10, 10 },
{ MEDIA_BUS_FMT_SBGGR12_1X12, 12 },
{ MEDIA_BUS_FMT_SGBRG12_1X12, 12 },
{ MEDIA_BUS_FMT_SGRBG12_1X12, 12 },
{ MEDIA_BUS_FMT_SRGGB12_1X12, 12 },
{ MEDIA_BUS_FMT_Y10_1X10, 10 },
};
The ‘2X8’ formats are reported with 8 bit per pixel but I suppose it should be 16. Fixing that should fix your underrated clock issue… can you give it a shot?
Hi Loic,
I changed the YUV formats of this table to these values:
static const struct csiphy_format csiphy_formats_8x96[] = {
{ MEDIA_BUS_FMT_UYVY8_2X8, 16 },
{ MEDIA_BUS_FMT_VYUY8_2X8, 16 },
{ MEDIA_BUS_FMT_YUYV8_2X8, 16 },
{ MEDIA_BUS_FMT_YVYU8_2X8, 16 },
I then set the V4L2_PIXEL_RATE = 207,360,000 in the camera driver and built everything. I am back to the image that I originally had 
The calculated csi freq=200MHz, with a settle-cnt of 12.
Yesterday I was trying some different V4L2_PIXEL_RATE in the camera driver. These were the results I got:
| pixel_rate(driver) pixels/s | Settle-cnt (calculated) | Csi frequency (calculated) | Result |
|---|---|---|---|
| 800 | 19 | 266MHz | works |
| 700 | 14 | 200MHz | works |
| 600 | 20 | 200MHz | works |
| 500 | 12 | 200MHz | No Video |
| 550 | 56 | 200MHz | No Video |
| 550 | 23 | 200MHz | No video |
| 400 | 12 | 200MHz | No video |
Thanks,
Kim
Hi Loic,
I made a couple of other changes in order to get the YUV 16 bit formats working with my camera’s pixel rate of 208MB/s.
1.) In camss-csid.c, I changed the csid_formats_8x96, bpp element, from 8->16 for YUV types:
static const struct csid_format csid_formats_8x96[] = {
{
MEDIA_BUS_FMT_UYVY8_2X8,
DATA_TYPE_YUV422_8BIT,
DECODE_FORMAT_UNCOMPRESSED_8_BIT,
16,
2,
2.) In camss-vfe.c, vfe_set_clock_rates funtion, there is a calculation for tmp:
tmp = (u64)pixel_clock[j] * bpp / 32; // was / 64
I changed the 64 to a 32. And now I get video with your suggestion. I’m not sure what the value 64 is in this case, but a value of 32 made things work. Any idea why?
Thanks,
Kim
Hi Loic,
I have another camera module which has a similar issue to the one I mentioned a couple of weeks ago. This camera module has an imx412 sony sensor. Pixel_rate = 4056x3040x40fps = 493,209,600 pixels/s.
It outputs 4 mipi data lanes and is a 10-bit bayer output.
The calculations in the camss source code show: u64 min_rate = pixel_clock * bpp / (2 * num_lanes * 4)
should be
min_rate = 493,209,600 * 10 / (2 * 4 * 4) = 154MHz.
but because of the missing cast of u64 to pixel_clock, pixel_clock * bpp = 0x2559D000, not 0x125F9D000. So min_rate ends up being = 637,128,704 / (2 * 4 * 4) = 19,910,272. And the image looks bad.
If I cast the pixel_clock (in three places of code) to u64 min_rate = (u64)pixel_clock * bpp / (2 * num_lanes * 4) = 154MHz, then I get a good image at the correct frame rate.
What do you think?
Thanks,
Kim
