Hi,
I want to write an algorithms, which suppose to run in real time. I searched for linux RT with board support package (drivers), but did not find one that build already. Also, I did not found any documentation and API on how to use Hexagon 500 DSP and OpenCL support for the GPU.
What is the straight forward approach for using DragonBoard for real time applications, such as robotics?
Thanks, Yuri
You can have a look at Xenomai [0] and this post [1] from some time ago.
what sort of realtime application do you want to do?
At Xerox I used Xenomai in the past to control image processing ASICs from user space [2] but the use cases are too many to mention (ie, NXP came with a recent offering that includes Xenomai as its real-time distro provider [3], it is use for Audio [4] …)
fyi, Linaro 96boards releases do not offer any sort of real-time support in their kernels btw
[0] http://www.xenomai.org
[1] Servo Motor Control: generating a stable PWM signal with Xenomai - 96Boards
[2] http://www.denx.de/en/pub/News/Xum2009AbstractsAndPresentations/Xenomai_and_Realtime_Image_Processing_Control.pdf
[3] http://www.openil.org/
[4] http://linuxgizmos.com/bb-black-audio-cape-promises-1ms-latency/
Thank you for the suggestion.
First, I want to elaborate on the application. I want to try to build a walking robot. The robot inputs are video camera (MIPI_CSI), accelerometers, gyros and force sensors (strain gauges) on the links. The outputs are control signals to tubular motors. I sample the sensors, and perform two algorithms: one for motion control, and one for Localization and Mapping (kind of SLAM). I can suffer 100-150ms processing delay, but the output and inputs have to be done with precise intervals.
Second, I saw Linaro Stable Kernel with RT patch in the following link (https://wiki.linaro.org/LSK). If you say, that Linaro does not provide RT, then I do not understand what these kernels are for.
Regarding the Xenomai. I am not familiar with Real Time OS. In two times I had to implement RT algorithm, I did it on bare metal (Netduino and SHARC SC584). It was very simple, only one thread on each core and interrupts, with no scheduler. So, I afraid that relatively esoteric dual-kernel Xenomai entering level is too high, unlike standard Linux program. Also, if RT patch will be accepted by the mainline kernel, it will become de-facto RT standard, which will increase the support and number of available tutorials.
I though if Qualcomm wanted to push it to medical and automotive, it already provided full programming stuck.
From what I said earlier you can infer that the Linaro 96Board releases do not ship the LSK - but you are welcome to use it of course.
If you are interested perhaps you should read the documentation, bibliography and google a bit before making your decision based on assumptions (seems to me that you are really misinformed).
Is not usually the case that a whole system needs to be realtime (usually it is only portions of it, ie some control paths). It is in those use-cases where Xenomai shines.
But sure, you can wait for PREEMPT_RT to be fully featured and accepted in mainline. It might take a few years but it should happen since the Linux Foundation is backing it up.
If you’re looking for a really simple approach then I’d consider running the motion/balance on a fast microcontroller mezzanine (STM32 Sensor Mezzanine Board - 96Boards is an example) and run CV, navigation and high level decision making on the DB410C.
To be honest this is merely using extra hardware to do the same partitioning one would do using Xenomai. Whether this is better for you really depends on your experience and needs (and whether a micro-controller is powerful enough to manage the motion/balance code).
I see. I read your article on software PWM. I will look into Xenomai.
Thank you
Btw I fully agree with Daniel’s comment; notice that real-time Linux is not a trivial matter. So for a quick prototype a mezzanine might be the best choice; later on, if the volume justifies the cost savings, a single board realtime solution makes more sense.
another alternative - unfortunately not supported by db410c - would be Siemens’ Jailhouse [1]; I don’t remember the details of why adding support for it was a bit of a pain but it might be something you could also explore.
Incidentally It’s author (Jan Kizska) will be the new Xenomai maintainer - this announcement happened just last month in the Xenomai mailing list.
ldts http://discuss.96boards.org/u/ldts
January 8another alternative - unfortunately not supported by db410c - would be
Siemens’ Jailhouse [1]; I don’t remember the details of why adding
support for it was a bit of a pain but it might be something you could
also explore.
I think the problem will be the same one that prevents Xen and KVM
working on DB410C: the Dragonboard bootloaders to not permit access to
the EL2 exception level needed to exploit the virtualization support.
yes, that is what it was - you are right as usual 
The option with micro-controller seems good, but I think it it really overkill. The only real time thing I need is to run short function, which sets outputs (GPIO or sends some data over I2S) periodically, each 50ms. So I do not need real time scheduler with priorities and multiple threads switching to meet the deadlines. Can it be done in the following way: some RTC (or ARM clock register) will connect to interrupt, which will fire each 50ms. Then this function will run. Can you think of some simple configuration like this?
The benefit is that I will use standard Linaro, but it allows many real time projects to run.
Hi @Yuretsss
It all comes down to your definition of “Real Time”.
In general Unix/Linux is not formally considered to be a real time kernel because there is no absolute guarantee of response time, in the worst case (when the CPU is heavily overloaded and swapping) it is theoretically possible to take many seconds to respond to an interrupt. This of course rules out using a stock Linux kernel in a life critical system like your ABS braking system.
I have found that on lightly loaded 410c system the interrupt response time is almost never longer than 100us, your interrupts are once every 50ms, and if you can tolerate up to, say, 1ms delay then stock Linux is “real-time-enough” for your application. In general you will be running vision (OpenCV), and walking and not overloading the CPUs so your 2kg walking robot will stay upright, but if you have a 1000kg walking robot and when it does fall over it could kill someone then you shouldn’t be using Linux.
OpenCL and Hexagon support are only in Android, we don’t have them available for Linux at this time.
Full Disclosure: I am an employee of Qualcomm Canada, any opinions I may have expressed in this or any other post may not reflect the opinions of my employer.
So you say that in [1], [2] and [3] the SDKs are for android only. The different versions are for different host OS.
By the way, does Qualcomm has something like TI-RTOS from TI, which completely supports their hardware?
[1] https://developer.qualcomm.com/software/adreno-gpu-sdk
[2] Hexagon DSP SDK - Qualcomm Developer Network
[3] Snapdragon LLVM Compiler - Qualcomm Developer Network
Maybe off topic @Yuretsss but just remembered this on the mailing list [1]
"
A position for a roboticist / software-engineer is available at the Max
Planck Institute for Intelligent Systems in Tuebingen (Germany).
The selected candidate will work on improving our software architecture
for the realtime control of our humanoid robots (which uses xenomai).
"
There are some good videos of the humanoids running realtime Linux via co-kernel.
[1] http://xenomai.org/pipermail/xenomai/2017-December/038191.html
To summarize the discussion. Snapdragon 410e has a rich android environment for robotics. It contains very efficient hardware utilization algorithm, which includes the following packages: Snapdragon Navigation, Machine Vision, GPGPU (OpenCL) and DSP support. All Qualcomm demonstrations work with no real time support. I still have no idea how their products are functioning. There is no headless Android. Also, it demands to write the code in the Android ecosystem.
The solution of external controller solves the problem partially, only if one is able to push all its real time code into the controller. So I think that the use of BeagleBoard solves the problem. First, it has it real time OS - TI-RTOS. Second, it has 200MHz PRU, which serves for the real time signals sampling and synchronization. The non-real time DragonBoard connects to BeagleBoard with SPI. The sensors connect to PRU. The only sensors on DB410c are camera, build-in accelerometer and gyro (I hope they are synchronized).
The issue is to synchronize the camera frame timespamp with BeagleBoard sensors timestamp. The Advantage of this configuration is the use only of supported tools, without dancing around the bonfire with shaman drum to get things work.
in case you decide to give Xenomai a try I updated the dual kernel support (ipipe/Xenomai) for the db410c on a 4.9 kernel.
Hi,
I am not able to access this link:
Nor any one of this:
- Download ipipe-jro-410c.linaro-15.07
(From URL: http://git.xenomai.org/ipipe-jro.git/tag/?h=410c.linaro-15.07
File name: ipipe-jro-410c.linaro-15.07.tar.bz2 )
I am trying to bring up Xenomai-3.0.6 with Linux Kernel 4.9.xx on Dragon Board 410c.
Can you help me with it.
Note: I have already done Xenomai-3 bring up on Ubuntu (x86), arm (BeagleBone, Rpi3), Hikey (arm64), so I am familiar with Xenomai porting.
However, now I am trying to do it on Qual Comm chipset.
Thanks,
@Yuretsss just a quick follow up have a look at this co-kernel
https://www.evenless.org/
https://git.evenless.org/linux-evl.git
it is simpler, follows mainline, should work on any arm/arm64 board and you can run it within QEMU. (I validated it on db410c running 5.0-rc1)
