HFP Client SCO audio routing?

doitright · January 25, 2018, 6:57pm

I was able to reproduce failure to open stream issue. The reason why it won’t work on subsequent attempts after it fails to open something once, is because there remain the pcm’s that were opened prior to the one that failed to open. So for instance, if it is going to fail, it will generally fail on one of the “near” streams, which means that the far streams remain open, and therefore can’t be opened a second time. I’ve added a fix for that to my device tree.

doitright · January 25, 2018, 9:10pm

This should safely handle the multiple-streams issue;

IsaacChang · January 26, 2018, 12:09am

Thank you.
It works well now.

doitright · January 29, 2018, 3:37am

I have HFP working on boot now.

From AOSP Master, needs these changes;

My device tree; https://github.com/lbdroid/android_device_linaro_hikey
My kernel; https://github.com/lbdroid/android_kernel_linaro_hikey
Change this line; https://android.googlesource.com/platform/system/bt/+/master/bta/hf_client/bta_hf_client_at.cc#1650
To read “AT+BAC=1\r”

*** Despite the code being in the hal, audio processing (echo cancellation) is not yet working. I have some pointers I need to straighten out.

doitright · January 29, 2018, 4:21pm

I now have WebRTC audio processing working.
It is very thoroughly cancelling out echo.
I tested it by putting the microphone right against the speaker on the hikey960 and taking my tablet (using for testing with hangouts dialer) and speaking into it – zero echo Previously, it was creating a very strong echo with the same procedure.

doitright · January 29, 2018, 5:22pm

Looks like the alsa driver method of setting up the PCM is a bit unreliable.
The outbound audio seems to work reliably, but incoming is about 50/50 going to be just noise. I think its because the PCM is being set up too late in the process, probably after the SCO stream is already running, or maybe on the verge of running.

doitright · February 2, 2018, 3:50am

I think I know how it is breaking the data. Not why, just how. Opening data captures in a hex editor and comparing good data with broken, it appears like the bad captures have their endianness reversed.

Tomorrow I’ll write a test program to switch endianness and see if that clears up the broken sample. If it does, then in the worst case, I should be able to detect it mathematically and repair it on the fly in the HAL.

doitright · February 2, 2018, 1:05pm

And that is confirmed!
It randomly switches endianness. I ask for PCM_FORMAT_S16_LE, and 50% of the time it gives me PCM_FORMAT_S16_BE.

I verified it experimentally by repairing a broken audio capture using this, which just reverses the bytes in each sample;

#include <stdio.h>

int main (int argc, char ** argv){
	FILE *readf;
	FILE *writef;
	unsigned char buffer[20];
	int i;

	readf = fopen(argv[1], "rb");
	writef = fopen(argv[2], "wb");

	// first copy 200 bytes header-ish from input to output;
	for (i=0; i<10; i++){
		fread(buffer, sizeof(buffer), 1, readf);
		fwrite(buffer, sizeof(buffer), 1, writef);
	}

	while (fread(buffer, 2, 1, readf) != 0){
		fwrite(&buffer[1], 1, 1, writef);
		fwrite(&buffer[0], 1, 1, writef);
	}

	fclose(readf);
	fclose(writef);	

	return 0;
}

doitright · February 2, 2018, 7:58pm

And the solution;

When the endianness is reversed, the average difference between samples grows by about a factor of 10 (experimentally). In general, good audio will have each subsequent sample being a relatively small difference from the one before it, resulting in a smooth wave. So for the first few seconds of the stream, I calculate the sum of the absolute differences between each subsequent audio frame for both “as received” and reverse byte order, and whichever one is the smallest is the mode I take as valid. I’ve tested it roughly 50 times, and it in every test, the resulting audio has been correct. Previously, it was about 50/50.

doitright · February 17, 2018, 2:34am

I finally got around to enabling and testing 16 kHz audio.
Funny thing is that I committed the code for it to my git 20 days ago, but didn’t enable it until today. Its working perfectly

billeee · February 22, 2018, 7:29am

Sorry to ask a question when the discussion is cold, but do you know where those I2S2 resistors are on the board ? Any points to a diagram or something of that nature that publicly available ?

doitright · February 22, 2018, 3:47pm

I doubt that it will be safe at all for you to mess around with those resistors. Its either going to be in the nest on the bottom of the board, or some of the tiny ones on the top of the board right up against the SoC.

Why do you need to know? If you need to access the i2s2 lines, you can access them right against the edge of the Wilink8.

danielt · April 5, 2018, 12:58pm

A post was split to a new topic: Bluetooth A2DP issue on dragonboard 410c

shanvasu · April 27, 2018, 3:49am

Hi doitright,

I am trying to use I2S2 to connect to an external Codec. Looks like you have done some amazing work figuring out BT to loopback I2S2. If you could answer these questions, I will really appreciate it.

If I want to connect I2S2 output to an external codec, can I just use the pins from
WL1837MOD (pins 56,57, 58,60) ?
Do I have to setup BT to be able to use the I2S output?
If I have to setup BT, what kernel /user space change do I have to use to get I2S2 output to external Codec?

PS: I already routed the 4 I2S pins from WL1836 and connected to a Saleae logic analyzer. And when play the audio through tinyplay (tinyplay xxx.wav –d 0) where device 0 is HDMI Audio through I2S2, I can hear the audio from HDMI. But I don’t see any output in I2S lines in Saleae.

Again your help here would greatly be appreciated.

Thanks!

doitright · April 27, 2018, 12:45pm

There is no loopback involved in what I did with it. The i2s2 pins on the SoC are directly connected to the bluetooth, and the bluetooth transmits the audio data to (and receives from) a paired cell phone. The other side of what I did routes that data to a USB audio card.

I don’t think it is at all practical to use i2s2 to connect to any codec besides the HDMI and Bluetooth. If there was a header available, then sure, but there isn’t, and the only option for attaching to those lines would provide a very fragile connection.

Yes, those are the correct pin numbers. Datasheet: http://www.ti.com/lit/ds/symlink/wl1807mod.pdf — you will note that the side with those pins is the side facing towards the SoC.
Yes, you need to send a special command to the bluetooth chip in order to enable the i2s. See relevant kernel commits;
https://github.com/HiKey960-Car/android_kernel_linaro_hikey/commit/61631d79cfe9ed762654ac0e0a2d412041b6e7e9
https://github.com/HiKey960-Car/android_kernel_linaro_hikey/commit/c8a7b77db20d0c70f906fcaca2deba1945072207
https://github.com/HiKey960-Car/android_kernel_linaro_hikey/commit/2f5ab1a0d51afad148bdceded17419b30025ba6b
https://github.com/HiKey960-Car/android_kernel_linaro_hikey/commit/7f69be32a82ca5c195bde0e6d83225c99b97f1b3
https://github.com/HiKey960-Car/android_kernel_linaro_hikey/commit/22d7578139793295f99bcffc8d5987150036e45e

If you aren’t seeing the data on the logic analyzer when playing to the HDMI output, then it means that your logic analyzer isn’t working. Which logic analyzer do you have? If its a genuine saleae, it would have cost at least $100. Possibly as much as $1000. As far as I’ve seen, NONE of the fake saleae’s support 1.8 volt logic, and of the genuine ones, not all of them support 1.8 volt. For example, my genuine saleae logic 4 does NOT support 1.8 volt logic, so to probe basically anything on the hikey960, I have to work through a level shifter.

But you can rest assured that if there is sound coming out the HDMI, then there is definitely data on the i2s pins beside the bluetooth chip.

Just make sure that you keep in mind that like the HDMI, the bluetooth chip is the external codec.

Can you describe what you are trying to accomplish with the i2s2 pins? There may be a better way than trying to hijack those pins.

Island_Zhao · July 5, 2018, 12:13pm

Hi, doitright

Your development is very helpful to me. And I apply for your patch. The pcm in and out could display successfully. But my code is Mar 2018. My native HFP and A2DP sink still could not run successfully. Part of log is as follows,
07-05 09:40:18.465 D/BluetoothHeadsetClientServiceJni( 3725): connection_state_cb: state 0 peer_feat 0 chld_feat 0
07-05 09:40:18.467 E/bt_btif ( 3725): btif_hf_client_get_cb_by_bda: could not find block for bdaddr
07-05 09:40:18.467 E/bt_btif ( 3725): btif_hf_client_upstreams_evt: event 3 but not allocating block: cb not found
07-05 09:40:18.467 I/bt_btif_queue( 3725): queue_int_advance: removing connection request: address=bc:a9:20:1c:24:60 UUID=111E busy=true
07-05 09:40:18.467 D/HeadsetClientStateMachine( 3725): Connecting process message: 100
07-05 09:40:18.467 D/HeadsetClientStateMachine( 3725): Connecting: event type: 1
07-05 09:40:18.467 D/HeadsetClientStateMachine( 3725): Connecting: Connection BC:A9:20:1C:24:60 state changed:0
07-05 09:40:18.467 D/HeadsetClientStateMachine( 3725): Exit Connecting: 100
07-05 09:40:18.467 D/HeadsetClientStateMachine( 3725): Enter Disconnected: 100
07-05 09:40:18.468 D/HeadsetClientStateMachine( 3725): Connection state BC:A9:20:1C:24:60: 1->0
07-05 09:40:18.471 I/LocalBluetoothProfileManager( 2550): Failed to connect HEADSET_CLIENT device

07-05 09:39:02.633 E/BluetoothManagerService( 2407): Fail to bind to: Intent { act=android.bluetooth.IBluetoothHeadset }
07-05 09:39:02.633 W/BluetoothManagerService( 2407): Unable to bind with intent: Intent { act=android.bluetooth.IBluetoothHeadset }
07-05 09:39:02.633 V/vold ( 2131): Waiting for FUSE to spin up…
07-05 09:39:02.633 E/BluetoothA2dp( 2407): Could not bind to Bluetooth A2DP Service with Intent { act=android.bluetooth.IBluetoothA2dp }

Could you help give some suggestions to me?
Thank you so much for your great help!

doitright · July 5, 2018, 12:25pm

Ok, well first thing is that you would be a lot better off with the APRIL2018 (that’s April 20, 2018), and apply it all against a known good manifest from close to the same date. You can find known good manifests here; https://github.com/96boards/aosp-known-good-manifests .

Also make sure that you are working from my gitLAB repositories; HiKey960-Car · GitLab
I’m not maintaining the ones on github.

I’m not entirely sure what to make out of your log, its a very small excerpt and doesn’t have any context with it. The vold/FUSE error is something I’ve seen before, and tells me that your system is not in a good state.

Island_Zhao · July 6, 2018, 10:28am

Thank you so much for your great help~
Now, I’m sure the music data transfer to hikey960 from my iphone. But there is no sound in the headphone. D I use the usb audio device is Audio Adapter. I disable TI wifi&BT and connect the external Marvell&Qualcomm Wifi&BT by sdwifi and uart bt. Do you have the email? could I communicate with you by email? If I succeed, I will provide the solution in this topic web.
Thank you so much.
log as follows,
07-06 09:41:56.399 E/bt_btif ( 3715): bta_av_proc_stream_evt: p_data is null
07-06 09:41:56.399 I/bt_bta_av( 3715): bta_av_link_role_ok: hndl:x41 role:1 conn_audio:x1 bits:1 features:x825b
07-06 09:41:56.399 W/bt_btif ( 3715): bta_dm_rm_cback:2, status:0
07-06 09:41:56.399 W/bt_btif ( 3715): bta_dm_rm_cback:2, status:7
07-06 09:41:56.399 I/bt_stack( 3715): [INFO:btm_pm.cc(195)] BTM_SetPowerMode already in requested mode 0, interval 0 max 0 min 0
07-06 09:41:56.399 I/bt_stack( 3715): [INFO:btm_pm.cc(195)] BTM_SetPowerMode already in requested mode 0, interval 0 max 0 min 0
07-06 09:41:56.400 D/android.hardware.bluetooth@1.0-impl( 3682): Send, Debug
07-06 09:41:56.400 D/android.hardware.bluetooth-hci-h4( 3682): Send, Debug
07-06 09:41:56.400 D/android.hardware.bluetooth-hci-h4( 3682): H4Protocol::Send: WriteSafely uart_fd = 7
07-06 09:41:56.400 D/android.hardware.bluetooth-hci-hci_protocol( 3682): WriteSafely, Debug, length = 11
07-06 09:41:56.400 D/android.hardware.bluetooth-hci-hci_protocol( 3682): writeSafely fd = 7
07-06 09:41:56.400 I/btif_av ( 3715): virtual bool BtifAvStateMachine::StateOpened::ProcessEvent(uint32_t, void *): Peer bc:a9:20:1c:24:60 : event=BTA_AV_START_EVT(0x4) status=0 suspending=0 initiator=0 flags=0x2(REMOTE_SUSPEND)
07-06 09:41:56.400 I/bt_btif_a2dp_sink( 3715): btif_a2dp_sink_set_rx_flush: enable=false
07-06 09:41:56.400 I/btif_av ( 3715): btif_report_audio_state: peer_address=bc:a9:20:1c:24:60 state=2
07-06 09:41:56.400 I/BluetoothA2dpSinkServiceJni( 3715): bta2dp_audio_state_callback
07-06 09:41:56.401 D/android.hardware.bluetooth@1.0-impl( 3682): sendHciCommand, Debug
07-06 09:41:56.401 D/android.hardware.bluetooth@1.0-impl( 3682): Send, Debug
07-06 09:41:56.401 D/android.hardware.bluetooth-hci-h4( 3682): Send, Debug
07-06 09:41:56.401 D/android.hardware.bluetooth-hci-h4( 3682): H4Protocol::Send: WriteSafely uart_fd = 7
07-06 09:41:56.401 D/android.hardware.bluetooth-hci-hci_protocol( 3682): WriteSafely, Debug, length = 8
07-06 09:41:56.401 D/android.hardware.bluetooth-hci-hci_protocol( 3682): writeSafely fd = 7
07-06 09:41:56.401 D/A2dpSinkStateMachine( 3715): Connected process message: 101
07-06 09:41:56.401 D/A2dpSinkStateMachine( 3715): processAudioStateEvent in state 2
07-06 09:41:56.402 D/A2dpSinkStreamHandler( 3715): process message: 0
07-06 09:41:56.402 D/A2dpSinkStreamHandler( 3715): audioFocus = 0
07-06 09:41:56.402 D/A2dpSinkStreamHandler( 3715): sendAvrcpPause
07-06 09:41:56.403 D/A2dpSinkStreamHandler( 3715): Pausing AVRCP.
07-06 09:41:56.403 V/AvrcpControllerService( 3715): sendPassThroughCmd keyCode: 70 keyState: 0
07-06 09:41:56.404 V/AvrcpControllerService( 3715): sendPassThroughCmd keyCode: 70 keyState: 1
07-06 09:41:56.404 D/AvrcpControllerSM( 3715): HandleMessage: REQ_PASS_THROUGH_CMD
07-06 09:41:56.404 I/BluetoothAvrcpControllerJni( 3715): sendPassThroughCommandNative: sBluetoothAvrcpInterface: 0x74005a4c90
07-06 09:41:56.404 I/BluetoothAvrcpControllerJni( 3715): key_code: 70, key_state: 0
07-06 09:41:56.404 E/bt_btif ( 3715): send_passthrough_cmd: calling btif_rc_get_device_by_bda
07-06 09:41:56.405 D/AvrcpControllerSM( 3715): inform AVRCP Commands to A2DP Sink
07-06 09:41:56.405 D/android.hardware.bluetooth@1.0-impl( 3682): Send, Debug
07-06 09:41:56.405 D/android.hardware.bluetooth-hci-h4( 3682): Send, Debug

doitright · July 6, 2018, 1:15pm

I prefer to keep discussions like this on the forum so that other people will be able to read it and learn from it.

Here is the thing; I did nothing at all with respect to A2DP/AVRCP. The guys at hisilicon did all that, and it was working before I even started with this board. So really, I can’t answer much about that except to say that IN THEORY, you should be able to make those work over a different BT adapter, since they run their data over HCI, which is carried over UART. But be aware that your UART would have to be running at a high enough rate to carry the audio data that you are sending over the link.

There is one other thing that I want to make sure you are aware of with regards to A2DP: It will not output sound unless you hit the PLAY button in the bluetooth music application on the hikey960.

Now regarding HFP… you see, it would have been very helpful if you had mentioned previously that you were using a different BT adapter. While technically, HFP audio can be run over HCI protocol in much the same way as A2DP, Android does not support this. While there is some code in AOSP regarding HFP over HCI, this code was never completed, has never been tested, and has never been maintained. That means that HFP audio must originate in some manner that is accessible to the AUDIO HAL, which typically means an ALSA device. If your bluetooth adapter has I2S capability, then you could attach it to the I2S pins of the LS adapter and configure simple-card to work with it. IF this is possible for your hardware, and you want to try it out, then be aware that my april kernel branch does NOT have the patch needed to enable i2s0 on the LS adapter. You can find the branch that ends with “CAR96” and cherry-pick these two commits from it;

c3080aaada9f8a97825bb050eb69788376998c7d
077e0189be3a6f1bd24aceb6c18872a066ed8be0
(* These first of those two commits will have conflicts that you will need to resolve)

In addition, understanding this commit will help you in writing a codec driver for your bt card;
71e95cfc04322263133721f7a3e7d623c92c71fe
Note that everything in the bt_sco_hw_params function is specific to the on-board wilink8 bt.

Do you mind if I ask you WHY you are using a separate BT adapter? In the very least, I would VERY STRONGLY recommend that you get it working with the built-in bluetooth card first, and then switch to yours.

Island_Zhao · July 9, 2018, 3:33am

Hi, doitright

Thank you very much for your great help! As I have tried your A2DP sink solution with build TI, it worked well, now my A2DP could work with Marvell and Qualcomm sd-Wifi & uart-BT card, not build in TI card. Sorry for late response as I won’t disturb you in your nice weekend. For A2DP, I comply with your advice: There is one other thing that I want to make sure you are aware of with regards to A2DP: It will not output sound unless you hit the PLAY button in the bluetooth music application on the hikey960. And it really worked. I could hear the music from the hikey960 and the music is from my iPhone. Thank you so much again.
For HFP, I need to debug this in details according to your very helpful advice.
For your last question, I’m an Android developer and interested in Wireless card. As Google prepared all the TI wifi bt card code that vendor could modified such as /device/linaro/hikey compile code, wifi&bt hal code, sepolicy code, and even TI wifi&bt driver code, kernel config file. I want to study more knowledge for wifi&bt from kernel, Android HAL, native, framework to wifi&bt APP. So I purchased Marvell&Qualcomm wifi&bt card to develop for all knowledge and then we could compare TI, Marvell and Qualcomm card index such as throughput and so on . I want to learn of all bring up process. Thank you so much for your precious advice. I will continue to debug the HFP function by follow your advice.