HFP Client SCO audio routing?

[doitright]

And that is confirmed!
It randomly switches endianness. I ask for PCM_FORMAT_S16_LE, and 50% of the time it gives me PCM_FORMAT_S16_BE.

I verified it experimentally by repairing a broken audio capture using this, which just reverses the bytes in each sample;

#include <stdio.h>

int main (int argc, char ** argv){
	FILE *readf;
	FILE *writef;
	unsigned char buffer[20];
	int i;

	readf = fopen(argv[1], "rb");
	writef = fopen(argv[2], "wb");

	// first copy 200 bytes header-ish from input to output;
	for (i=0; i<10; i++){
		fread(buffer, sizeof(buffer), 1, readf);
		fwrite(buffer, sizeof(buffer), 1, writef);
	}

	while (fread(buffer, 2, 1, readf) != 0){
		fwrite(&buffer[1], 1, 1, writef);
		fwrite(&buffer[0], 1, 1, writef);
	}

	fclose(readf);
	fclose(writef);	

	return 0;
}

[doitright]

And the solution;

Detection and correction for reversed endianness in SCO input

When the endianness is reversed, the average difference between samples grows by about a factor of 10 (experimentally). In general, good audio will have each subsequent sample being a relatively small difference from the one before it, resulting in a smooth wave. So for the first few seconds of the stream, I calculate the sum of the absolute differences between each subsequent audio frame for both “as received” and reverse byte order, and whichever one is the smallest is the mode I take as valid. I’ve tested it roughly 50 times, and it in every test, the resulting audio has been correct. Previously, it was about 50/50.

[doitright]

I finally got around to enabling and testing 16 kHz audio.
Funny thing is that I committed the code for it to my git 20 days ago, but didn’t enable it until today. Its working perfectly

[billeee]

Sorry to ask a question when the discussion is cold, but do you know where those I2S2 resistors are on the board ? Any points to a diagram or something of that nature that publicly available ?

[doitright]

I doubt that it will be safe at all for you to mess around with those resistors. Its either going to be in the nest on the bottom of the board, or some of the tiny ones on the top of the board right up against the SoC.

Why do you need to know? If you need to access the i2s2 lines, you can access them right against the edge of the Wilink8.

[shanvasu]

Hi doitright,

I am trying to use I2S2 to connect to an external Codec. Looks like you have done some amazing work figuring out BT to loopback I2S2. If you could answer these questions, I will really appreciate it.

1. If I want to connect I2S2 output to an external codec, can I just use the pins from
   WL1837MOD (pins 56,57, 58,60) ?

2. Do I have to setup BT to be able to use the I2S output?

3. If I have to setup BT, what kernel /user space change do I have to use to get I2S2 output to external Codec?

PS: I already routed the 4 I2S pins from WL1836 and connected to a Saleae logic analyzer. And when play the audio through tinyplay (tinyplay xxx.wav –d 0) where device 0 is HDMI Audio through I2S2, I can hear the audio from HDMI. But I don’t see any output in I2S lines in Saleae.

Again your help here would greatly be appreciated.

Thanks!

[doitright]

There is no loopback involved in what I did with it. The i2s2 pins on the SoC are directly connected to the bluetooth, and the bluetooth transmits the audio data to (and receives from) a paired cell phone. The other side of what I did routes that data to a USB audio card.

I don’t think it is at all practical to use i2s2 to connect to any codec besides the HDMI and Bluetooth. If there was a header available, then sure, but there isn’t, and the only option for attaching to those lines would provide a very fragile connection.

1. Yes, those are the correct pin numbers. Datasheet: http://www.ti.com/lit/ds/symlink/wl1807mod.pdf — you will note that the side with those pins is the side facing towards the SoC.

2. Yes, you need to send a special command to the bluetooth chip in order to enable the i2s. See relevant kernel commits;
   https://github.com/HiKey960-Car/android_kernel_linaro_hikey/commit/61631d79cfe9ed762654ac0e0a2d412041b6e7e9
   https://github.com/HiKey960-Car/android_kernel_linaro_hikey/commit/c8a7b77db20d0c70f906fcaca2deba1945072207
   https://github.com/HiKey960-Car/android_kernel_linaro_hikey/commit/2f5ab1a0d51afad148bdceded17419b30025ba6b
   https://github.com/HiKey960-Car/android_kernel_linaro_hikey/commit/7f69be32a82ca5c195bde0e6d83225c99b97f1b3
   https://github.com/HiKey960-Car/android_kernel_linaro_hikey/commit/22d7578139793295f99bcffc8d5987150036e45e

If you aren’t seeing the data on the logic analyzer when playing to the HDMI output, then it means that your logic analyzer isn’t working. Which logic analyzer do you have? If its a genuine saleae, it would have cost at least $100. Possibly as much as $1000. As far as I’ve seen, NONE of the fake saleae’s support 1.8 volt logic, and of the genuine ones, not all of them support 1.8 volt. For example, my genuine saleae logic 4 does NOT support 1.8 volt logic, so to probe basically anything on the hikey960, I have to work through a level shifter.

But you can rest assured that if there is sound coming out the HDMI, then there is definitely data on the i2s pins beside the bluetooth chip.

Just make sure that you keep in mind that like the HDMI, the bluetooth chip is the external codec.

Can you describe what you are trying to accomplish with the i2s2 pins? There may be a better way than trying to hijack those pins.

[Island_Zhao]

Hi, doitright

Your development is very helpful to me. And I apply for your patch. The pcm in and out could display successfully. But my code is Mar 2018. My native HFP and A2DP sink still could not run successfully. Part of log is as follows,
07-05 09:40:18.465 D/BluetoothHeadsetClientServiceJni( 3725): connection_state_cb: state 0 peer_feat 0 chld_feat 0
07-05 09:40:18.467 E/bt_btif ( 3725): btif_hf_client_get_cb_by_bda: could not find block for bdaddr
07-05 09:40:18.467 E/bt_btif ( 3725): btif_hf_client_upstreams_evt: event 3 but not allocating block: cb not found
07-05 09:40:18.467 I/bt_btif_queue( 3725): queue_int_advance: removing connection request: address=bc:a9:20:1c:24:60 UUID=111E busy=true
07-05 09:40:18.467 D/HeadsetClientStateMachine( 3725): Connecting process message: 100
07-05 09:40:18.467 D/HeadsetClientStateMachine( 3725): Connecting: event type: 1
07-05 09:40:18.467 D/HeadsetClientStateMachine( 3725): Connecting: Connection BC:A9:20:1C:24:60 state changed:0
07-05 09:40:18.467 D/HeadsetClientStateMachine( 3725): Exit Connecting: 100
07-05 09:40:18.467 D/HeadsetClientStateMachine( 3725): Enter Disconnected: 100
07-05 09:40:18.468 D/HeadsetClientStateMachine( 3725): Connection state BC:A9:20:1C:24:60: 1->0
07-05 09:40:18.471 I/LocalBluetoothProfileManager( 2550): Failed to connect HEADSET_CLIENT device

07-05 09:39:02.633 E/BluetoothManagerService( 2407): Fail to bind to: Intent { act=android.bluetooth.IBluetoothHeadset }
07-05 09:39:02.633 W/BluetoothManagerService( 2407): Unable to bind with intent: Intent { act=android.bluetooth.IBluetoothHeadset }
07-05 09:39:02.633 V/vold ( 2131): Waiting for FUSE to spin up…
07-05 09:39:02.633 E/BluetoothA2dp( 2407): Could not bind to Bluetooth A2DP Service with Intent { act=android.bluetooth.IBluetoothA2dp }

Could you help give some suggestions to me?
Thank you so much for your great help!

[doitright]

Ok, well first thing is that you would be a lot better off with the APRIL2018 (that’s April 20, 2018), and apply it all against a known good manifest from close to the same date. You can find known good manifests here; https://github.com/96boards/aosp-known-good-manifests .

Also make sure that you are working from my gitLAB repositories; HiKey960-Car · GitLab
I’m not maintaining the ones on github.

I’m not entirely sure what to make out of your log, its a very small excerpt and doesn’t have any context with it. The vold/FUSE error is something I’ve seen before, and tells me that your system is not in a good state.

[Island_Zhao]

Thank you so much for your great help~
Now, I’m sure the music data transfer to hikey960 from my iphone. But there is no sound in the headphone. D I use the usb audio device is Audio Adapter. I disable TI wifi&BT and connect the external Marvell&Qualcomm Wifi&BT by sdwifi and uart bt. Do you have the email? could I communicate with you by email? If I succeed, I will provide the solution in this topic web.
Thank you so much.
log as follows,
07-06 09:41:56.399 E/bt_btif ( 3715): bta_av_proc_stream_evt: p_data is null
07-06 09:41:56.399 I/bt_bta_av( 3715): bta_av_link_role_ok: hndl:x41 role:1 conn_audio:x1 bits:1 features:x825b
07-06 09:41:56.399 W/bt_btif ( 3715): bta_dm_rm_cback:2, status:0
07-06 09:41:56.399 W/bt_btif ( 3715): bta_dm_rm_cback:2, status:7
07-06 09:41:56.399 I/bt_stack( 3715): [INFO:btm_pm.cc(195)] BTM_SetPowerMode already in requested mode 0, interval 0 max 0 min 0
07-06 09:41:56.399 I/bt_stack( 3715): [INFO:btm_pm.cc(195)] BTM_SetPowerMode already in requested mode 0, interval 0 max 0 min 0
07-06 09:41:56.400 D/android.hardware.bluetooth@1.0-impl( 3682): Send, Debug
07-06 09:41:56.400 D/android.hardware.bluetooth-hci-h4( 3682): Send, Debug
07-06 09:41:56.400 D/android.hardware.bluetooth-hci-h4( 3682): H4Protocol::Send: WriteSafely uart_fd = 7
07-06 09:41:56.400 D/android.hardware.bluetooth-hci-hci_protocol( 3682): WriteSafely, Debug, length = 11
07-06 09:41:56.400 D/android.hardware.bluetooth-hci-hci_protocol( 3682): writeSafely fd = 7
07-06 09:41:56.400 I/btif_av ( 3715): virtual bool BtifAvStateMachine::StateOpened::ProcessEvent(uint32_t, void *): Peer bc:a9:20:1c:24:60 : event=BTA_AV_START_EVT(0x4) status=0 suspending=0 initiator=0 flags=0x2(REMOTE_SUSPEND)
07-06 09:41:56.400 I/bt_btif_a2dp_sink( 3715): btif_a2dp_sink_set_rx_flush: enable=false
07-06 09:41:56.400 I/btif_av ( 3715): btif_report_audio_state: peer_address=bc:a9:20:1c:24:60 state=2
07-06 09:41:56.400 I/BluetoothA2dpSinkServiceJni( 3715): bta2dp_audio_state_callback
07-06 09:41:56.401 D/android.hardware.bluetooth@1.0-impl( 3682): sendHciCommand, Debug
07-06 09:41:56.401 D/android.hardware.bluetooth@1.0-impl( 3682): Send, Debug
07-06 09:41:56.401 D/android.hardware.bluetooth-hci-h4( 3682): Send, Debug
07-06 09:41:56.401 D/android.hardware.bluetooth-hci-h4( 3682): H4Protocol::Send: WriteSafely uart_fd = 7
07-06 09:41:56.401 D/android.hardware.bluetooth-hci-hci_protocol( 3682): WriteSafely, Debug, length = 8
07-06 09:41:56.401 D/android.hardware.bluetooth-hci-hci_protocol( 3682): writeSafely fd = 7
07-06 09:41:56.401 D/A2dpSinkStateMachine( 3715): Connected process message: 101
07-06 09:41:56.401 D/A2dpSinkStateMachine( 3715): processAudioStateEvent in state 2
07-06 09:41:56.402 D/A2dpSinkStreamHandler( 3715): process message: 0
07-06 09:41:56.402 D/A2dpSinkStreamHandler( 3715): audioFocus = 0
07-06 09:41:56.402 D/A2dpSinkStreamHandler( 3715): sendAvrcpPause
07-06 09:41:56.403 D/A2dpSinkStreamHandler( 3715): Pausing AVRCP.
07-06 09:41:56.403 V/AvrcpControllerService( 3715): sendPassThroughCmd keyCode: 70 keyState: 0
07-06 09:41:56.404 V/AvrcpControllerService( 3715): sendPassThroughCmd keyCode: 70 keyState: 1
07-06 09:41:56.404 D/AvrcpControllerSM( 3715): HandleMessage: REQ_PASS_THROUGH_CMD
07-06 09:41:56.404 I/BluetoothAvrcpControllerJni( 3715): sendPassThroughCommandNative: sBluetoothAvrcpInterface: 0x74005a4c90
07-06 09:41:56.404 I/BluetoothAvrcpControllerJni( 3715): key_code: 70, key_state: 0
07-06 09:41:56.404 E/bt_btif ( 3715): send_passthrough_cmd: calling btif_rc_get_device_by_bda
07-06 09:41:56.405 D/AvrcpControllerSM( 3715): inform AVRCP Commands to A2DP Sink
07-06 09:41:56.405 D/android.hardware.bluetooth@1.0-impl( 3682): Send, Debug
07-06 09:41:56.405 D/android.hardware.bluetooth-hci-h4( 3682): Send, Debug

[doitright]

I prefer to keep discussions like this on the forum so that other people will be able to read it and learn from it.

Here is the thing; I did nothing at all with respect to A2DP/AVRCP. The guys at hisilicon did all that, and it was working before I even started with this board. So really, I can’t answer much about that except to say that IN THEORY, you should be able to make those work over a different BT adapter, since they run their data over HCI, which is carried over UART. But be aware that your UART would have to be running at a high enough rate to carry the audio data that you are sending over the link.

There is one other thing that I want to make sure you are aware of with regards to A2DP: It will not output sound unless you hit the PLAY button in the bluetooth music application on the hikey960.

Now regarding HFP… you see, it would have been very helpful if you had mentioned previously that you were using a different BT adapter. While technically, HFP audio can be run over HCI protocol in much the same way as A2DP, Android does not support this. While there is some code in AOSP regarding HFP over HCI, this code was never completed, has never been tested, and has never been maintained. That means that HFP audio must originate in some manner that is accessible to the AUDIO HAL, which typically means an ALSA device. If your bluetooth adapter has I2S capability, then you could attach it to the I2S pins of the LS adapter and configure simple-card to work with it. IF this is possible for your hardware, and you want to try it out, then be aware that my april kernel branch does NOT have the patch needed to enable i2s0 on the LS adapter. You can find the branch that ends with “CAR96” and cherry-pick these two commits from it;

c3080aaada9f8a97825bb050eb69788376998c7d
077e0189be3a6f1bd24aceb6c18872a066ed8be0
(* These first of those two commits will have conflicts that you will need to resolve)

In addition, understanding this commit will help you in writing a codec driver for your bt card;
71e95cfc04322263133721f7a3e7d623c92c71fe
Note that everything in the bt_sco_hw_params function is specific to the on-board wilink8 bt.

Do you mind if I ask you WHY you are using a separate BT adapter? In the very least, I would VERY STRONGLY recommend that you get it working with the built-in bluetooth card first, and then switch to yours.

[Island_Zhao]

Hi, doitright

Thank you very much for your great help! As I have tried your A2DP sink solution with build TI, it worked well, now my A2DP could work with Marvell and Qualcomm sd-Wifi & uart-BT card, not build in TI card. Sorry for late response as I won’t disturb you in your nice weekend. For A2DP, I comply with your advice: There is one other thing that I want to make sure you are aware of with regards to A2DP: It will not output sound unless you hit the PLAY button in the bluetooth music application on the hikey960. And it really worked. I could hear the music from the hikey960 and the music is from my iPhone. Thank you so much again.
For HFP, I need to debug this in details according to your very helpful advice.
For your last question, I’m an Android developer and interested in Wireless card. As Google prepared all the TI wifi bt card code that vendor could modified such as /device/linaro/hikey compile code, wifi&bt hal code, sepolicy code, and even TI wifi&bt driver code, kernel config file. I want to study more knowledge for wifi&bt from kernel, Android HAL, native, framework to wifi&bt APP. So I purchased Marvell&Qualcomm wifi&bt card to develop for all knowledge and then we could compare TI, Marvell and Qualcomm card index such as throughput and so on . I want to learn of all bring up process. Thank you so much for your precious advice. I will continue to debug the HFP function by follow your advice.

[doitright]

@Island_Zhao

For your last question, I’m an Android developer and interested in Wireless card. As Google prepared all the TI wifi bt card code that vendor could modified such as /device/linaro/hikey compile code, wifi&bt hal code, sepolicy code, and even TI wifi&bt driver code, kernel config file. I want to study more knowledge for wifi&bt from kernel, Android HAL, native, framework to wifi&bt APP. So I purchased Marvell&Qualcomm wifi&bt card to develop for all knowledge and then we could compare TI, Marvell and Qualcomm card index such as throughput and so on . I want to learn of all bring up process. Thank you so much for your precious advice. I will continue to debug the HFP function by follow your advice.

That is an excellent reason. I’m glad that I could help you with it.

[dey_arnab]

Hi Mr. doitright,

You thread is really very helpful for those who are working on SCO on Hikey960.
I am also trying to do the same but facing some issues.

I have modified the kernel code similarly and ‘tinypcminfo’ produces following -

hikey960:/ # tinypcminfo -D 0 -d 0
Info for card 0, device 0:

PCM out:
Access: 0x000009
Format[0]: 0x000004
Format[1]: 00000000
Format Name: S16_LE
Subformat: 0x000001
Rate: min=8000Hz max=48000Hz
Channels: min=1 max=2
Sample bits: min=16 max=16
Period size: min=1024 max=2048
Period count: min=4 max=32

PCM in:
Access: 0x000009
Format[0]: 0x000004
Format[1]: 00000000
Format Name: S16_LE
Subformat: 0x000001
Rate: min=8000Hz max=48000Hz
Channels: min=1 max=2
Sample bits: min=16 max=16
Period size: min=1024 max=2048
Period count: min=4 max=32

However, I can see, after sending the required vendor commands - Write_Codec_Config, Write_Codec_Config_Enhanced and enabling pcm loopback, I2S2_DOUT is always zero i.e. no sound from loopback(probing BT pin57 using ‘Saleae USB DX’) though DIN seems to be proper. Mr. doitright, Mr. Loic, can you please help me to debug?

My VSCs are -

Write_Codec_Config -
06 fd 22 00 0c 01 80 bb 00 00 00 00 00 00 00 10 00 01 00 01 10 00 01 00 00 00 10 00 11 00 01 10 00 11 00 00 00

Write_Codec_Config_Enhanced -
07 fd 18 00 00 00 00 00 00 04 04 01 00 00 00 00 00 00 04 04 01 00 00 00 00 00 00

PCM loopback enable -
28 fe 01 01

I am sending the VSCs inside Bta_hf_client_main.bta_hf_client_api_enable() function so that during hfp cleint profile init, pcm will be initialized. bt snoop log is proper as I can see controller is sending command status = 0x00. Do you see any problem in sending VSCs here?

Thanking you,
Arnab Dey

[doitright]

This topic was solved quite a long time ago. I’ve been actually using it in my car for a few months now, and am finding it to be the absolute most reliable HFP device I’ve ever used. Over months now of testing, every single call has been successful.

On the kernel side, these three commits are all you need;

HiKey960: Enable 8 and 16 kHz sample rates on hisi-i2s (077e0189) · Commits ·...

ASoC add codec driver for TI Wilink8 (71e95cfc) · Commits · HiKey960-Car /...

HiKey960: Build ASoC codec driver for TI Wilink 8 (0dd668e0) · Commits ·...

Note that the VSCs are sent from the ALSA driver in the second of those 3 commits.

[dey_arnab]

Hi Mr. doitright,

Thank you very much for the consolidated answer. It worked like a charm!!! I am sending the VSC from userspace. Thank you.

Regards,
Arnab Dey

[Island_Zhao]

Hi, doitright

Could you help share your HFP connection picture?
Such as the Mic, headset or speaker, and the Hikey960. HFP didn’t work in my native environment.

Thank you so much for your great help!

[doitright]

I’m not sure what you mean by “picture”. Could you explain what you’ve done? And what steps you’ve taken to debug it?

EDIT: I looked back on your previous posts, and now I remember that you were the one who was working with an external bluetooth device. So I presume that your biggest problem is that you don’t have any audio data lines connected with your bluetooth device, or they aren’t working.

The picture that you are looking for is then this;

Microphone → Vantec NBA-200U (left microphone input)
Speakers → Vantec NBA-200U (front speakers output)
Hikey960 USB → Vantec NBA-200U

Audio input path, in my configuration;
Microphone → Vantec NBA-200U → 960 → I2S2 → Wilink8

Audio output path;
Wilink8 → I2S2 → 960 → Vantec NBA-200U → Speakers

Now if you’re not using the Wilink8, then you have basically two options;

1. If you have managed to attach your bluetooth device to the CPU via I2S0 (on the low speed port), then you have an updated path;
   Microphone → Vantec NBA-200U → 960 → I2S0 → Your bluetooth chip
   Your bluetooth chip → I2S0 → 960 → Vantec NBA-200U → Microphone
2. If you have NOT managed to attach your bluetooth device to the CPU via I2S, then you cannot make HFP work, since Android does not actually support SCO over HCI.

You may want to chat with @dey_arnab, since he’s managed to get HFP working with a bluetooth chip attached via LS header (including i2s), or at least read the thread where he got it working; How to make I2S0(on LF connector) a PCM slave?
**Note however, most of the conversation actually revolved around converting the audio to work with a 24 bit sound card.

[Island_Zhao]

Hi, Mr doitright

Thank you so much for your great help.
The picture I mentioned is what you described. I want to know how you use the mic or speaker and I2S0 at the same time. Thank you for your details info.
Actually I didn’t learn of I2S0 knowledge. And I know I must use I2S0 to pass the voice audio to 960 according to your advice. But i don’t know how to use the I2S or LS header. And I bought the device Vantec NBA-200U you used in these days. I will check another topic thread to check how to use external bt chip to make hfp work. Thank you very much for your details sharing.