If I don’t re-use the existing stream,what solution do I have? Closing the existing steam will kill the system. Any other solution to resolve the problem?
If I mark ‘free(stream)’, there will be one successful phone call that I can hear sound from USB and speak into the mic. If I dial number again, the system will crash with an unknown reason.
There is no reason why you can’t close the input stream. You just need to be a lot more careful about how you manipulate it.
I’ve already explained that what you need to do is WHEN you close the stream, you need to manipulate the behavior or any and all functions that this will impact such that they behave in exactly the way they are expected to behave.
So for a more detailed explanation;
Add a new field to your stream_in and stream_out structures. This field will store a boolean value indicating that the stream is closed so that the hardware can be used by HFP.
When you are about to open your own streams, first mark the streams using that new field, then use pcm_close on the pcm’s associated with those streams (don’t close the stream, close the pcm).
You will also need to find EVERYWHERE in the HAL that actually uses pcm’s, and where it would access the pcm, instead just return the expected value but don’t actually let it try touching any pcm.
Once you close the SCO, you need to go through all of the streams that are still present, and RE-OPEN their PCMs – the alsa_device_proxy structure has a pcm_config as its second field.
Finally, if there is anything at all that attempts to OPEN a new stream while you have the SCO running, you need to mark that new stream, and skip the actual pcm_open.
I was able to reproduce failure to open stream issue. The reason why it won’t work on subsequent attempts after it fails to open something once, is because there remain the pcm’s that were opened prior to the one that failed to open. So for instance, if it is going to fail, it will generally fail on one of the “near” streams, which means that the far streams remain open, and therefore can’t be opened a second time. I’ve added a fix for that to my device tree.
This should safely handle the multiple-streams issue;
Thank you.
It works well now.
I have HFP working on boot now.
From AOSP Master, needs these changes;
*** Despite the code being in the hal, audio processing (echo cancellation) is not yet working. I have some pointers I need to straighten out.
I now have WebRTC audio processing working.
It is very thoroughly cancelling out echo.
I tested it by putting the microphone right against the speaker on the hikey960 and taking my tablet (using for testing with hangouts dialer) and speaking into it – zero echo 
Previously, it was creating a very strong echo with the same procedure.
Looks like the alsa driver method of setting up the PCM is a bit unreliable.
The outbound audio seems to work reliably, but incoming is about 50/50 going to be just noise. I think its because the PCM is being set up too late in the process, probably after the SCO stream is already running, or maybe on the verge of running.
I think I know how it is breaking the data. Not why, just how. Opening data captures in a hex editor and comparing good data with broken, it appears like the bad captures have their endianness reversed.
Tomorrow I’ll write a test program to switch endianness and see if that clears up the broken sample. If it does, then in the worst case, I should be able to detect it mathematically and repair it on the fly in the HAL.
And that is confirmed!
It randomly switches endianness. I ask for PCM_FORMAT_S16_LE, and 50% of the time it gives me PCM_FORMAT_S16_BE.
I verified it experimentally by repairing a broken audio capture using this, which just reverses the bytes in each sample;
#include <stdio.h>
int main (int argc, char ** argv){
FILE *readf;
FILE *writef;
unsigned char buffer[20];
int i;
readf = fopen(argv[1], "rb");
writef = fopen(argv[2], "wb");
// first copy 200 bytes header-ish from input to output;
for (i=0; i<10; i++){
fread(buffer, sizeof(buffer), 1, readf);
fwrite(buffer, sizeof(buffer), 1, writef);
}
while (fread(buffer, 2, 1, readf) != 0){
fwrite(&buffer[1], 1, 1, writef);
fwrite(&buffer[0], 1, 1, writef);
}
fclose(readf);
fclose(writef);
return 0;
}And the solution;
When the endianness is reversed, the average difference between samples grows by about a factor of 10 (experimentally). In general, good audio will have each subsequent sample being a relatively small difference from the one before it, resulting in a smooth wave. So for the first few seconds of the stream, I calculate the sum of the absolute differences between each subsequent audio frame for both “as received” and reverse byte order, and whichever one is the smallest is the mode I take as valid. I’ve tested it roughly 50 times, and it in every test, the resulting audio has been correct. Previously, it was about 50/50.
I finally got around to enabling and testing 16 kHz audio.
Funny thing is that I committed the code for it to my git 20 days ago, but didn’t enable it until today. Its working perfectly
Sorry to ask a question when the discussion is cold, but do you know where those I2S2 resistors are on the board ? Any points to a diagram or something of that nature that publicly available ?
I doubt that it will be safe at all for you to mess around with those resistors. Its either going to be in the nest on the bottom of the board, or some of the tiny ones on the top of the board right up against the SoC.
Why do you need to know? If you need to access the i2s2 lines, you can access them right against the edge of the Wilink8.
Hi doitright,
I am trying to use I2S2 to connect to an external Codec. Looks like you have done some amazing work figuring out BT to loopback I2S2. If you could answer these questions, I will really appreciate it.
If I want to connect I2S2 output to an external codec, can I just use the pins from
WL1837MOD (pins 56,57, 58,60) ?
Do I have to setup BT to be able to use the I2S output?
If I have to setup BT, what kernel /user space change do I have to use to get I2S2 output to external Codec?
PS: I already routed the 4 I2S pins from WL1836 and connected to a Saleae logic analyzer. And when play the audio through tinyplay (tinyplay xxx.wav –d 0) where device 0 is HDMI Audio through I2S2, I can hear the audio from HDMI. But I don’t see any output in I2S lines in Saleae.
Again your help here would greatly be appreciated.
Thanks!
There is no loopback involved in what I did with it. The i2s2 pins on the SoC are directly connected to the bluetooth, and the bluetooth transmits the audio data to (and receives from) a paired cell phone. The other side of what I did routes that data to a USB audio card.
I don’t think it is at all practical to use i2s2 to connect to any codec besides the HDMI and Bluetooth. If there was a header available, then sure, but there isn’t, and the only option for attaching to those lines would provide a very fragile connection.
Yes, those are the correct pin numbers. Datasheet: http://www.ti.com/lit/ds/symlink/wl1807mod.pdf — you will note that the side with those pins is the side facing towards the SoC.
Yes, you need to send a special command to the bluetooth chip in order to enable the i2s. See relevant kernel commits;
https://github.com/HiKey960-Car/android_kernel_linaro_hikey/commit/61631d79cfe9ed762654ac0e0a2d412041b6e7e9
https://github.com/HiKey960-Car/android_kernel_linaro_hikey/commit/c8a7b77db20d0c70f906fcaca2deba1945072207
https://github.com/HiKey960-Car/android_kernel_linaro_hikey/commit/2f5ab1a0d51afad148bdceded17419b30025ba6b
https://github.com/HiKey960-Car/android_kernel_linaro_hikey/commit/7f69be32a82ca5c195bde0e6d83225c99b97f1b3
https://github.com/HiKey960-Car/android_kernel_linaro_hikey/commit/22d7578139793295f99bcffc8d5987150036e45e
If you aren’t seeing the data on the logic analyzer when playing to the HDMI output, then it means that your logic analyzer isn’t working. Which logic analyzer do you have? If its a genuine saleae, it would have cost at least $100. Possibly as much as $1000. As far as I’ve seen, NONE of the fake saleae’s support 1.8 volt logic, and of the genuine ones, not all of them support 1.8 volt. For example, my genuine saleae logic 4 does NOT support 1.8 volt logic, so to probe basically anything on the hikey960, I have to work through a level shifter.
But you can rest assured that if there is sound coming out the HDMI, then there is definitely data on the i2s pins beside the bluetooth chip.
Just make sure that you keep in mind that like the HDMI, the bluetooth chip is the external codec.
Can you describe what you are trying to accomplish with the i2s2 pins? There may be a better way than trying to hijack those pins.
Hi, doitright
Your development is very helpful to me. And I apply for your patch. The pcm in and out could display successfully. But my code is Mar 2018. My native HFP and A2DP sink still could not run successfully. Part of log is as follows,
07-05 09:40:18.465 D/BluetoothHeadsetClientServiceJni( 3725): connection_state_cb: state 0 peer_feat 0 chld_feat 0
07-05 09:40:18.467 E/bt_btif ( 3725): btif_hf_client_get_cb_by_bda: could not find block for bdaddr
07-05 09:40:18.467 E/bt_btif ( 3725): btif_hf_client_upstreams_evt: event 3 but not allocating block: cb not found
07-05 09:40:18.467 I/bt_btif_queue( 3725): queue_int_advance: removing connection request: address=bc:a9:20:1c:24:60 UUID=111E busy=true
07-05 09:40:18.467 D/HeadsetClientStateMachine( 3725): Connecting process message: 100
07-05 09:40:18.467 D/HeadsetClientStateMachine( 3725): Connecting: event type: 1
07-05 09:40:18.467 D/HeadsetClientStateMachine( 3725): Connecting: Connection BC:A9:20:1C:24:60 state changed:0
07-05 09:40:18.467 D/HeadsetClientStateMachine( 3725): Exit Connecting: 100
07-05 09:40:18.467 D/HeadsetClientStateMachine( 3725): Enter Disconnected: 100
07-05 09:40:18.468 D/HeadsetClientStateMachine( 3725): Connection state BC:A9:20:1C:24:60: 1->0
07-05 09:40:18.471 I/LocalBluetoothProfileManager( 2550): Failed to connect HEADSET_CLIENT device
07-05 09:39:02.633 E/BluetoothManagerService( 2407): Fail to bind to: Intent { act=android.bluetooth.IBluetoothHeadset }
07-05 09:39:02.633 W/BluetoothManagerService( 2407): Unable to bind with intent: Intent { act=android.bluetooth.IBluetoothHeadset }
07-05 09:39:02.633 V/vold ( 2131): Waiting for FUSE to spin up…
07-05 09:39:02.633 E/BluetoothA2dp( 2407): Could not bind to Bluetooth A2DP Service with Intent { act=android.bluetooth.IBluetoothA2dp }
Could you help give some suggestions to me?
Thank you so much for your great help!
Ok, well first thing is that you would be a lot better off with the APRIL2018 (that’s April 20, 2018), and apply it all against a known good manifest from close to the same date. You can find known good manifests here; https://github.com/96boards/aosp-known-good-manifests .
Also make sure that you are working from my gitLAB repositories; HiKey960-Car · GitLab
I’m not maintaining the ones on github.
I’m not entirely sure what to make out of your log, its a very small excerpt and doesn’t have any context with it. The vold/FUSE error is something I’ve seen before, and tells me that your system is not in a good state.
Thank you so much for your great help~
Now, I’m sure the music data transfer to hikey960 from my iphone. But there is no sound in the headphone. D I use the usb audio device is Audio Adapter. I disable TI wifi&BT and connect the external Marvell&Qualcomm Wifi&BT by sdwifi and uart bt. Do you have the email? could I communicate with you by email? If I succeed, I will provide the solution in this topic web.
Thank you so much.
log as follows,
07-06 09:41:56.399 E/bt_btif ( 3715): bta_av_proc_stream_evt: p_data is null
07-06 09:41:56.399 I/bt_bta_av( 3715): bta_av_link_role_ok: hndl:x41 role:1 conn_audio:x1 bits:1 features:x825b
07-06 09:41:56.399 W/bt_btif ( 3715): bta_dm_rm_cback:2, status:0
07-06 09:41:56.399 W/bt_btif ( 3715): bta_dm_rm_cback:2, status:7
07-06 09:41:56.399 I/bt_stack( 3715): [INFO:btm_pm.cc(195)] BTM_SetPowerMode already in requested mode 0, interval 0 max 0 min 0
07-06 09:41:56.399 I/bt_stack( 3715): [INFO:btm_pm.cc(195)] BTM_SetPowerMode already in requested mode 0, interval 0 max 0 min 0
07-06 09:41:56.400 D/android.hardware.bluetooth@1.0-impl( 3682): Send, Debug
07-06 09:41:56.400 D/android.hardware.bluetooth-hci-h4( 3682): Send, Debug
07-06 09:41:56.400 D/android.hardware.bluetooth-hci-h4( 3682): H4Protocol::Send: WriteSafely uart_fd = 7
07-06 09:41:56.400 D/android.hardware.bluetooth-hci-hci_protocol( 3682): WriteSafely, Debug, length = 11
07-06 09:41:56.400 D/android.hardware.bluetooth-hci-hci_protocol( 3682): writeSafely fd = 7
07-06 09:41:56.400 I/btif_av ( 3715): virtual bool BtifAvStateMachine::StateOpened::ProcessEvent(uint32_t, void *): Peer bc:a9:20:1c:24:60 : event=BTA_AV_START_EVT(0x4) status=0 suspending=0 initiator=0 flags=0x2(REMOTE_SUSPEND)
07-06 09:41:56.400 I/bt_btif_a2dp_sink( 3715): btif_a2dp_sink_set_rx_flush: enable=false
07-06 09:41:56.400 I/btif_av ( 3715): btif_report_audio_state: peer_address=bc:a9:20:1c:24:60 state=2
07-06 09:41:56.400 I/BluetoothA2dpSinkServiceJni( 3715): bta2dp_audio_state_callback
07-06 09:41:56.401 D/android.hardware.bluetooth@1.0-impl( 3682): sendHciCommand, Debug
07-06 09:41:56.401 D/android.hardware.bluetooth@1.0-impl( 3682): Send, Debug
07-06 09:41:56.401 D/android.hardware.bluetooth-hci-h4( 3682): Send, Debug
07-06 09:41:56.401 D/android.hardware.bluetooth-hci-h4( 3682): H4Protocol::Send: WriteSafely uart_fd = 7
07-06 09:41:56.401 D/android.hardware.bluetooth-hci-hci_protocol( 3682): WriteSafely, Debug, length = 8
07-06 09:41:56.401 D/android.hardware.bluetooth-hci-hci_protocol( 3682): writeSafely fd = 7
07-06 09:41:56.401 D/A2dpSinkStateMachine( 3715): Connected process message: 101
07-06 09:41:56.401 D/A2dpSinkStateMachine( 3715): processAudioStateEvent in state 2
07-06 09:41:56.402 D/A2dpSinkStreamHandler( 3715): process message: 0
07-06 09:41:56.402 D/A2dpSinkStreamHandler( 3715): audioFocus = 0
07-06 09:41:56.402 D/A2dpSinkStreamHandler( 3715): sendAvrcpPause
07-06 09:41:56.403 D/A2dpSinkStreamHandler( 3715): Pausing AVRCP.
07-06 09:41:56.403 V/AvrcpControllerService( 3715): sendPassThroughCmd keyCode: 70 keyState: 0
07-06 09:41:56.404 V/AvrcpControllerService( 3715): sendPassThroughCmd keyCode: 70 keyState: 1
07-06 09:41:56.404 D/AvrcpControllerSM( 3715): HandleMessage: REQ_PASS_THROUGH_CMD
07-06 09:41:56.404 I/BluetoothAvrcpControllerJni( 3715): sendPassThroughCommandNative: sBluetoothAvrcpInterface: 0x74005a4c90
07-06 09:41:56.404 I/BluetoothAvrcpControllerJni( 3715): key_code: 70, key_state: 0
07-06 09:41:56.404 E/bt_btif ( 3715): send_passthrough_cmd: calling btif_rc_get_device_by_bda
07-06 09:41:56.405 D/AvrcpControllerSM( 3715): inform AVRCP Commands to A2DP Sink
07-06 09:41:56.405 D/android.hardware.bluetooth@1.0-impl( 3682): Send, Debug
07-06 09:41:56.405 D/android.hardware.bluetooth-hci-h4( 3682): Send, Debug
