Tales of open80211s in the wild

One thing I like about working at cozybit is learning about clients’ novel uses of mesh technology. Today I’m going to tell you about one client’s use case and how we found a bug in the 802.11s mesh implementation while diagnosing their network performance.

The client has open80211s deployments of about a dozen nodes, including mobile video camera streaming 200 kbps unicast data to a viewing station. Simultaneously, the mesh supports push-to-talk broadcast voice data from any station. The camera is usually moving, so the topology is changing often. Since the data is high throughput, quickly achieving that topology is super important. They reported issues with data delivery and some strange behavior with path selection.

802.11 HWMP routing dictates the procedure for handling how paths from one node to another are determined and updated. In particular, there is a target sequence number to determine path freshness. A lower number indicates a staler path. During path selection, stale paths lose immediately and ties in path freshness are broken by best airtime link metric.

To resolve the observed problem with path selection, we dug into the packet captures they provided to analyze their traffic and gather stats on their mesh’s performance. Before I show you excerpts from packet captures, here is a diagram which illustrating the mesh topology at the time of the capture. Direct links between stations are represented by lines:


Here’s the excerpt from the packet capture which exposes the relevant parts of the packets. Here we have station 0, the target of a path request from station 3, sending path replies in turn to station 1, then station 5, and then station 9. The first sequence number, 10359, is the originating station target sequence number. The second, 155 in the reply to station 1, is the target station number.

sta−0: prep−>sta−1, orig=sta−3 (sn 10359), tgt=sta−0 (sn 155)
sta−0: prep−>sta−5, orig=sta−3 (sn 10359), tgt=sta−0 (sn 154)
sta−0: prep−>sta−9, orig=sta−3 (sn 10359), tgt=sta−0 (sn 154)

Even though station 0 sent replies to stations 5 and 9 after station 1, it used a lower target sequence number! The latter two path replies will never be considered, even if they are a better data path. Since station 0 is using 155 as the target sequence number in its reply to station 1, and the subsequent path replies are at least as fresh, it should use at least 155 in the replies to stations 5 and 9.

Here’s the fix (Thanks, Bob!), which ensures that when the target station is sending replies to the same originator, the sequence number never decrements:

Signed-off-by: Bob Copeland <bob@cozybit.com>
net/mac80211/mesh_hwmp.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/net/mac80211/mesh_hwmp.c b/net/mac80211/mesh_hwmp.c
index 03ff5ea..94758b9 100644
--- a/net/mac80211/mesh_hwmp.c
+++ b/net/mac80211/mesh_hwmp.c
@@ -544,9 +544,10 @@ static void hwmp_preq_frame_process(struct ieee80211_sub_if_data *sdata,
if (time_after(jiffies, ifmsh->last_sn_update +
net_traversal_jiffies(sdata)) ||
time_before(jiffies, ifmsh->last_sn_update)) {
- target_sn = ++ifmsh->sn;
+ ++ifmsh->sn;
ifmsh->last_sn_update = jiffies;
+ target_sn = ifmsh->sn;
} else if (is_broadcast_ether_addr(target_addr) &&
(target_flags & IEEE80211_PREQ_TO_FLAG)) {


Another issue that we had to address was tuning path refresh time. The shorter the path refresh time, the quicker the paths can adapt, but you lose more network overhead to path request/response management frames. So, we worked with our client to tune the path refresh time based on real data on their mesh and achieve an acceptable balance, but we’ll save that discussion for a future post.

How anonymous is Firechat?

Someone recently was asking how anonymous is Firechat, an iOS app that allows chatting using ad-hoc wifi connections, and that is marketed as a hyperlocal anonymous chat network.

Firechat uses Apple’s Multipeer Connectivity Framework, which in turn relies on a custom proprietary ad-hoc protocol developed by Apple. Our friends at Open Garden implement forwarding at the application layer in an attempt to overcome the limitations in the ad-hoc protocol.

A simple traffic capture of a chat session quickly revealed that messages are encrypted, but anonymity requires more than that. The metadata about the chat session is not encrypted and observable in the traffic capture. And, as our NSA friends know, metadata contains valuable information, and some claim it is even more intrusive than content itself… In this case our traffic capture reveals the real names of the session participants, the application they used to communicate as well as the duration and number of messages exchanged during the session.

As an example, see the conversation that took place between a mysterious Mr. Secret and someone else in my office.


Just scanning for Action frames from Apple (identified by their public identifier 00:17:f2) reveals that Mr. Spy is actually yours truly. I will not disclose who my chat partner was to avoid embarrassing him, but that information is also available.


Having said that, the application is really fun to use. Just be mindful about what you say to whom.

Update: By popular demand I’ve uploaded the traffic capture file here.

Mesh peering vs. WiFi Direct group formation

We’ve been asked numerous times why 11s peering is so fast compared with WiFi Direct. WiFi Direct peering (or group formation) entails the following steps:

1. Group Owner Negotiation
2. Beaconing
3. Authentication (Open System)
4. Association (Open System)
5. 802.1X Wireless Protected Setup Authentication
6. Failure (yes, this is part of the specification!)
7. Deauthentication
8. Authentication (RSN)
9. Association (RSN)
10. 802.1X RSN Authentication

This is for the general case, called Standard Group Formation. The specification defines different behaviors for two other scenarios: when the Group Owner is pre-configured (Autonomous Group Formation) and when the network credentials are pre-stored (Persistent Group Formation). These scenarios can omit some of the phases enumerated above. For more details, check out this great paper about WFD.

You can see a wireshark capture of these exchanges in the figure below. These capture is generated from wpa_supplicant‘s own test suite on simulated hardware and an ideal medium. This means that there are no frame losses, and even then, it takes 34 frames to establish a link between two nodes. On a lossy medium, a lost frame may require restarting the authentication process and this is why WFD peering may take a long time.


Mesh, on the other hand, does not assign roles to nodes and therefore has no concept of Group Owner, nor the need for a Group Owner negotiation. Security is also much simpler, as there is no authenticator/supplicant assimetry.

An open mesh peering takes only 4 frames without security and 4 more to establish a secure peer link. The entire peer link takes exactly 8 frames, as shown below.

Screenshot from 2014-03-21 18:57:41

No one summarized it better than security expert and SAE designer Dan Harkins:

It’s 34 frames of compromised and questionable security versus 8 of strong security. And both options can use the identical credential for authentication, it’s just that mesh does it right and WFD does it wrong.

Think about that next time you are waiting on your Android phone to connect with WiFi Direct…

How to crack Bluetooth LE security using crackle

I just watched this great talk about Bluetooth LE’s flawed security and thought I’d give crackle a try.

What you need:

1. A Bluetooth LE peripheral that uses security.  I used Broadcom’s WICED Smart and built the sample health_thermometer_plus application that uses encryption by default.

2. An Ubertooth, the totally awesome Bluetooth sniffer.

3. A BTLE capable host. I used an iPhone 5.

4. (Optional) Wireshark with the BTLE plugin.

BTLE cracking ingredients

Steps to crack:

1. Build WICED sample app. In order to do this on Linux you’ll need a bit of Wine magic:

 diff -Naur wiced_toolchain_common.orig wiced_toolchain_common.mk
--- wiced_toolchain_common.orig    2014-02-11 14:18:09.128750842 -0800
+++ wiced_toolchain_common.mk    2014-02-11 14:14:21.160744316 -0800
@@ -101,6 +101,10 @@
 export SHELL       = $(COMMON_TOOLS_PATH)dash
 OPENOCD_FULL_NAME := "$(OPENOCD_PATH)Linux64/openocd-all-brcm-libftdi"
+CGS_FULL_NAME     := wine $(CGS_PATH)Win32/cgs.exe
+CHIPLOAD_FULL_NAME    := wine $(CHIPLOAD_PATH)Win32/chipload.exe
+HEX_TO_BIN_FULL_NAME  := wine $(HEX_TO_BIN_PATH)/Win32/ihex2bin.exe
 PRINT_SLASH       :=\\\\
 SLASH_QUOTE       :=\\\"
 ESC_QUOTE         :=\"

and tell wine where to find COM1 serial port:

ln -s /dev/ttyUSB0 ~/.wine/dosdevices/com1
echo com1 > /path/to/WICED-Smart-SDK/com_port.txt

Then build and download app to the target:

 ./make ROM.health_thermometer_plus-BCM920732TAG_Q32 UART=com1 download VERBOSE=0
Linking target ELF
OK, made elf.
Writing Hex image
Call to health_thermometer_plus_spar_crt_setup @ 00209551
Total RAM footprint                    15416 bytes (15.1kiB)

Converting CGS to HEX...
Conversion complete

Creating OTA images...
Conversion complete

Downloading application...
Download complete

Application running

2. Start capturing traffic with the ubertooth.

ubertooth-util -r ; ubertooth-btle -f -c /tmp/btle.cap

3. Connect with your iPhone. We really like LightBlue from Punch Through Design. You will be asked to pair. Do it.


4. Then, using the app, read some characteristics to get some data traffic flowing.

5. (Optional) Inspect your capture file with Wireshark. Confirm that you have an LL_START_ENC_REQ control PDU. And that after that point, all the traffic is encrypted (wireshark will report that as malformed L2CAP data packets.


6. Now run cryptle on the capture file.

crackle -i /tmp/btle.cap -o clear.cap
Warning: found multiple pairing requests, only using the latest one

TK found: 000000
ding ding ding, using a TK of 0! Just Cracks(tm)

7. You can now open clear.cap with wireshark and observe that the earlier malformed L2CAP packets show up as clear GATT frames. 


Rock on!

Enabling the Serial Console on an RK3188 (Tronsmart MK908 v5.0)

Rockchip devices are a great platform for development: small, cheap and very hackable.  In particular, here at cozybit, we bought some Tronsmart MK908 (RK3188) to do some Android hacking. It wasn’t long before we realized serial console was a feature we wanted, so basing on omegamoon’s work, this is what we did to gain access:

  • Disassemble the plastic case to extract the board. Don’t worry, it’s simple to do without breaking anything.
  • Take a look at the MK908 board and identify its version number. This recipe only works for v5.0. Other versions have different layouts (click here for v1.0 boards).
  • Get a FTDI Basic Breakout for 3.3V (or similar) and a 5 pin breakaway male header.
  • Solder the breakaway male header to the board matching the FTDI Basic Breakout pins with the next soldering points:


  • The coppered square on top of the mentioned points is also a GND point. So, to make soldering easier, you can always solder the GND pin to it.


  • Now connect it to your machine, start minicom (or the tool of your choice) and configure it with the next parameters: 115200 – 8N1 – NOR.
  • Finally, secure your mod: make a small hole in the case, glue the breakaway male header and close it.



  • One last note: if you are already using a custom kernel, make sure that all the FIQ options are enabled in your config file and that “ttyFIQ0” is the selected console.

Enjoy your MK908 v5 mod and happy hacking!!

Enabling peripheral mode in Android KitKat (4.4)

If you were hoping to see support for Bluetooth LE peripheral mode in Android Kit Kat we have bad news for you:  it is not there.  The bluedroid stack supports that capability but it is not hooked up to the Android framework, and therefore inaccessible to applications.

Our team could not wait, so we rolled up our sleeves and got to work.   After some digging around we noticed that the function that does what we want is buried deep in the stack:

** Function         GATT_Listen
** Description      This function start or stop LE advertisement and listen for
**                  connection.
** (...) 
BOOLEAN GATT_Listen (tGATT_IF gatt_if, BOOLEAN start, BD_ADDR_PTR bd_addr)

On the app side, we extended the BluetoothGattServer.java to support a new functions, listen().  Connecting the two requires many (but straighforward) changes in the following files:

project external/bluetooth/bluedroid:


project frameworks/base:


project hardware/libhardware:


project packages/apps/Bluetooth:


After compiling and updating the framework, you can easily advertise a service by:

public void startPeripheralGattServer() {
    final BluetoothManager bluetoothManager =
        (BluetoothManager) mContext.getSystemService(Context.BLUETOOTH_SERVICE);

    mGattServer = bluetoothManager.openGattServer(mContext, new BluetoothGattServerCallback() {

        public void onCharacteristicReadRequest(BluetoothDevice device, int requestId,
            int offset, BluetoothGattCharacteristic characteristic) {

            if (mGattServer != null) {
                mGattServer.sendResponse(device, requestId, BluetoothGatt.GATT_SUCCESS, offset, new byte[] { bogusValue++ });

    UUID serviceUUID = UUID.randomUUID();
    UUID characteristicUUID = UUID.randomUUID();
    UUID descriptorUUID = UUID.randomUUID();

    BluetoothGattCharacteristic characteristic = new BluetoothGattCharacteristic(characteristicUUID, BluetoothGattCharacteristic.PROPERTY_READ, BluetoothGattCharacteristic.PERMISSION_READ);
    characteristic.setValue(77, BluetoothGattCharacteristic.FORMAT_UINT8, 0);

    BluetoothGattDescriptor descriptor = new BluetoothGattDescriptor(descriptorUUID,     BluetoothGattDescriptor.PERMISSION_READ);


    BluetoothGattService service = new BluetoothGattService(serviceUUID,     BluetoothGattService.SERVICE_TYPE_PRIMARY);


The video below shows how an Xperia Z phone with our updated Android image and minimal Bluetooth application is seen as a Bluetooth LE peripheral by an iPhone.

And now I was going to give details on how we implemented the glue code, but the Android folks seem to be already working on this as can be seen here. So it’s very likely this will be supported in 4.5.  Yay!