Technical Analysis: Blackbox HD3 Wi-Fi Solution

If you’ve never heard of Black Box’s “InnerWirelessHD3” solution, this blog is dedicated to a technical analysis of it. I will abbreviate the name to just “HD3” in this blog.

I will start by saying that I find no additional benefits beyond what a standard deployment provides, and that HD3 is grossly mis-marketed by Black Box.

Technical Operation

The HD3 “solution” is created by housing multiple of a vendor’s APs (external antenna port models) into an ~18″ square ceiling-mounted enclosure, along with tuned cavity filters (that they call “triplexors” so that it sounds more sophisticated), a nest of RF cables, and various mounting accessories.

The primary advantages of using 5GHz channels, with any solution, are:

• More available channels, which yields many benefits
• Cleaner spectrum (less interferers)
• Less penetration of obstacles

One of the advantages of having more channels is having more channels available for your channel reuse plan. This makes it possible to eliminate CCI (or come close to it) in medium and large environments. The more channels that are in the channel plan, the further apart the network designer can move APs using the same channel, which not only minimizes CCI caused by APs, but also minimizes the CCI caused by clients located between APs that are on the same channel.

HD3 creates small channel subsets (of 4-5 channels) from a portion the large set of available 5GHz channels and calls these subsets “layers”. This forces a trade-off of coverage versus capacity/performance. If you have proper coverage so as to allow roaming, you will have high CCI. If you have low CCI, you will have coverage gaps and poor roaming characteristics.  It’s for this reason that Black Box does not put CCI heat maps or data into their final report.

Colocation of APs is an acceptable tactic when deployed properly and in very specific situations, but should not be employed in a static fashion across an entire facility like HD3 does and not in the way that HD3 implements it.

In order to help you understand what CCI looks like in a good deployment, please reference the graphic below. The areas shown in gray (where the orange arrow points) are areas of overlapping coverage (on the same channel). This can happen both on 2.4GHz and 5GHz, but we typically view them separately. This graphic shows 5GHz using a 24 channel plan. Even with 24 20MHz channels, it’s difficult to completely eliminate CCI in a multi-story building. With 4 channels, CCI will be rampant, and CCI is a major capacity and performance killer.

Roaming with HD3 solution, when designed and deployed by the manufacturer, is dismal, as shown in the screenshots below. The thin blue line should be mostly horizontal near the top of the window with small dips during roams. There should be no point at where there are large coverage gaps because it breaks connectivity (illustrated by the vertical gray bars between the sections of blue line).

   

For those of you familiar with the terms SCA (single channel architecture) and MCA (multiple channel architecture). This is a multi-layer MCA model. That wasn’t a typo — MCA. It’s well-known why one would use layers in an SCA model (e.g. Meru Networks), but with MCA, even Meru wouldn’t dare use a layered model! With SCA, all APs share the same channel(s) and a virtual BSSID, forming a “layer”, but with this MCA deployment methodology, the layers consist of logical groupings of APs within each UNII band.

In the deployment that I just witnessed, which was designed directly by Black Box, the “Low” layer was UNII-1 channels (36-40), the “Mid” layer was UNII-2e channels (100-116), and the “High” was UNII-3 channels (149-161). Not all channels are being used (e.g. 52-64, 120-144, & 165), which significantly decreases capacity and performance. The cavity filters (“triplexors”) exist to isolate inbound spatial streams between the APs within an enclosure and for the outbound spatial streams on the AP side of the connection. There is no isolation on the antenna side of the cavity filters, which results in high ACI for 5GHz channels.

Having worked at two enterprise WiFi manufacturers, I’ve learned that it’s always better for performance to use single band (tuned) antennas for 5GHz and 2.4GHz rather than using dual-band antennas. In the case of HD3, there are only three cables (one from each cavity filter) going to the single antenna in a 3×3 system. This means that the antenna is dual-band and that performance suffers.

I find the following drawbacks to the HD3 solution:

1. Low performance, capacity, & density handling 
   1. Forces a mandatory trade-off between coverage and CCI. You can’t have both due to the small number of channels in each layer’s reuse plan.
   2. High ACI (on both bands) due to zero transmit (outbound) isolation at the antenna. This is why the HD3 solution only has one 2.4GHz radio enabled per enclosure.
      1. Black Box doesn’t realize that their 5GHz radios are also experiencing significant ACI, and I’ve proven this in testing.
   3. Produces erratic and unstable throughput
2. Causes poor device behavior
   1. Poor roaming characteristics because there are too many APs at high signal strength for clients to make a consistently good decision
   2. When trying to reduce CCI, HD3 produces coverage gaps (and vice versa)
3. Extremely expensive
   1. Requires up to 3X the necessary number of APs, Ethernet cables, and switch ports
   2. Requires unnecessary tuned cavity filters, ceiling enclosures, and an external ground plane antenna.
4. Unnecessary Complexity
   1. The 5GHz UNII bands are each used separately, each forming a 4-channel “layer” (channel re-use plan). This injects crazy complexity by requiring three separate channel plans rather than one.

Missing The Mark

The HD3 brochure says that it provides Application Grouping and 802.11 Protocol Optimization.

Application Grouping

They claim that separating applications across “layers” improves performance. It doesn’t. The entire point of a packet-switched network is to share resources so that available capacity isn’t dedicated to specific users/devices and thereby goes unused. QoS (WMM in a WiFi network) allows us to properly prioritize multiple application flows while maintaining the advantages of a packet-switched network.

There is obviously going to be more applications than there are layers (up to 3), so how do they know how to group them? Perhaps it’s by QoS type. If so, they should know that with too many voice connections on an AP, they’ll have excessive collisions. Voice traffic starts to degrade around ≤20% airtime utilization and breaks around 35%. Mixed voice/data traffic starts to break down around 50% airtime utilization, and data starts to break down around 80%. These are due to the mechanics of default WMM contention windows. This is a good reason why we use CAC (call admission control) features in APs/controllers.

As another example, if you have too many video flows, from 1SS (single spatial stream) clients (like iPhones) on an AP, you’ll quickly consume all available airtime for the BSS (basic service set).  Those are just a couple of examples, and a comprehensive list would fill pages. There is a very good reason why the WiFi Alliance created WMM – so that data types could be co-mingled while maintaining their respective priorities. In this way, you can have voice clients (AC_Vo) having high priority (yet taking very little airtime) while simultaneously having several video flows (AC_Vi) and data flows (AC_BE & AC_BK). The best performance happens when there’s a prioritized mix, NOT when you put the same types of Apps on a single “layer” (small set of APs).  A great resource for WiFi network design is Andrew Von Nagy’s WiFi Capacity Planner.

Note that the HD3 layering solution applies ONLY to 5GHz because 2.4GHz only ever has a 3-channel reuse plan.

802.11 Protocol Optimization

Black Box claims that the HD3 solution optimizes 802.11 protocol operations by separating PHYs (physical layer specifications). This is to say that they want to separate 11b from 11g from 11n and to separate 11a from 11n from 11ac in order to improve performance.

To actually be able to do that, you’d have to have 6X the number of necessary radios. Two of every three 2.4GHz radios per enclosure is disabled due to huge ACI. That’s an automatic loss of 1/3 of your purchased radios and a huge chunk of capacity. There is only one 2.4GHz “layer” (meaning re-use plan) both in normal deployments and the HD3 solution.

If the goal was to eliminate protection mechanisms, it’s pretty important to note that since there is only one 2.4GHz “layer”, there’s no way to separate 11b, 11g, and 11n.  For 5GHz, they create three layers, but how many 802.11a/g clients have you seen lurking around in the last year? Nearly none. Dual-band 11n/11ac clients are less than $30 in nearly any bus format. It’s FAR more cost effective to upgrade a couple of legacy clients than to re-architect your network into performance hell.  What they totally missed when designing this “HD3 mess” is that the Airtime Fairness feature in almost every WiFi manufacturer’s equipment accomplishes this exact thing…and it’s already baked into the AP code!  Why add $1k in additional hardware trying to reinvent a software feature. There’s an app for that! 🙂

HD3 is considered “WLAN Vendor Agnostic”, meaning that Black Box can fill its ‘black box’ with any vendors’ APs. The Feature set of the controller and APs determine any protocol behavior, and HD3 has no positive (“optimization”) impact whatsoever. This is pure mis-marketing.

Customer References

I was directly involved when my customer (who was evaluating HD3) called two of Black Box’s references. Neither “reference” had ever heard of Black Box and admitted never having worked with them or having their solution installed.

The Black Box engineer said that they have lots of happy reference installs, but couldn’t produce any reference contact info.

In response, I will say that lots of bad designs/deployments doesn’t suddenly mean that your solution is a good one. Many companies over the years have sold horrible solutions to a large, unsuspecting customer base. Black Box should’ve vetted this solution before deploying it anywhere. I will bet that Black Box cannot produce even one viable customer reference account for this solution.

Engineering

The Black Box engineer explained to me exactly how this solution worked, but could not explain:

* Any advantages over a standard deployment

* How this solution avoids CCI and ACI

Black Box’s documentation does not cover any of these details.

I see HD3 as adding a large additional cost and complexity without providing additional benefit. It’s my opinion that selling such an unvetted solution is negligent.

Support Costs

Because HD3 is non-standard, using “layers”, where each layer gets its own SSIDs, it will cause both internal and external WiFi engineers much more time to troubleshoot client problems.  Most WiFi problems will be infrastructure related, and each attempt to make this solution perform better will be yet another exercise in futility. Time is money. Support OPEX is one of the main reasons that large organizations (e.g. EDU, healthcare, etc.) should buy mainline solutions from name brand vendors.

Performance

Because HD3 forces a trade-off between coverage and CCI, as soon as a network experiences any significant load or client density, it will fall right over. This means that Black Box is making its money by taking advantage of the fact that in some environments there isn’t yet a heavy network load. Since enterprise networks have an intended lifetime of 3-5 years, they are obviously hoping that it will be time to refresh (with some different solution) before there is any real network loading.

Possible (Likely) FCC Violation

Parts 15.203 and 15.204 are clear about the rules on use of antennas on intentional radiators. As best I can tell, The HD3 solution does not have its own FCC ID because it uses various vendors’ APs, not just one specific AP. for that reason, they would have to use the AP manufacturer’s FCC Authorization for the specific AP in use, and from there, make (and have approved) class-1 permissible changes, citing that the antenna they are using is of the same type, has equal or lesser gain, and has a very similar radiation pattern as one of the AP manufacturer’s authorized antennas. Such changes and authorization would be required for every AP that they deploy with.

I was told by an AP manufacturer, who Black Box is currently using as part of their HD3 solution, that they do not recommend or support such a deviation from their deployment best practices and will not be responsible for FCC related legalities involved in the sell and deployment of HD3 with their APs.

Summary

Since no manufacturer (or WiFi engineer worth his/her salt) would recommend such a “hardware hack”, Black Box is, and will continue to be alone in selling such a “solution.” Either Black Box (and their self-admitted ONE “engineer” knows more than the entire WiFi industry (including the FCC, IEEE, all manufacturers, all consultants, and all WiFi instructors) or they are selling some seriously expensive snake oil.

UPDATE 8/20/2015

I’m teaching a class this week for a high-profile, extremely reputable VoWiFi hardware vendor, and multiple Solutions Architects in the class said they’ve had no luck at all in getting their solution to work on the few HD3 networks they’ve seen. That wasn’t a surprise at all to me, as I had the same situation. I was just last week told by my existing customer, that after having Blackbox’s Wi-Fi engineer on-site for over 2 months, they still have been unable to get roaming to work properly, and CCI is still a mess.  Surprise, surprise.