WiFi for Over the Air 13
A few folks have commented (in a nice way) about the WiFi for Over the Air 2013 at Bletchley Park last weekend. It stood up pretty well to 100s of developers with lots of devices each and I’ve been asked a bit about how it was done this year…
Raw Bandwidth
Our biggest challenge was time. We got involved only a few weeks before the event and, ironically, the week after the event Openreach were due to deliver a new 1Gbit/s fibre bearer to site that had been ordered months ago.
The current Internet capacity to Bletchley Park is pretty heavily used Mon-Fri by the on site technology businesses here and prior to the upgrade doesn’t have the 70Mbit/s headroom during the working day that we knew from previous experience would be eaten by OTA. Weekends are fine as the business usage goes down, but that crucial bit between 10am on Friday when OTA started and 6–7pm when the site residents drifted off for the weekend was a bit of a bandwidth gap.
Thankfully, our friends at Host-IT had suggested
a rather appropriate “over the air” temporary microwave link via a local tower block. They knew a company that could do the neat radio stuff so with a week to go before OTA, contracts were signed with Skyline Networks, access to the tower block arranged via the council and on Tuesday morning they arrived on site with cherry pickers to find a suitable location for the dish.
It took a while to find a decent location. Bletchley Park is pretty green and whilst it all looked OK on a map, there was practically nowhere on-site that didn’t have a tree blocking the line of sight. Drilling holes in listed buildings to provide Internet access via thick bits of co-ax is a bit frowned upon so it also needed to be somewhere that we could get easy access through a window or similar to put a small rack inside the building and connect on to the rest of our network. No joy in any of the obvious places, either no line of sight, no easy way to get cabling into the building, or no way to connect from there to the network we were putting together for OTA.
In the end we worked out that if we used the roof of the National Museum of Computing in H-Block, we could get down into their boiler room and from there connect to the Gigabit fibre that runs down into our main comms room. The microwave L2 was running at 150Mbit/s at installation so it all looked great.
We subsequently had a few problems with the actual achievable Internet speeds on the link that had ourselves and Skyline tied up until the early hours of Friday morning, but in the end everything went brilliantly with the bandwidth.
WiFi
The WiFi challenges we knew plenty about from previous OTA events. The Mansion is a hard space to get radio capacity into because of its construction. The Leon’s just didn’t have WiFi in mind when they built extension after extension on to the outside of the buildings. Most of its “internal” walls started life as 60cm or more thick outside walls which means there is very little consistent 2.4Gz or 5GHz propagation horizontally between rooms. This wouldn’t be so bad but there is often excellent vertical propagation to the upper floors which now house lots of technology businesses that rely on their WiFi so we have to be pretty smart about how we manage spectrum, especially in the crowded 2.4GHz band.
Drilling holes in panelling and running permanent cable around decorative finishes in the downstairs rooms is apparently very naughty so we tend to be somewhat restricted to making permanent access point installations in the upstairs areas. Very careful surveying and manual colouring of the spectrum map gives us pretty good coverage downstairs for average conference use but OTA is anything but average.
This year we added four temporarily rigged access points to the Ballroom/Billiard Room area where lots of development was going on (two on the only vaguely available bits of 2.4GHz spectrum and two on 5GHz). Lots of velcro and cable ties meant that it could all be ripped out at the end of the event so that you would never know it had been there.
We also put a wired network into the Marquee for this event — made easier by relocating it a convenient 90M cable throw away from our nearest fibre in the Hut4 catering building. We really wanted to do this for last year, but there wasn’t the budget for it so we resorted on that occasion to directional antenna on nearby buildings which didn’t work at all well for OTA scale load.
This year we pulled out the stops and put key bandwidth consumers like the AV folks and lecterns on hardwired connections and put 7 hardwired access points in the
marquee (1 omni on 2.4GHz in each of the four corners of the tent and 3 on 5GHz in the middle and stage area). We could have used highly directional antenna to stripe the coverage inside the marquee but instead took a bet that the 2.4GHz omnis would be adequate provided enough folks used 5GHz. This also gave us the advantage that a reasonable amount of coverage leaked well outside the tent onto the lawn and camping area.
2.4GHz vs 5GHZ
Most cheap WiFi kit uses 2.4GHz spectrum. This spectrum is easiest to make work on a small scale as it has better propagation through objects so you get a better signal radius. It is also very crowded with only 4 useable 20MHz wide frequency bands and lots of noise and competing uses.
5GHz on the other hand has many more useable channels in most environments. It isn’t as easy to cover large areas from one base as its propagation is more limited, but this is a very good thing if you have lots of bandwidth hungry clients in a small area. You can deploy a high density of access points without any spectrum issues and without resorting to low power/directional antenna which are required to do a similar thing in the 2.4GHz band.
To make OTA WiFi work well we needed to get as many clients as possible onto 5GHz. This is a challenge as many devices only support 2.4GHz so we needed to get as many as possible of the devices that do support 5GHz onto that band. One technique we used was a bit of social engineering that I first saw at Google Campus in London. As well as the standard SSID, broadcast on all APs, we also broadcast a “Fast” SSID only on 5GHz and told people to use it. That way, folks with dual frequency devices were persuaded to associate their devices only with the 5GHz bases rather than letting the device make an arbitrary choice. Of course we also made sure that the 5GHz access points were faster by allocating them 40MHz bands and leaving the standard SSIDs on them so that hopefully 5GHz capable device would choose them anyway.
The good news is that all of this worked well as nearly 50% of all sessions were on 5GHz access points this year — thank you Apple & Google Nexus which seemed to be the most numerous devices with this support.
End results
This is what the Internet usage looked like over the 2 days of the event (5 minute average utilisation graphed):
A big spike overnight, but pretty much everyone had nodded off by 4am and a bit of a slow start on Saturday. Lots of frantic uploads and downloads up to the start of judging at 2:30 and then pretty quiet until the AV guys started pushing videos up just before we took the network to bits.
Thanks
Host-IT and Skyline Networks for getting 100Mbit/s of extra Internet bandwidth into the Park at less than a week’s notice, the National Museum of Computing for letting us use their building for the microwave link, the Bletchley Park Estates Team for rigging a most elegant rope catenary to get our cable into the Marquee, and Matt for building the network inside the tent single handedly on Thursday night while I was tied up with mangling routers!