The PMY Pulse - Live Event Measurement: Life on Knife's Edge
No second chances. Precision, speed, and resilience - because every second counts.
Written by Gary Angel - SVP Analytics and Data Strategy | PMY Group
We just finished up a huge effort over 3 weeks at one of those massive live events that PMY specializes in. For the team on the ground, it’s an intense, high-wire experience with no safety net below. Systems cannot fail without deep repercussions: for fans at the event, for the event itself, and, of course, for us as a company. For those of us lucky enough to be well behind the firing lines, it’s still a moment of truth as software and infrastructure gets stressed in ways that are difficult to imagine let alone plan for.
What makes live events a unique measurement challenge? The challenge of sensor installation. The ephemeral nature of measurement. The short time frames. The enormous number of people. The density of the crowds. The dynamic nature of the flows. The extraordinary customer value in play at any given minute. The percentage of one-off staffers. And the challenge of creating institutional memory around a once-a-year event. It’s a perfect storm of factors. It’s the hardest kind of system to get right AND it’s the kind of system where getting it right is the only option. You can’t fail fast in this environment because you’ll never get another chance to succeed.
Some of the things that make live event measurement so damn hard are also factors in why it can be particularly rewarding. In most static-state measurement systems (e.g., Websites, stores, airports, etc.), one day is very much like another. The biggest challenge for analytic systems is finding something fresh to say. Analytics reports go unread because their basic lesson has long since been learned. And, of course, people measurement is always meant to supplement eyes-on-the-ground and the more experienced those eyes and the less variation there is on the ground, the harder it’s going to be for analytics to improve what they do. With live events, every day is fresh. No one really knows what’s going to happen. It’s usually been a full year since the last event and a bunch of stuff always changes. Most of the people on the ground are part-time, one-timers. And it is the unique aspect of live events that what people do is incredibly dynamic. It all adds up to an environment where good measurement can add enormous value.
But to add that value, you have to successfully navigate all those challenges.
The Challenge of Sensor Installation
For a lot of people-measurement projects, the most time-consuming step is the installation of sensors into the environment. There’s a lot of reasons why this step can take so long, most of them institutional. It’s also expensive. The cost of installing sensors is often roughly equivalent to the cost of buying sensors. That seems insane but that’s the reality and, given broader technology trends, the balance will probably tip even more toward the human costs. In nearly every public space and retail measurement system we create, we rely on the client’s third-party installers to go the actual implementation. That means getting them up to speed and working on their timelines. For events, the traditional design, vendor implementation, and testing process has to be dramatically short-circuited. Equipment has to go up in a couple of days. Fixed mounting points are much harder to find and often have to be created. Cabling can be hard or impossible, and we have to rely on more distributed systems. We may well even have to bring our own internet infrastructure. The short timeframes also put a lot of stress on the initial onboarding of measurement into our People Measurement Platform. You can’t take a week to clean or train data or the event might be over!
We have three different and complementary approaches to mitigating all of these challenges. First, and most important, is boots on the ground. We can put the new equipment in because that’s the quickest, easiest and by God the cheapest way to get it done and done right. And let me emphasize the “new” part of that because while there is always new equipment, if a facility has existing equipment, we may be able to use that as well with the video ML capabilities I’ve talked about elsewhere. That’s the second part of doing this well, because using as much existing equipment as we can makes the whole process easier, faster, cheaper and more reliable. Finally, we can almost always find a measurement solution that works for the environment because we have such a flexible measurement toolkit. Measurement cameras, CCTV, lidar and electronics can all be deployed. We can hard-wire the sensors or use them in distributed mode. And we often support multiple variations within each class. If you need front-facing solid state lidar on a huge area, we have a solution. If you want a top-down lidar on a high-leverage that will track full journey without occlusion? We have a solution. And if you want to just count heads in an incredibly crowded field area? Well, we have a solution for that too.
The Challenge of Short Timeframes
Setting up a new retail store is a much, much simpler process than setting up a giant live event. We might be putting 2 sensors in a store and more than a hundred around an event. Yet we often get to spend a couple weeks getting a store system installed, vetted, configured and running. It can be done quicker, but there’s usually no imperative to speed things up. Having that extra time makes everything a lot easier. It creates space where you can find problems and fix them without losing data that anyone cares about. With live events, all that space evaporates. Everything gets done in a few days before the event, and you’re often collecting vitally important data and distributing it to crowds within days of installation.
The short timeframes involved put a huge premium on several aspects of people measurement including sensor onboarding, mapping and configuration, and data cleaning. These are all areas we’ve spent a LOT of time on over the years trying to get better at, and I can tell you that while that’s paid huge dividends in this live event world, that prep work gave us just a faint echo of how challenging the timeframes are. I think the thing that has been most beneficial to us is the focus we’ve put on making sure that EVERY aspect of sensor and location configuration is in the GUI. You never need an engineer to go into a file and tweak something. That not only makes life vastly easier for everybody, it lets us provide true 24-hour global support to live events with distributed personnel.
The short timeframes also make uptime during the event incredibly important. You cannot be down. You cannot lose data. You cannot lose visibility. There’s never an end to hardening your systems in this kind of environment and I think about half of our post-event debrief focused on what we can do to be even more fault tolerant. It's a process that never ends, not least because we keep expanding our footprint at the events.
Lastly, and this one was really an eye-opener for me, but the incredibly short timeframes put a huge premium on having on-the-ground analysts doing work as the data happens. This just isn’t something we do in other verticals. But if you want to really take advantage of the information, you can’t wait until the week after the event. You have to be changing the merch store setup tomorrow. Analyst boots on the ground and what amount to analytical daily drops are critical to delivering full value from live measurement.
Too Many People
There are few locations that get as crowded as a live event. The crowd density and the number of people in a field of view are just off the charts. Even airports don’t really feature the same kind of crowd density as a concert event or a grandstand. In fact, the crowd density is so high that measuring density itself becomes critically important to monitor and prevent crowd-crush. This forms an important use-case but also a challenge to technology because most sensors have never been optimized for this use case.
This isn’t just a problem of density (though density matters). A large airport may run similar daily volumes, but that volume is spread over a longer effective day and usually a much bigger space. The number of people streaming through key areas in a live event is often way higher than in other people-measurement scenarios. That stresses all sorts of things. It stresses the lidar Perception software (which can max out in terms of the number of simultaneous objects it can track). It stresses real-time processing systems handling 10 frames a second from that number of people, and it stresses UI components trying to replay that data.
I remember the first time (years ago) we tried to do a casino in our software not a store. It was a disaster. Everything broke when our volumes and space size went up by an order of magnitude. Live events are another jump in magnitude up that scale.
The Dynamic Nature of Flows, One-Off Staff, and Institutional Memory
I grouped these three factors together because they all get at the same basic problem – the lack of deep knowledge on staff about what happens on location. If you go out and talk to a good store manager in retail, you’ll find that they are deeply knowledgeable about how their customers and Associates behave. They know when the store is crowded, they know how long lines get, they know which areas need the most staff. They know because the store does the same thing day after day after day and they are there to see it.
Live events break every part of that paradigm. They happen once a year and they change quite a bit every year. Nobody – nobody – knows how the crowd is going to flow at Lolla or Ultra or the U.S. Open because it will change based on the artists or athletes involved. And, of course, many of the people who are handling those crowds on the ground are total newbies. They are hired and there for a once a year event. Even institutional memory is far more challenging than in retail. A year is a long time. When I go out and talk to retailers, they often struggle to remember details of the seasonal setup from a year ago. They know last month's setup – but it’s easy to forget the details of what you did last Spring. Not only does measurement support that institutional memory, the presence of real-time measurement lessons the need for it while the event is live - because you get real-time eyeballs on the dynamic flows in the system. You don’t have to know or even guess what will be crowded or what the density will be, you see it.
All of this matters because the onsite time at live events is incredibly valuable. Not only do people usually pay a lot of money for live events (and reasonably expect a commensurate experience), they are eager and happy to do more and spend more on location. And folks at live events aren’t like commuters in a subway system – they’ve mostly never been before or at least haven’t been since last year. They need more guidance and support. All of which creates great opportunities for people measurement whether in minimizing entrance times so that people maximize their enjoyment, increasing ticket sales, improving the merch store efficiency and experience, or just keeping everyone safe.
High value makes for great opportunity but it also ups the challenge. There’s a huge difference between doing analytics about something and doing analytics about something people CARE about. Of all the challenges to live event measurement, that might be the best. Measurement may be harder the more it matters, but it’s a lot more satisfying to get right.
The extraordinary demands of live event measurement naturally put a premium on past real-world experience. There's a reason event organizers prefer event-focused companies. We were lucky to be able, last year, to piggy-back our people measurement onto PMY teams that knew this world inside-out. And we’ve found that having those teams there before, during and after the event is often what makes the difference between success and failure.