The Long Game: Why Enterprise Campus Automation is a Marathon, Not a Sprint
Summarizing Robert Mertling Blake's enterprise automation journey from AutoCon3
"Perfect is the enemy of good enough," admitted Robert Mertling Blake as he reflected on his automation journey. "I do tend to overthink things, try to foresee situations that probably won't ever happen... and that means nothing gets done."
It's a candid admission from someone managing networks across 25 venues worldwide for AEG, the global sports and entertainment company. His talk wasn't about flashy automation wins—it was about the grinding reality of transforming enterprise campus networks when business can't stop for your transformation.
The Unique Challenge of Entertainment Networks
AEG operates venues like London's O2 Arena, along with music venues, ticketing platforms, and entertainment districts globally. From a technology perspective, Blake's team faces a peculiar challenge: "We want the unforgettable experiences to be customers having a good time at concerts, not remembering that ticket scanners didn't work and they had to wait five minutes to get in."
The stakes are immediate and visible. When automation fails during a live event with 20,000 people, everyone knows instantly.
Blake's team of eight people maintains networks across three continents, serving as ISP (providing connectivity), service provider (centralized services to venues), and enterprise network team (managing infrastructure). It's an enormous scope that makes automation not just helpful, but essential for survival.
The Data Problem Comes First
"The first challenge we found that we really needed to fix was the data," Blake explained. Every site had its own documentation, its own spreadsheets. Regions maintained different systems. When opening new venues, vendor sponsorships meant using that vendor's hardware—"it's not a great look if you have an arena sponsored by Juniper with Cisco access points."
Their first major step was deploying NetBox as a "source of record" during the pandemic, when live events were shut down. But Blake deliberately chose "source of record" over "source of truth" for an important reason: "Truth sounds a bit overarching, like the network team is trying to control things... It's not my decision to make" whether other teams use your system as their truth.
The approach worked because it acknowledged reality: different teams need different sources of truth, but they should sync with the network's source of record.
Document Before Deploy
Blake introduced a concept that deserves wider adoption: "document before deploy." Too often, equipment arrives, gets installed quickly, and documentation happens "later" (which often means never).
"A lot of problems we have are because of short notice asks—the kit turned up, we put it out, and we go 'I'll come back and document it.' But none of that gets documented."
Instead, they drive configuration and assurance from documentation created during the planning phase, not from Word documents or email chains, but from structured data that can be maintained and used to detect drift.
Learning from Real Deployments
Blake shared two illuminating deployment stories:
The 2022 UK Venue: Everything was planned in NetBox, templates were ready for automatic deployment. But construction delays meant no external connectivity when they needed to deploy. Last-minute schedule changes meant they weren't prepared and had to fall back to manual configuration.
The 2024 LA Campus Refresh: They tried again with a more complex challenge—refreshing hardware while events happened nightly (three sports teams used the arena). This time, they fed NetBox with device details, generated configurations from Jinja2 templates, and reduced a 15-page deployment SOP to just 5 pages (mostly screenshots of NetBox forms).
The key insight? They didn't attempt zero-touch provisioning. "We had an incredibly capable team more than capable of copying and pasting... We didn't have time to spin up infrastructure for plug-and-play."
The Operational Reality
Blake's current architecture diagram reveals the pragmatic approach: orange components are manual processes, green are automated, gray are work-in-progress. Notably, they avoid event-driven automation during live events.
"We're not scared, but we're anxious that if something event-driven happened while a band was on stage or a sport was playing and our services fell over, it would not be good."
Instead, they trigger changes during safe windows, though determining "safe" across different venue calendars remains unsolved.
Real-World Complications
Some challenges are uniquely enterprise:
Microsoft Teams 911 Compliance: In the US, softphones must provide location information for emergency calls using LLDP or BSSID data. "We have to ensure all our devices are in the right place... otherwise the fines get messy."
Vendor Ecosystems: Different vendor platforms don't integrate well. "If you buy a Ubiquiti switch and Unify access point, you need two different controllers that don't talk to each other."
Daily Changes: Commons facilities change constantly depending on events. "Stuff is repatched daily. Frontline teams won't want to update NetBox every time." They solved this with temporary cables in different colors.
Hard-Won Lessons
Talk to Vendors Early: "We tried to do our own thing and discovered that on hardware vendors' roadmaps was this problem we're trying to solve." Don't reinvent what others are building and supporting.
80% Right Beats 100% Perfect: "A script that does something 80% right and gets it done is better than something that gets 100% right for all scenarios."
Requirements Always Change: "We've divested parts of our business, gained other parts... requirements are always changing, which means network automation requirements will always change."
Team Buy-in is Critical: Despite time zone challenges between US and UK teams, keeping everyone engaged throughout the process made the difference.
The Long View
Blake's approach acknowledges what many automation talks gloss over: this is genuinely hard work that takes years, not months. You can't just "trip around the world documenting everything" while business continues.
But the incremental progress adds up. DHCP scope creation now happens automatically from NetBox data. Device configurations deploy consistently. The team can respond faster to the summer period when all indoor venues simultaneously demand attention for refurbishment projects.
Most importantly, they're building capability that will accelerate future projects. "When we do another site, we can do it better."
Why This Matters
Blake's presentation was valuable precisely because it wasn't a victory lap. It was an honest account of automation in the messy real world where:
Business can't stop for your transformation
Perfect solutions aren't available for unique requirements
Teams span continents and time zones
One mistake during a live event affects thousands of customers immediately
His message for others starting similar journeys? "Take a try, have a play, see how it goes." Don't let the scope of the challenge prevent starting. Don't overthink every scenario.
As he concluded, "It's not a failure if something has to change. You've learned lessons along the way."
Sometimes the most valuable automation advice comes not from those who've reached the destination, but from those honestly documenting the journey.
Watch the full presentation: A Marathon and Not a Sprint: Automating Enterprise Campus Networks
