From Scripts to Services: API Gateways for Network Automation
The network automation community has been wrestling with a fundamental question: how do you build flexible, consumable automation without becoming locked into a single orchestrator or vendor? A recent discussion among network engineers revealed some practical insights about using API gateways as an abstraction layer for automation (micro)services.
The Problem: Orchestrator Dependency
Traditional automation approaches often center around a single orchestrator, whether it's Ansible Tower, open-source software, commercial workflow engines, or cloud-native solutions. While these tools solve orchestration problems well, they create new challenges:
Lock-in: Switching orchestrators after building hundreds of workflows is painful
Cost scaling: Enterprise orchestrators can become expensive at scale (15,000+ devices)
Integration complexity: Getting different teams and tools to work with your chosen orchestrator
Adoption barriers: Complex orchestrators can discourage usage by other teams
The API Gateway Approach
The core idea is simple: create a unified API interface that can trigger any automation, regardless of what runs it behind the scenes. This means:
SoT (source of truth) webhooks can trigger the same automation as manual requests
Playbooks, scripts, and workflows all become interchangeable backends
Teams consume automations through consistent APIs rather than learning tool-specific interfaces
You can replace implementation details without breaking consumers
Real-World Architecture Patterns
Engineers in production are implementing this in several ways:
Microservices Style: Each automation becomes a service behind an API gateway. This works well for complex environments but brings the usual microservices complexity tax; message brokers, distributed tracing, and service mesh considerations.
FaaS/Serverless: Running automation scripts as functions (like AWS Lambda or open-source FaaSD). One engineer reported three years of success with this approach, calling individual Python scripts or Ansible playbooks as containerized functions.
Hybrid Approaches: Using tools like Temporal or Prefect to handle workflow orchestration while exposing everything through consistent APIs.
Practical Implementation Strategies
Start Simple, Evolve Gradually
The biggest mistake is trying to build a complete microservices architecture on day one. Instead:
Single API service: Start with one service that calls your existing scripts and playbooks
Add abstraction gradually: Extract common patterns as your needs become clear
Avoid premature optimization: Don't add message brokers and distributed tracing until you actually need them
The Streamlit Pattern
An interesting approach mentioned is using Streamlit as a self-service portal that makes API calls to trigger automation tasks. This provides:
Quick wins with minimal development overhead
Self-service capabilities for other teams
Authentication and authorization (recently added)
Rapid prototyping for new automation workflows
One engineer reports success building business-specific functions quickly, then throwing them away when requirements change rather than maintaining complex, generic solutions.
Focus on Event-Driven Design
Modern automation increasingly follows event-driven patterns:
SoT device creation triggers EOL checks and compliance validation
IP allocation events trigger documentation updates and monitoring configuration
Manual prefix allocations trigger validation workflows and notifications
This aligns well with API gateway patterns where events from any source can trigger standardized automation workflows.
Tool Selection Reality Check
The discussion revealed some practical insights about tooling:
Temporal: Gaining significant traction for complex workflows requiring durability, error handling, and multi-step processes. The learning curve is steep but worthwhile for sophisticated automation.
Simple Orchestrators: For many use cases, lightweight orchestrators or even cron jobs calling APIs may be sufficient.
Existing Tools: Many organizations already have pieces that just need better integration rather than complete replacement.
The Build vs. Buy Decision
Building your own API gateway solves vendor lock-in but creates different lock-in; to your own code and architectural decisions. Key considerations:
Team size: Custom solutions require ongoing maintenance
Complexity requirements: Simple use cases may not justify custom development
Migration costs: The pain of switching tools vs. the cost of building and maintaining
Lessons from Production
Engineers with years of production experience offered several key insights:
Embrace Impermanence
In fast-changing network environments, building for immediate business value rather than long-term architectural purity often makes sense. If vendor changes every few years are inevitable, couple your automation closely to current tools and plan for replacement cycles.
API-First Design
Making everything communicate via APIs—even internal components—provides genuine flexibility. When automation task A needs to trigger task B, forcing communication through APIs rather than direct calls creates cleaner boundaries.
The Complexity Tax is Real
Every abstraction layer adds operational overhead. Message brokers, service discovery, distributed tracing, and observability tooling all require expertise to operate reliably. Add them when you need them, not because blog posts say you should.
Self-Service is a Force Multiplier
The most successful automation projects enable other teams to consume services themselves rather than requiring operator intervention. Whether through web portals, APIs, or both, reducing friction for consumers dramatically increases automation adoption.
Practical Next Steps
For engineers considering API gateway approaches:
Audit existing automation: Catalog what you already have and how it's currently triggered
Start with high-value, low-risk workflows: Choose automations that are well-understood and frequently used
Build thin wrappers first: Create simple APIs around existing scripts before adding orchestration complexity
Measure actual usage: Track which automations get used to guide further investment
Plan for evolution: Design APIs that can evolve as backend implementations change
The Bottom Line
API gateways for network automation aren't revolutionary—they're sensible abstraction that addresses real operational challenges. The key is balancing architectural flexibility with implementation pragmatism.
Start simple, solve real problems, and evolve based on actual usage patterns rather than theoretical requirements. Whether you use Temporal, Streamlit, or carrier pigeons, the best automation is the one that actually gets used by your teams.
The network automation community is converging on similar patterns for good reasons. The challenge isn't choosing the perfect architecture. It's shipping automation that provides immediate value while maintaining flexibility for future changes.
This post is based on community discussions and represents the collective experience and opinions of individual practitioners, including: Adam Angell, Ryan Shaw, Urs Baumann, Steinn (Steinzi) Örvar, Pete Crocker, John Howard, and Dan Peachey. Approaches should be evaluated and adapted based on your specific network environment and requirements.
The conversation continues in the Network Automation Forum community – find us on Slack or LinkedIn.
