Skip to main content

Traffic Engineering

Authoritative Resources​

BGP Traffic Engineering:

MPLS and Segment Routing:

BGP Traffic Engineering​

Local Preference (Outbound TE)​

Control outbound traffic by setting LOCAL_PREF on received routes.

Mechanism: Higher LOCAL_PREF wins in path selection (criterion 2).

Use cases:

  • Prefer certain peers/transit for specific prefixes
  • Implement primary/backup path selection
  • Direct traffic based on business relationships

Example scenario:

ISP has two transit providers: Tier1-A (expensive, low latency) and Tier1-B (cheap, higher latency)
Strategy: Set LOCAL_PREF=200 for routes from Tier1-A, LOCAL_PREF=100 for Tier1-B
Result: Outbound traffic prefers Tier1-A unless those paths are unavailable

AS Path Prepending (Inbound TE)​

Artificially lengthen AS_PATH on outbound announcements to make paths less attractive.

Warning: Cloudflare research shows prepending is often ineffective and can cause harm. Use communities instead when possible.

Why prepending is problematic:

  • BGP decision process checks LOCAL_PREF before AS_PATH length
  • Many networks override with LOCAL_PREF policies
  • Excessive prepending makes routes vulnerable to hijacking
  • Creates routing table pollution

When prepending might be appropriate:

  • Last resort when communities aren't available
  • Limited to 1-3 prepends maximum
  • Only on specific announcements, not entire table

Better alternative: Use provider communities for selective depreference.

MED (Multi-Exit Discriminator)​

Suggest to neighboring AS which of your entry points to prefer.

Scope: Only compared between routes from same neighboring AS.

Use cases:

  • Multiple peering points with same AS
  • Prefer geographically closer ingress
  • Load balance across multiple links

Limitations:

  • Not transitive (stripped by most ASes)
  • Many networks ignore MED from customers
  • Only works with single peer AS

Configuration approach: Set MED based on IGP cost or geographic preference.

BGP Communities for Traffic Engineering​

How it works:

  • Transit/peer publishes community values with associated actions
  • Customer tags routes with community
  • Provider applies action (prepend toward regions, adjust LOCAL_PREF, filter)

Example community actions:

Provider's community 64512:100 = deprioritize in North America
Provider's community 64512:200 = no export to peers
Provider's community 64512:666 = blackhole/null route

Advantages:

  • More predictable than prepending
  • Provider controls how traffic is influenced
  • Can combine multiple communities for fine-grained control

Check provider's documentation: Most major transit providers and IXs publish community lists.

Conditional Advertisement​

Announce prefixes only when specific conditions are met.

Use cases:

  • Backup path advertisement (only announce when primary fails)
  • Anycast load balancing
  • Maintenance windows

Mechanism: Conditional based on presence/absence of route in BGP table.

MPLS Traffic Engineering​

MPLS-TE enables explicit path setup with bandwidth reservations.

Protocols:

  • RSVP-TE: Resource Reservation Protocol with TE extensions
  • Segment Routing (SR-MPLS): Modern alternative without per-LSP state

Advantages over IGP shortest path:

  • Reserve bandwidth along path
  • Explicit paths independent of IGP metric
  • Fast reroute (FRR) for sub-50ms convergence
  • Multiple paths to same destination

Disadvantages:

  • RSVP-TE requires state on every node
  • Complex to configure and troubleshoot
  • Doesn't scale beyond thousands of tunnels

Segment Routing​

SR eliminates per-tunnel state while maintaining TE capabilities.

How it works:

  • Source encodes entire path as stack of segment IDs
  • Transit nodes don't maintain tunnel state
  • Uses existing MPLS or IPv6 SRv6 data planes

Advantages:

  • Scales better than RSVP-TE
  • Simpler operations
  • Enables SDN controllers to compute paths
  • Fast reroute without state

Migration path: Can coexist with MPLS-TE and LDP.

For details: See Cisco's SR migration guide

Monitoring and Measurement​

Essential metrics:

  • Per-peer traffic volume (inbound/outbound)
  • Per-prefix performance (latency, packet loss)
  • Path diversity (how many paths to critical destinations)
  • Traffic distribution across links

Tools:

  • NetFlow/IPFIX: Traffic accounting and analysis
  • sFlow: Sampled flow monitoring
  • gRPC/Streaming Telemetry: Real-time metrics from routers
  • BMP (BGP Monitoring Protocol): BGP-specific monitoring

What to monitor:

  • Utilization on all links (aim for <70% sustained)
  • Latency to major destinations via each path
  • BGP path changes (excessive churn indicates problems)
  • Traffic ratio shifts (sudden changes may indicate issues)

Load Balancing​

ECMP (Equal-Cost Multipath)​

Distribute traffic across multiple equal-cost paths.

Hash-based: Traffic is hashed (src/dst IP, port, protocol) to select path.

Considerations:

  • Per-flow, not per-packet (avoids reordering)
  • Hash distribution may not be perfectly equal
  • Elephant flows can cause imbalance
  • Path failure causes rehashing (some flows move)

Configuration: Most vendors support ECMP by default when multiple paths have equal cost.

Unequal-Cost Load Balancing​

Distribute traffic proportionally across paths with different metrics.

BGP Add-Path (RFC 7911):

  • Advertise multiple paths for same prefix
  • Receiver can use all paths, not just best
  • Enables ECMP across eBGP sessions

Weighted ECMP: Some platforms support varying the weight per path.

Traffic Steering vs. Path Computation​

Local decisions (traditional):

  • Each router makes independent path selection
  • Based on BGP attributes and IGP metrics
  • Distributed decision making

Centralized computation (SDN):

  • Controller computes optimal paths
  • Programs paths via SR, OpenFlow, or PCEP
  • Global optimization possible
  • Requires accurate topology and demand matrix

Hybrid approach: Use BGP/IGP for baseline, SDN for specific high-value traffic.

Practical Recommendations​

  1. Start with LOCAL_PREF for outbound control - Simple, effective, well-understood.

  2. Use provider communities instead of prepending - More reliable, less harmful.

  3. Monitor before optimizing - You can't improve what you don't measure.

  4. Avoid excessive complexity - Every TE knob you turn is a failure point.

  5. Document your policies - Future you will need to understand current you's logic.

  6. Test changes carefully - TE changes can have unexpected consequences.

  7. Keep it simple - The best TE is often just good network design.

Integration​

With BGP Operations: See BGP for path selection algorithm and attribute details.

With Routing Security: See Routing Security - Don't let TE bypass security.

With Network Automation: See Network Automation for automating TE policy deployment.

Sources: