IXP FARM Tech Overview

Tech Overview

The IXP FARM (IXF) is an ultra-resilient dual stack IPv4 & IPv6 BGP 4+ service mesh. The IXF mesh takes advantage of BGP 4+ standards to scale both vertically and horizontally and allow 100% interoperability with all manufacturers and equipment conforming to IETF Internet standards.

The IXF mesh gains it’s resiliency by using a very efficient disaggregated model over a pure IPv6 backbone. The backbone supports Equal-Cost Multi-Path Routing (ECMP) capabilities and the IXF mesh allows up to 64 load balanced paths to each route. This allows for throughput in multiples of interface speeds, while also maintaining flows for traffic like VoIP and deterministic data. Taking advantage of standards, and sticking to the standards, makes our network unique and flexible as the Internet was designed to be. It allows the use of commodity hardware with 10g/40g/100g network interfaces, standard software packages trusted by every major corporation and government entity, and a network configuration that can be understood by anyone in the industry. Standards give us the flexibility to use any hardware anywhere to fit a multitude of scenarios now and into the future.

Why IPv6? The IPv4 standard was first introduced in 1980, making it over 40 years old. Less than 20 years later IPv6 was first
developed to overcome many issues with the IPv4 standard. While IPv4 is still heavily utilized, and fully supported by the IXF mesh, there are many advantages of using a pure IPv6 backbone for transit of v4 and v6 traffic. IPv6 has almost unlimited scalability. It has significantly improved on the IP protocol itself making it not only highly scalable, but also highly efficient and easy to automate at scale. We utilize the ideas behind unnumbered BGP, but we use global IPv6 addresses for many reasons. Each node also only consumes 1 single /32 of IPv4. This is an order of magnitude smaller than the typical BGP router uses, therefore significantly improving IPv4 usage in the network.

The IXF mesh is an ever-expanding set of Internet Exchange Points which offer enhanced services and features beyond what you would expect from a typical IXP.

Here are some of those features:

• • IPv4 and IPv6 global route tables installed in backbone by all participants.
  • Fully redundant peering available directly to backbone (includes transit ASN)
  • Fully redundant route servers available in each location (removes transit ASN)
  • All routes are directly injected into the backbone where eBPF, XDP, and SR SR-IOV allow 14MM+ packets per second processing.
  • Each participant with a full peer may announce any valid route from a valid ASN within RPKI and IRR.
  • IP announcements from /32 up to /16 for IPv4
  • IP announcements from /128 to /32 for IPv6.
  • This allows participants to utilize the entire network as their own transit and transport network. For example, if AS40805 wants to announce /30s or /29s at 200 sites across the fabric, those routes will be accepted, and summarized at every edge in the network.
  • Each participant may use a single IPv4 and a single IPv6 loopback address for all machines, routers, devices, which allows very efficient utilization of IP address space and a much simpler understanding of the network path.
  • Flow based Equal Cost Multipathing (ECMP) available across entire fabric. The more interfaces available the more this can be taken advantage of.
  • Community based egress filtering – Participants receive everything, and filter what they want, using community strings. All traffic types are described in our community string index.
  • Highly-tuned Linux Kernel capable of storing 10MM+ IPv4 and 10MM+ IPv6 routes.
  • BFD used across every fabric connection; BFD fully available to all participants. BFD allows for very fast failure and rerouting of traffic. While it can take BGP 30 seconds or more to converge, using BFD allows for convergence times in the millisecond.
  • Only pay per pair of ports. We like simple and fast. A bunch of traffic filtering rules to slow you down does no good for you or for us. More rules equals more complexity and less throughput.
  • Easy to identify and tag traffic by taking advantage of IPv6 nibbles, and extended communities.
  • Can be built, scaled, upgraded all in real time with zero downtime.
  • Supply chain problems will never be a problem, due to use of global and readily available commodity hardware.
  • Secure and able to handle multi 100gb DDoS attacks due to use of XDP and BPF. These are the same technologies used by Cloudflare and other CDNs.

 Communities

• • 33000:33000 – Learned from Internal/Self
  • 33000:11111 – Learned from Clients
  • 33000:22222 – Learned from other IXPs
  • 33000:44444 – Learned Transit from Provider or Client
  • 33000:66666 – Blackhole Routes

Local Preference

• • 170 – Routes received from Customers
  • 150 – Routes received from Peering
  • 140 – Routes received from Peering Backup/Remote
  • 120 – Preferred Transit Routes
  • 100 – Primary Transit Routes (BGP Default)
  • 90 – Backup Transit Routes