First things first, I have been getting a lot of requests to upload the lab’s which i illustrate as is, so i shall be uploading them to a Github page with initial and final-configs and Instead of vagrant i shall be using EVE-NG as a tool so that you guys can import them easily.


Going through Fabric-Path and CLOS concepts, got myself started with 3 Stage Clos and as a part of understanding it, discovered something.

Why –  To make sure Servers at one end have equal cost path to the servers-at other end, at scale the spine accordingly optimizing the CAPEX.

Simple words, in the below topology, we need to make sure that R6 has equal cost to R7 and vice-versa.


Protocols and setup

-> OSPF for the entire domain and Ibgp to peer between RR (R2) and all other loopbacks, we use OSPF so that Ibgp peering will be over Loopback and also for load-balancing protocol Next-hops

-> Default routes on R6 and R7, load-balance (per-packet) on all-routers (where technically required)

-> R3 AND R4,R5 has static back to loopbacks of R6 and R7 respectively, advertising them into OSPF will defeat the purpose obviously and these static routes are advertised into IBGP.


Lets quickly hop to R2 RR and see the bgp status, should be fine.

Ok, we have routes coming from R4 and R5, looks like there is a problem here, RR is only accepting the routes from R4 because of lower Router-id and R5 is not best, when this happens, from an End-host perspective, if R6 Sends packets to R2, it will only forward to R4 as it has only best path, can let that happen so lets do “MULTIPATH”


Added multipath, but looks like protocol has accepted it but Junos is yet to declare the other route as Active Next-hop, we need to write “EXPORT-LB” policy, lets quickly do so that it install the Next-hop into forwarding table.


Ok, things finally looking good, let see from R6 , if its getting Load-balanced.


Things look bad here, looks like R4 is still the same router which is getting burdened even though R5 is active on the network, let see why, lets get down to R3 and see how the route towards R7

Ok, we know the culprit, we have two next-hops but LB policy is missing, so lets quickly add the policy so that R3 can send packets to Two nodes equally.

Lets test the theory

Looks like we have accomplished what we have got, but wait I title this article as ADD-PATH, so how is that this is going to end ?

Well its not, Lets do one thing, Break the link between R4 and R1, it should not be a big problem right, R2 has still got the link,  R1 has got the link to R5 still a perfect candidate for LB.


Intrestingly, as Soon as R1-R4 link goes down, the path from R3 is removed , while technically R1-R5 is still available and healthy, Now comes the core topic of the post, Lets look at R1, am enabling the links.

As we can see above, RR R2 is only giving one route via R4 when it has two routes for forwarding( Remember we did multipath and export-lb on R2). Well, its not a bug, thats how BGP works, BGP would consider and advertise the best-path , in this case via R4 because of its lowest router-id.

In-order to fix this and let R1 know that there is another path available, you might think “why not multipath” , well multipath will only work when you have two paths, when you have only one-path here, there is no point of multipath.

Lets add BGP ADD-PATH send on RR as it has to send and ADD-PATH RECEIVE on R1 as it is the client. Adding ADD-PATH will reset the neighbor so do it with caution if you are on production.

Ah, finally we have R1 with two paths advertised, after this again all rules apply, first do multipath and then do LB for the new-hop.

After doing that if we go back and do the test on R6, Perfect.


Next Post will be on EBGP and IP-CLOS.



Rakesh M