Introduction
Recently, I have been working on some network segmentation projects at work involving L3VPNS, MPLS, BGP, and firewalls. This is a new area for both myself and many of my co-workers, so there is a lot to learn as we go along.
Recently, we got our firewalls deployed into our network and the BGP peering turned up. As some of my co-workers started poking around to make sure everything was working correct, they notice the test networks weren't being passed into our core. After some investigation, it turned out to an issue with a simple BGP principle: don't accept routes from your own ASN.
Network Setup
I was able to mock this setup in our lab to be able to tackle this problem. Here is the setup I used:
These devices don't handle production traffic, so mucking around with them was not an issue. Here is the break down of what I have setup:
AS65200
With in AS65200, there are 6 routers:
- BAKER:TRUSTED is a logical system on the Juniper MX80 named BAKER. It also serves as one of the route reflectors.
- BAKER:PRIME is also a logical system on BAKER. It is serving as an additional hop with in the core ring.
- HULK is a physical Brocade MLXe 16 router.
- COBBLER:TRUSTED is a logical system on the Juniper MX80 named COBBLER. It also serves as one of the route reflectors.
- FLOUNDER is also a Brocade MLXe 16.
These devices are all running:
- OSPF for the IPv4 IGP
- IS-IS for the IPv6 IGP
- iBPG peerings to the route reflectors
- LDP for label distribution for MPLS
AS65003
This AS contains a single device, BAKER:FAKE-FW, a logical system on the BAKER MX80. The firewalls were taken from me for this testing, so I had to setup a router to act as the firewall for the purposes of this testing.
AS65004
This is a special AS. As we are building out network segmentation, HULK:NOC1_FW_TEST is a VRF on HULK. It is also a member of an L3VPN within our lab network.
Goals
The goal for this design is to get the routes from within AS65004 to appear in AS65200 while passing through AS65003. This will give us the effect of the "firewall" bridging the two networks.
Problem
When investigating the problem, I was really curious about why the routes weren't even showing up in the BGP route table on HULK. If they were in there, I would at least be able to see why they weren't being installed into the routing table itself.
After a bunch of poking round, I discovered that HULK:NOC1_FW_TEST was adding the AS65200 to the end of the AS path. If we look at BAKER:FAKE-FW we can see this:
charlesr@baker> show route receive-protocol bgp 10.120.40.10 logical-system
FAKE-FW
inet.0: 11 destinations, 12 routes (11 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
10.120.40.8/30 10.120.40.10 0 65004 65200 ?
* 10.120.40.123/32 10.120.40.10 0 65004 65200 ?
* 10.120.40.125/32 10.120.40.10 0 65004 65200 ?
* 10.120.40.128/26 10.120.40.10 0 65004 65200 ?
charlesr@baker>
As you can see, we are receiving the routes from HULK:FAKE-FW on BAKER:FAKE-FW with the AS65200 in the AS path. This is going to cause issues when BAKER:FAKE-FW advertises the routes up to HULK.
Remember, in an effort to prevent routing information loops, BGP will not accept any routes for which the AS path contains its own AS. In the output above, HULK isn't going to accept this route as AS62000 is in the AS path.
What I had discovered is even though I had established the peering between HULK:NOC1_FW_TEST and BAKER:FAKE-FW using the AS65004 as the local AS to HULK:NOC1_FW_TEST, the VRF still considered itself apart of AS652000. This results in the routes being originated from AS65200.
Solutions
There are a number of different manners to tackle this problem. Some of them are better than others.
Allow AS Option
The allowas option can be added to neigbors within the BGP configuration. This allows the router to accept routes with its on AS in it. There has to be a limit to the number of times the AS can appear.
This would allow the routes into the HULK. The problem with this method is it has be configured on all routers connected to the route reflectors, and the route reflectors themselves.
Talk about high touch configuration.
Strip Private AS
One of the reasons this issues caught me off guard was in the lab I was striping off private ASNs within HULK:NOC1_FW_TEST before advertising them up to BAKER:FAKE-FW. This removed the AS65200 from the AS path, and allowed the route to move on it merry old way through the network.
The probelm with this is in production, a private AS isn't used for our route reflectors. So, this isn't going to do us much good there.
Change the AS of the VRF
While washing some dishes later in the evening, I started pondering how to take care of this issue. Nothing I had tried was working. Thinking through the problem, I realize that I either needed to rip off the AS65200 from the AS path, or change it.
I opted to find a way to change it. After a flurry of hitting the question mark key on HULK I discovered your could change the AS of the VRF itself. Baza!
If the following following change is made to the BGP configuration on HULK:
router bgp
address-family ipv4 unicast vrf NOC1_FW_TEST
no neighbor 10.120.40.9 local-as 65004
local-as 65004
exit-address-family
!
I get the results I was looking for! This changes the origin AS for the route, and doesn't apply the AS65200 to the AS path. After making the change, we can see the changes to the received routes on BAKER:FAKE-FW:
charlesr@baker> show route receive-protocol bgp 10.120.40.10 logical-system
FAKE-FW
inet.0: 11 destinations, 12 routes (11 active, 0 holddown, 0 hidden)
Prefix Nexthop MED Lclpref AS path
10.120.40.8/30 10.120.40.10 0 65004 ?
* 10.120.40.123/32 10.120.40.10 0 65004 ?
* 10.120.40.125/32 10.120.40.10 0 65004 ?
* 10.120.40.128/26 10.120.40.10 0 65004 ?
charlesr@baker>
The routes also appear in the hulk routing table as on would expect:
SSH@hulk#show ip route 10.120.40.0 255.255.255.0 longer
1 10.120.40.0/30 DIRECT ve 950 0/0 D 1d22h -
2 10.120.40.96/30 192.5.44.24 ve 101 110/11 O 37d21h -
3 10.120.40.120/32 192.5.44.24 ve 101 110/10 O 37d21h -
4 10.120.40.123/32 10.120.40.2 ve 950 20/0 Be 4m6s -
5 10.120.40.125/32 10.120.40.2 ve 950 20/0 Be 4m6s -
6 10.120.40.128/26 10.120.40.2 ve 950 20/0 Be 4m6s -
SSH@hulk#
This has been applied in our production environment, and is working as expected.
It should be noted the same problem arised for our IPv6 peerings also. On the Brocades, when you apply the local-as to the IPv4 VRF within the BGP configuraton, it gets applied to the WHOLE VRF, not just the IPv4 side. On the Junipers, this can be accomplished with the following:
set routing-instances NOC1_FW_TEST routing-options autonomous-system 65004