Fully routed does not mean fully meshed

Whenever I go to a place and start to learn the ins and outs, I always try to create three lists: The first one is “Frequently Asked Questions”. The second one is “Frequently Made Mistakes”. And the third one is “Frequent Misconceptions”. You see, to help the new guys and gals, I do not find a “Frequently Asked Questions” list to be enough just on its own. And these lists are in addition to step-by-step procedures and other material.

A misconception can lead to an error, sometimes a serious one. It is up to us that have already “walked the path” to identify and correct wrong assumptions as early as possible. Otherwise, it is more of our fault when errors are made because of them.

A misconception that I have witnessed emerging involves Active Directory replication topologies and is about the meaning of (not life, I’ll cover that in another post, but) the phrase “fully routed network”. For example, Microsoft advises to disable the default setting “Bridge all site links” if the network is not “fully routed”. So, what does this mean?

Let us take things from the beginning. A fully routed network is a network where you can ping everything from everywhere, that is, any computer in any site can ping any other computer in any other site. For a network to be fully routed, the router or routers in each site must be setup to know how to route traffic to every other site in the network.

I used ping to describe the effect, but you do not have to allow ping packets in your network if you do not want to. It is the domain controller replication traffic that you have to allow and that is important in this discussion.

In other words, if each site is reachable from every other site, the network is fully routed. A network path has to exist from any site to any other site and the routers have to be setup to forward packets from any site to any other site.

Please note that any two sites do not have to have a direct link between them for the network to be fully routed. Fully routed does not mean fully meshed.

A network where any site is connected to any other site with a direct link is called a full mesh network. Figure 1 shows examples of full mesh networks. Please note that the crossing over of physical links in the figure does not imply a connection between them. A link in the figure only connects two sites and links do not connect with each other.

A fully meshed network can certainly be fully routed but so can a network that is not fully meshed. What determines whether a network is fully routed is the way the routers are setup, not the direct or indirect connectivity between sites.

You will not find many full mesh networks in the real world. There are, of course, organizations that need to pay, setup and support connections from every site to every other site, but hub-and-spoke or partial mesh topologies are way more common. In the end, the usefulness of a certain topology depends on the business model of each particular organization.

In Figure 2, I’ve drawn a fully routed network with 3 sites. (You cannot see in the figure that the network is fully routed, you just have to assume that the routers are setup this way.) Is it possible for a domain controller in site A to replicate with a domain controller in site C? It depends. For these two domain controllers to replicate, there needs to be a path of site links that connects the two sites and also all site links in this path must be connected via a site link bridge. In this figure, there is a path of site links that connects site A and site C, namely site link 1 and site link 2. Now, if we create a site link bridge for these two site links, then domain controllers in site A can replicate with domain controllers in site C. Actually, in the figure, the site links and the site link bridge allow any domain controller to replicate with any other, no matter which site or sites they are located in.

But should a domain controller in site A replicate with a domain controller in site C? If there are no domain controllers in site B (or all of them go down), then yes, replication between sites A and C not only should, but must occur. It is the Knowledge Consistency Checker that decides these things and it usually makes good decisions, even for complex networks. Well, anyway, you help it yourself by either bridging all site links, bridging no site links or creating site link bridges as appropriate to your business model and network topology.
 
In Figure 3, I’ve drawn a fully routed network with 4 sites. (Again, you cannot see in the figure that the network is fully routed, you just have to assume that the routers are setup this way.) How can a domain controller in site A replicate with a domain controller in site D? We see that a path of site links exists, namely site link 1, site link 2 and site link 3, that leads from site A to site D. We also need a site link bridge to connect the three site links. Then domain controllers in site A and site D can directly replicate with each other. Again, in the figure, the site links and the site link bridge allow any domain controller to replicate with any other, no matter which site or sites they are located in.

Please note that we have only one site link bridge in Figure 3. This site link bridge connects three sites links. Compare this with the fully routed network configuration I’ve drawn in Figure 4. In the latter, the only difference is that instead of a single site link bridge, we have two, one site link bridge for site link 1 and site link 2 and another site link bridge for site link 2 and site link 3. Now, site link bridges are not transitive (for that we would need bridges for the site link bridges!, and there is no such thing), so in Figure 4, domain controllers in site A cannot replicate with domain controllers in site D. If you really want that, then that’s fine, otherwise implement the configuration in Figure 3.

I just noted that site link bridges are not transitive, and this is a good thing. This way you can create any configuration you want. If you want a group of site links to be transitive, put them all in the same site link bridge as in Figure 3. If you do not want to connect two site links, put them in different site link bridges, or in no site links bridges at all.

When you accept the default option “Bridge all site links”, then this is the equivalent of all site links being placed in one (big happy) site link bridge. This is the effect that we see in Figure 2 and Figure 3, but not in Figure 4. By accepting “Bridge all site links” in Figure 2 and Figure 3, the site link bridge is conceptually created for us. To achieve the configuration in Figure 4, we must disable the option “Bridge all site links” and create the two site link bridges ourselves.

The networks in Figure 2 and Figure 3 are fully routed, because the routers have been setup in such a way that packets can be directed to any site they are destined to. Let us suppose for a minute that these networks are not fully routed. Then it would be wrong to create one site link bridge connecting all sites. Site link bridges have meaning only when there is site link transitivity, that is, the routers are setup for the corresponding site links to route transitively.

Neither the network in Figure 2 nor the networks in Figures 3 and 4 are fully meshed. The network in Figure 2 is not fully meshed because site A is not connected to site C by a direct link. The networks in Figures 3 and 4 are not fully meshed because site A is not connected to site C by a direct link, site A is not connected to site D by a direct link and site B is not connected to site D by a direct link.

Remember when creating site link bridges that, as Microsoft states, each site link in a bridge must have a site in common with another site link in the bridge. And as a best practice, site link names should (in some way) denote the sites they connect and site link bridge names should (in some way) denote the site links they connect. I do not follow that practice here (because for some strange esoteric reason I want to emphasize enumeration), but you should.

Ring topologies are actually quite common, so here is an example for those as well. In Figure 5, I’ve drawn a fully routed network with 4 sites and 4 site links in a ring topology. If you accept the default “Bridge all site links”, one site link bridge connecting the 4 site links will conceptually exist and is depicted in the figure. If we wanted to create it ourselves (there is no reason to do it manually, but just to illustrate), we would have to disable the option “Bridge all site links” and create one site link bridge, to which we would add the 4 site links.

And if we had a fully meshed and fully routed network, then accepting the default option “Bridge all site links”, a conceptual site link bridge would be created containing and connecting all site links.
 
And some parting thoughts. If your network is not fully routed, this means that not all your site links are transitive. You must disable the default setting “Bridge all site links” in this case. Then create site link bridges where you want to denote site link transitivity.

It is possible that you create no site link bridges whatsoever. Any network with a domain controller in every site can function and replicate without any site link bridges. And this can be stretched even further. You can add edge sites with no domain controllers, which use the “central” sites for authentication. It depends on what you want to achieve. But you do have to account for domain controller failures in this or any other configuration and plan accordingly.

If your network is fully routed, it is up to you to leave the default setting “Bridge all site links” to denote that all site links are transitive or to disable the setting and create site link bridges on your own.

Even if your site is fully routed, you will not always benefit from bridging all site links. There are cases in which you need to logically segment your network, if you do not want replication traffic to flow directly between satellite sites in a hub-and-spoke topology or if you want only specific domain controllers to communicate between parts of your networks, due to firewall placements or other issues.

 

Figure 1: Full Mesh Networks

Figure 1: Full Mesh Networks

Figure 1: Full Mesh Networks.

 

Figure 2: A fully routed network with 3 sites

Figure 2: A fully routed network with 3 sites

Figure 2: A fully routed network with 3 sites.

 

Figure 3: A fully routed network with 4 sites

Figure 3: A fully routed network with 4 sites

Figure 3: A fully routed network with 4 sites.

 

Figure 4: Two site link bridges instead of one

Figure 4: Two site link bridges instead of one

Figure 4: Two site link bridges instead of one.

 

Figure 5: Another fully routed network with 4 sites

Figure 5: Another fully routed network with 4 sites

Figure 5: Another fully routed network with 4 sites.

Advertisements

About Dimitrios Kalemis

I am a systems engineer specializing in Microsoft products and technologies. I am also an author. Please visit my blog to see the blog posts I have written, the books I have written and the applications I have created. I definitely recommend my blog posts under the category "Management", all my books and all my applications. I believe that you will find them interesting and useful. I am in the process of writing more blog posts and books, so please visit my blog from time to time to see what I come up with next. I am also active on other sites; links to those you can find in the "About me" page of my blog.
This entry was posted in Administration. Bookmark the permalink.