Should I Setup My Rig Outside of My Environment’s DMZ?

My old team would get this question all of the time when we were scoping potential customer engagements. The customer wanted to make sure we could simulate the “Internet” for their testing. We have hardware WAN simulators in our labs, so we could easily do this. However, when we were asked to include them in the setup, we would always leave them disabled to start the engagement. We would work through the rest of the planning phase and get through initial portions of the testing phase, and then; if there was still a valid need for this type of testing, then we would enable the simulators. Below is a very brief overview of why we took this approach:

Have you ever tried to “simulate” the Internet? Go ask any network guy/gal how to do that, but be sure that person is not drinking water or a soda or coffee, because they would likely spray it all over the place trying not to gag or bust out laughing. Sorry, I am using an extreme example to illustrate the fact that the Internet is very sporadic and random, and therefore does not really meet one of the core tenets of testing: REPEATABILITY. hat kind of testing will this test rig be used for? If the rig is outside of the firewall, will you ever be able to get reliable numbers on the application itself? hat is the purpose of the test (see this post on defining the goals and objectives)? If the purpose is end-to-end testing, you can still do valid testing by having single clients on the Internet side (see this post on end-to-end testing). f the purpose is to test how the server behaves through a load balancer, then use the load balancer, but configure it to allow connections from your internal network. If this requires a separate load balancer for your test environment, then you should get one.

Here are a couple of concerns I have seen directly from development teams about testing locally instead of through the Internet:

• [Concern] To truly test something accessed from across the internet some in our organization believe our rig should be connected outside the firewall.
• [Response] You can (and should) test this part, but it does not need to be the full rig. A single client can execute meaningful tests outside of the environment while a test rig provides a stable and known load directly.
• [Concern] Would our server management’s proposed configuration (test rig should be in the same DMZ as the app) still allow us to simulate an outside connection?
• [Response] No. See previous comment
• [Concern] We don’t usually test network capacity but if a potential application requires high network capacity is there a method to gauge that?
• [Response] There are ways of simulating network behaviors, both built into V.S. and through external software and/or hardware. However, you need to have a well defined plan for when and why to do this type of testing, as mentioned in the first couple of paragraphs above.

I am not an expert at WAN simulators and all of the different hardware available, but a simple BING search on WAN Emulator Hardware will bring up plenty of good information.