This is the second of three posts about planning, executing, and validating network configuration changes. The first post discusses the planning phase of network configuration changes. The last part is coming up in two weeks.
Executing Network Configuration Changes
This is the day of the launch when the rubber hits the road. Here are some recommendations for proceeding at this phase:
- Don’t panic – If you have thoroughly prepared and documented every step and detail of the procedure, you should be calm and confident. It is normal to be nervous before a change, but make sure to use this energy to be alerted in this delicate phase so you will avoid stupid mistakes.
- Get enough sleep – Especially if the change is happening at night and takes several hours, make sure to get enough sleep before it. Make sure to plan with your manager or team lead to leave work earlier, work from home or even just not come to work, so you are rested enough. You want to make sure to have enough energy and attention to troubleshoot and follow-up with the outcome of the change if things don’t go as planned. Unexpected and technicalities may always happen when you are dealing with a large and complex network environment. The best you can do is to have enough energy to troubleshoot and repair the problem in the quickest way possible.
- Review your assumptions – You should expect to find the network and its resources in the state they were when you planned the configuration change. If there’s a single deviation (see Snapshot section in the Planning phase), you will have to assess whether this will impact the outcome of your network configuration change. If this is the case you may have to cancel and postpone it.
- Outside factors – If there are other events or conditions inside or outside the network (weather, unrelated network and/or application outages, …) that might negatively impact or add noise to the outcome of the change you might want to consider canceling or postponing the configuration change.
- Tools – First of all, make sure to use all the tools that you are most comfortable with: don’t try to use new tools that may introduce errors or problems due to their improper use. Second, bring all the necessary ones that you may need based on the type of change that you are dealing with. Does your work include …
- … adding/rerouting fiber links? Bring a fiber cleaning kit in case you have to deal with old fibers.
- … working with private circuits from carries? Make sure to have all the escalation/support numbers to get an engineer on the line to troubleshoot suspected circuits issues.
- … coordinating with other network engineers deployed in the field? Make sure you have a conference bridge available that you can use to communicate. If phone connectivity is not available, consider using radios.
- IP Inventory – If your change affects user subnets, run an IP scan with your preferred tool (I generally use “nmap –sP <subnet/ip range>”) before and after your change to make sure that all the workstations/servers/desktops are back online without any surprise the next day.
- Ground control – When you execute the configuration change procedure, make sure to get the approval to proceed from your Network Operation Center as well as from the affected users and or business departments if needed.
In the upcoming blog post we will discuss the different tools (NetBeez included) and best practices available to network engineers during the validation phase of network configuration changes.
How do you implement your configuration changes? Please feel free to comment and share any feedback you have.
