Merging Trace Files
In this post, I’ve written about how to merge trace files – both the original way and a neat shortcut to provide a way around some of the problems that you might run into while using the traditional method.
One technique that protocol analysts like to use is some sort of ring buffer or a way to capture many smaller files instead of one gigantic trace file. This scenario can range from long-term captures to capturing from a busy network/device. With either scenario, you end up with a lot of data.
For those of you who haven’t had the pleasure of experiencing opening a 1 GB trace file in a protocol analyzer, I recommend having a lot of processing power, RAM, time and definitely patience.
Let’s say that you have twenty, 500 MB trace files and you want to put them together. Here are a few things to keep in mind:
– You need to realize that technically there is a ‘gap in time’ of time where the analyzer had to stop, save and start again, resulting in potentially lost packets. Depending on your specific circumstances this may not be an issue, but if you’re not sure, you should note the last packet time in each trace file, or add a comment in the last packet to easily identify any false positives that Wireshark may flag from missing packets. This point also helps understand capture techniques that various vendors use like stream to disk.
– Always capture packets locally and then copy them to a removable drive or network drive. My general rule is to avoid capturing or streaming data to network or removable drives.
– Put your files in a separate folder to make life a bit easier – especially if you decide to merge or process packets from the command line or if you use batch files for post-capture processing.
– Try to filter the trace files by MAC, IP or TCP to reduce the file size. If you decide to filter a trace file, save them with a new file name. For example outlook.pcapng might now be outlook_Server_Tony_IP_filter.pcapng. I always say,”When in doubt about what to filter on, start with the MAC address”.
– If you don’t need the data payload, consider slicing your trace files using editcap with the –s option. So outlook.pcapng will now have a new file called outlook_128Byte_slice.pcapng. I covered packet slicing in another article in a bit more detail (http://tinyurl.com/yb38lw9j) where I packet slice a 100MB trace file, resulting in a 18MB file.
In this video, I show you how to merge trace files using the Wireshark Merge option as well as a trick to merge multiple trace files.
