Wireshark: Capture CDP and LLDP

A couple of years ago, I wrote a short piece about filtering CDP and LLDP packets using Wireshark. Since that time, I have simplified the way that I filter these packets, and based on feedback, and additional use of that information, I wanted to post an update. This will hopefully guide people to the best answer immediately. 

CDP

CDP sends all packets to the L2 multicast address of 01:00:0C:CC:CC:CC. Therefore, our filter can be:

ether host 01:00:0c:cc:cc:cc

However, VTP (VLAN Trunking Protocol) also sends packets to this address. Since the default timer for VTP is 300 seconds, and the default timer for CDP is 60 seconds, this shouldn’t be an issue. Additionally, since VTP packets are only sent out trunk ports, if you see VTP packets on a port that a user should be connected to, you may have just found your problem.

 LLDP

Link Layer Discovery Protocol, AKA 802.1AB, is an IEEE standard. While Cisco doesn’t support LLDP out of the box, it can be enabled on your Cisco gear. HP, Juniper, Dell, and everyone else that I have ever worked with supports LLDP by default. The L2 multicast address for LLDP is: 01:80:C2:00:00:0E. However, LLDP has the benefit of a unique EtherType. That type is: 0x88cc. Based on that information, we can filter with either:

ether host 01:80:C2:00:00:0E

OR

ether proto 0x88cc

The default timer for LLDP seems to vary across vendors, although 30 seconds is the default for Cisco and quite a few others.

Wireshark Portable

If you are the roving type that walks out to the users desk, Wireshark can be ran as a portable app from a USB device.

Cisco IP Phones

Cisco IP Phones will send out CDP packets onto the PC port. What good does this do? I don’t know. However, hit the webpage hosted on the phone and you can find the CDP and LLDP info on the Network Statistics >Network page.

Firewalls

Embarrassing story time. Like a lot of engineers, I regularly use Wireshark to look at packet captures from other devices. After doing this for months, I needed to use Wireshark on my local LAN port. I started by spending 20 minutes trying to figure out why I wasn’t seeing CDP packets. Of course, once I remembered that I had a local firewall to contend with, I quickly fixed the issue, and haven’t made that mistake since. Don’t make that mistake. Disable the local firewall.

Finding new coworkers

We have once again started the process of expanding our team at my workplace. We always bring new employees in as a contractor first, and if things work out, the contractor is usually offered a full time position.

Our interview process is fairly hard. First, all recruiters are provided with 10 questions, of which each candidate must answer 8 out of 10.These are all basic CCNA level questions.

Next, we schedule a phone screening, where we ask more specific questions that are broken up into different areas. Layer 2, Layer 3, OSPF, and QOS are all on tap for this portion of the interview process. If we feel that the person was able to answer enough questions correctly without frantically searching for answers to recite back to us on the internet, we move them onto the third phase.

In the third phase, the person comes into our offices, and we provide them with equipment and instructions. They have 2.5 hours to configure a router, switch, and an AP per our instructions and answer a few questions based on those configurations. Once they have completed the configuration, we move into a Q&A portion of the interview, where we ask off the wall questions, mixed with troubleshooting scenarios of increasing complexity.

Every person who has ever left an interview felt well abused. If they paid attention, they know their weaknesses, and could use it to start a personal improvement plan. We in-turn, have a solid read on each candidates abilities, strengths, and weaknesses, and whether they would make a good addition to the team.

This process is long and arduous; the last time we went through this process, we started the interview process on almost 60 people before we found three we liked. 

I can’t knock our process though. In-fact, our team is so strong that I have turned down multiple offers at other positions, which payed more, simply because I like my teammates in my current role.

It seems that every time I consider taking a different role, I get pulled into interviewing more candidates, and am reminded what it is like out there in the rest of the world. Case-in-point, here is an email excerpt from a potential job candidate:

What did I say about scheduling issues earlier in one of your calls.  All day long not a single trouble call comes in.  5 minutes before the time for the phone interview I get a call and 3 tickets logged into our dell kace service desk.  Figured since I was finally done withe the remote assistance calls working from my terminal I would drop you a line while I am on the phone with one of the users that is having problems at the entire locationlocations that is having a problem that I am trying to get through to them it is sunding like a provider problem to let me let them go and get a hold of the provider.  Always love a network that uses back up internet connections that are all from the same cable provider(comcast) so come off the same pole and think that it is a gfood redundancy feature.  Not the one I am working with but we have one service center location that has all 3 retail branches of our company and instead of  getting an upgrade on the connection type with 3 static ips for the way they like to do things but really makes no sense what so ever they have 3 cable modems all coming off the same pole so that they can supposedly have a better more stable connection makes me have nightmares about wasted money and the stupidity of the outside consultants that engineered this network.  

After speaking to you prior to the interview time and you mentioning questions about switch configurations I will kinda admit you got me thinking it has been almost 10 years since I have programmed a true cisco switch do little netgear knock offs almost weekly and switches had always been my weak point give me a router or a pix device and I could make it sing but even on those I am rusty.  Put me in a lab environment and it would be just like riding a bicycle but just giving me verbal questions I would be stumbling all over myself.  Which looking at things makes me belive that this wouldn’t be the right position for me until I get back into the game and work away some of this rust.

If you don’t feel sick after reading that email, then a part of your soul is dead already.

Hey Apple, Help Us, Help You!

When the iPhone debuted on the AT&T network, AT&T was clearly not expecting the demand that was created. They were caught off-guard by the influx of customers, and more importantly they were surprised by the data consumption of users, who had purchased a device created to consume data. Problems were reported at a ridiculous rate, and rumors abounded everywhere within the Tech blogs that Apple was threatening to take their ball phone and go home to Verizon if AT&T didn’t do something fast.

In the mean time, Apple began working on ways of optimizing the iPhones use of the carriers network, and kept pushing AT&T for improvements. It took AT&T a couple years, and a LOT of money to build their network up. Some people will argue that if the iPhone had not been made available on other carriers that AT&T would still be having issues.

Apple studies, lives and dies by user experience. They knew that a poorly performing network would reflect on their device. It was not enough to simply blame the network. If the network wasn’t available, then features of their phone weren’t available either.

With that in mind… Apple DOES NOT provide developer access to wireless API’s in IOS. Troubleshooting WLAN issues for IOS devices can only be accomplished from the infrastructure side. Without jailbreaking an iPhone, there is no way to access RSSI, SNR, or other WLAN statistics.

Which device is best for troubleshooting iPad connectivity issues on a WLAN? If you answer anything other than “another iPad”, go directly to jail, do not pass go, and do not collect $200. This is an oversight decision that Apple needs to quickly reconsider.

Apple, we are the network. Without WLAN Engineers, iPads and iPhones won’t function correctly on corporate networks. Without the proper tools, WLAN engineers cannot support IOS devices when there are issues on the WLAN. Without tools, our network problems reflect on your devices. Help US, help YOU.

Supporting Apple devices on the WLAN

Since the iPad was released, it has received a mixed welcome within Enterprise environments. While a lot companies have at least some plan to move forward with iPads, these drivers are usually coming from the business side, instead of IT. In-fact, most IT shops are being dragged into IOS support with strong reluctance.

The broad questions which are causing resistance can be summed up in one word: SUPPORT. IT departments must figure out how to support the device in multiple areas. Information integrity and control, end-user support, and connectivity support all must be dealt with. Since this is a networking blog, I want to deal with the last one; and will do so in the next two articles.

Supporting iPads on the network is more complex than connecting them to an SSID and providing login credentials. If we look at the standard iPad user in most organizations, we see a highly mobile user, users who also have laptops. Most of these users requested an iPad after having a positive experience with their company issued iPhones. That translates to a user having three wireless devices at there desk at any given time: their laptop, their iPhone and their iPad.

To understand the problem this creates, let’s look at how we survey for a wireless network. There are two considerations: coverage and capacity.

Wireless Coverage
A survey can be  based on square footage, and provide a certain RSSI from wall-to-wall. This is a perfectly acceptable way to survey if everyone has their own office. However in Cube-ville, a single AP may cover 100 desk or more. If each desk has one wireless device, you now have a physical medium (the channel or airspace) that is incapable of supporting all of the connected clients.

Wireless Capacity
The other way to perform a wireless survey is based on capacity. In a high capacity environment, the wireless spectrum, not the AP is the bottleneck. More on this later…

In a capacity based scenario, a number of desk are chosen, lets say 25. For every 25 desk, there is an AP. Those AP’s are placed based on coverage area, and in to minimize channel overlap. For the same 100 desk in Cube-ville, you now have 4 AP’s. Since there will be channel overlap, the radios are turned way down, and in general, the physical medium is now capable of handling the number of clients.

Taking this environment to the next step, each desk gets an iPhone, and a few months later, 1 in 4 request an iPad. We can safely assume that complaints will begin coming into IT about the wireless network. The AP airspace that was previously servicing 25 clients now contends with 62 per AP. Time for another wireless survey and at least twice as many AP’s!

Now we can see the problem that many companies are facing. The i-devices are here, and businesses seem to love them. The network team must begin planning and building now. I would like to make a few suggestions which might keep network teams from finding themselves behind the eight  ball.

  • Budget to begin surveying your high density environments now.
  • Develop a plan for support, complete with timelines and cost. Present this to the highest management level you can reach, so that it can be considered as the business begins planning device deployments.
  • If your company has a charge-back system for devices, be certain a cost is associated with each IOS device to support the wireless network going forward.
  • Be certain to include a survey and additional equipment as a cost of any iPad rollout projects, make certain the business can see the total cost of deploying iPads and iPhones.
  • Finally, be first in line to get an iPad if you don’t already have one. You can’t support what you don’t understand; besides, it really is a great device.

I realize that there are other options out there other than the “i” devices. However, I haven’t heard of, or seen, a single enterprise level roll out. However, these rules apply to the world of Android and Windows too. More devices per square foot equals more demand on the wireless network.

Link of the Day – IP SLA Basics

I’ve known for awhile that Cisco had an SLA feature built into IOS. I have also known for awhile that I should spend a half hour to understand the feature, and that if I did, I would begin making use of it. Sadly, I never got that far.

That’s OK though, Tony Mattke has made it so simple on his blog, you won’t need 30 minutes, you’ll need five.

http://routerjockey.com/2011/05/06/ip-sla-basics/

The velociraptor died after choking on a rib bone, so creating IPv7 is out of the question

OK, I admit it. I’ve had my head stuck firmly in the sand for almost 11 years. 11 years ago, to the month, I was sitting in my first TCP/IP class. I had fought through the first two days of class feeling mentally exhausted. I was finally beginning to wrap my head around IPv4 and variable length subnet mask. In fact, I was understanding IPv4 well enough that I could help my fellow students decipher the statements coming from our newly minted (and very proud of it) CCIE.
I was feeling pretty good about myself, and may have started to strut, just a little, as I moved from desk to desk, helping other students.
I should mention now, that I’m fairly quick on the up-take. I’m not bragging, simply stating that I meet the minimal requirements to be a geek. For some reason, I had really struggled with IPv4, so once I felt like I had a firm grasp of the concept, I was feeling pretty good.
My CCIE instructor, from his seat of power, saw a little pride develop in his class as more people caught the basics of VLSM. He, in the ultimate wisdom which comes with that coveted CCIE number, decided it was time to strangle those good feelings until they were most certainly dead. He did so, by launching into a 30 minute diatribe of how IPv4 would die a “quick death” and how IPv6 would take its place.
I’m sure you can imagine the look of horror on the faces of the students in the room. He certainly saw it, and fed off the fear as he blew through the broad topic that is IPv6. He delighted in mentioning that every device would have multiple IP’s, that each IP would be part of a different subnet. He threw out new words like anycast to a group of people who barely understood muilticast and unicast.
Wait, what?
In 30 minutes, he convinced three students that IT was not really the field they wanted to pursue, and the rest that IPv6 was EVIL. I was so affected and confused by that 30 minute rant, it took me five years before I had a practical understanding of subnetting IPv4 networks again.
Since that time, I have done my best to ignore the existence of IPv6. I used the fact that vendors were still releasing new products without IPv6 support as a reason to keep my eyes and ears firmly closed.
<My IPv6 Rant>
I believe that when IPv6 was being created someone said, “Yes, we COULD do that, but SHOULD we do that”. The rest of the attendees sat silently as he was taken from the room, and forced to watch his organs being fed to a genetically engineered, but very bored, velociraptor. The group then hired a soothsayer to read the velociraptor droppings, which gave us IPv6, reality TV, and the song “Friday”. The velociraptor died after choking on a rib bone, so creating IPv7 is out of the question.
</My IPv6 Rant>
With that said, IPv6 is here to stay, and it’s time for us, as Network Engineers, to get on board. We can’t complain about NAT64, without being willing to make the commitment to IPv6. When new protocols like TRILL are brought up for discussion, it’s easy to get excited. TRILL takes something that we already know (IS-IS, L2, etc) and simply builds on it. It is also transparent to layers 4-7, so it doesn’t affect non-network types.
IPv6, causes us to backtrack. It changes all of the rules. It’s not just IPv6, it’s new routing protocols, DNS, application stacks, etc. We have to forget what we learned in IPv4, and relearn it for IPv6. Server admins and developers will also have to update their skills. It’s painful.
With that acknowledged, we can’t put off learning to subnet, route, and filter IPv6. It’s time to begin examining IPv6 routing protocols, and buying equipment or ordering circuits which don’t support IPv6 should be out of the question. Yes, it does feel like starting from scratch. Yes, you will have to learn every protocol that you thought you knew all over again. Yes, IPv6 makes everything more complicated.
System Admins and developers can’t support IPv6 until we do. We must move forward, so that they can move forward.
Most network engineers agree that NAT is a poor solution to the problem staring us down. There are only a few other options. We can upgrade our skills, beginning the long arduous task of becoming experts in IPv6. We can ignore the change, until we are required to upgrade; then deal with entire IT teams being unprepared, learning on the fly, while implementing poor solutions in the near-term. Finally, we can make the same choice that those three classmates of mine did. “Maybe networking isn’t for me, I’ll go do something easier, like lion taming.”

Comparing Cisco and Juniper – Why doesn’t Cisco do this?

In Cisco land, we have all learned that when making changes to a remote router, you have to be ready for anything. Most admins that I know start any changes with the command:

Reload in X ( x=time in minutes to make and test changes)

The idea being that if a command is made incorrectly, and the remote router stops responding, it will reload itself in X minutes, and become accessible again since the changes would not have been saved. It’s a good idea to do this, and it’s saved my butt a few times.

However, it’s not the most elegant solution. If the router is a production level router, and the change that made the terminal unresponsive hasn’t affected it’s ability to route, it’s still going to reboot. Clients will figure out that you screwed up while they wait for the router to post, load, rebuild routing tables, etc. Darn those fat fingers, and enter key muscle memory.

Enter Juniper’s Commit Confirmed X command.

As some of you may know, changes in JunOS aren’t applied to the running config until they have been committed. (ANOTHER great idea.) Thus, it is easy to check, and recheck your changes without losing your session because the commands have yet to be applied. Once they are committed, they are all applied at once.

Now, I know what you are thinking. If you are able to cue all of your changes, review those changes for errors, and then apply them all at once, you shouldn’t ever need to rollback those changes, right? *Clears throat* Right.

BUT, if you did, somehow miss an important command, or maybe that subnet should have been a /22 instead of a /30, that’s where the Confirmed part of the command comes in. If the router stops responding, the changes are rolled back in X minutes to the previous config. No reloads, no angry emails, no voicemail, and no meetings.

To stop the router from rolling back those changes, simply commit the changes a second time after it is clear everything is working (and before X expires).