Top-Down vs. Bottom-Up Approach
By Emil Prysak
Introduction
 NOC (Network Operations Center) Engineer at Weight Watchers
 Resolving various networking problems
 Wireshark used frequently
 Using top-down and bottom-up approaches depending on the given
symptoms
 Two examples:
 IP Address Loss  Bottom-Up Approach
 RightFax Email Delay  Top-Down Approach
Overview of OSI Model
 Application Layer – file transfer, email, network management,






software
Presentation Layer – data representation, encryption, protocol
conversion
Session Layer – managing connections between applications
Transport Layer – error recovery, flow control
Network Layer – routing, switching, packet sequencing, IP addressing
Data Link Layer – frame synchronization, error control, physical
(MAC) addressing
Physical Layer – cabling, bit stream, electric signal
Troubleshooting Approaches for
Networking Problems
Top-Down Approach
Bottom-Up Approach
 From Application to Physical
 From Physical to Application
 Software  Hardware
 Hardware  Software
 Works best when problem is
 Works best when problem is
isolated to one user or device
affecting multiple users or
devices
Wireshark
 Open source packet analyzer
 Able to sniff out various types of network traffic
 Can use custom display and capture filters
 Can also use Perl compatible regular expressions
 Helpful when using both top-down and bottom-up
approaches
 Version used: 1.8.1
IP Address Loss
 Some users are losing network connectivity
 “ipconfig/release” – “ipconfig/renew” gives temporary
fix, problem reoccurs when computer restarts
 Obtaining different addresses each
 Bottom-up approach will be used
 Involves physical connectivity
 Multiple users affected
IP Address Loss:
List of Components Involved
 Core router
 wwjer-ro6509-1 (Cisco Catalyst 6500 series, IOS)
 Core switches
 Wwjer-sw6509-1 (Cisco Catalyst 6500 series, CatOS)
 Wwjer-sw6509-2 (Cisco Catalyst 6500 series, CatOS)
 DHCP server “LICNAAD01” 10.80.40.34
 Wireshark server “LICPSHARK01” 10.80.40.41
 Version 1.8.1
 Multiple end users
 Jack C-345

IPs: 192.168.0.136 and 10.80.110.50
 Jack C-336

IPs: 192.168.0.137 and 10.80.110.126
IP Address Loss: First Steps
 Needed to wait until issue was confirmed by more users
 All affected users appeared to be on vlan 110
 Catalyst Switched Port Analyzer (SPAN)
 Allows certain type of traffic to be picked up by designated sniffing server
(LICPSHARK01)
 set span 110 4/17 session 1
IP Address Loss: Wireshark Trace
 Both interfaces used for
capture, encompassing
users on both switches
 Filter needed for DHCP
Packets:
 Udp port 67 or udp port
68
IP Address Loss: Wireshark Trace
IP Address Loss: Wireshark Trace
 D-Link Wireless Router
IP Address Loss:
Finding The Wireless Router
 Router was not always on, had to wait until another
user was affected
 Find router with switching commands
 Show cam 00:15:E9:F3:EF:B0
 Router was found at jack E-649
 DHCP was enabled, and giving off addresses before
DHCP server could do so
 Router was removed and problem resolved
RightFax Email Delay
 RightFax
 client software by OpenText
 receives faxed documents, converts to email, and sends
to destination mailbox
 RightFax emails delayed by hours
 TallyFax1 reported the issue first
 RightFax runs on separate server (NYCPFAX)
 Top-Down approach will be used
 Related to RightFax application
 Reported by one user
RightFax Email Delay: Sample Mail
 Fax received by server at 3:05: PM
 Email received by recipient at 7:38 PM
 Approximately 4.5 hours of delay
RightFax Email Delay:
List of Components Involved
 Field Users sending faxes
 RightFax server (NYCPFAX) 10.40.40.46
 SMTP server (NYAPSMTP03) 10.75.10.6
 Office 365 Cloud
RightFax Email Delay: First Steps
 Ideally, configuration of RightFax would be checked for problems.
 However, RightFax is an unfamiliar program, and not much was discovered.
 Saw that outgoing server was NYAPSMTP03 (Application Layer)
 SMTP logging was enabled to find a root cause
RightFax Email Delay:
SMTP Log Results
 Initial thoughts: HRBenefits mailbox was full, so no emails were getting through.
 Not valid: HRbenefits mailbox was using only 2.12 GB of 25 GB space allocated
RightFax Email Delay:
Wireshark Trace
 Jump to Network Layer
 Set up Wireshark trace on NYAPSMTP03
 Capture Filter: tcp port 25
 Display Filter: smtp.rsp.parameter matches “Mailbox
full“
RightFax Email Delay: Results
 Error message “452 Mailbox Full” was coming from
10.75.10.6 (NYAPSMTP03)
 Further research found that the
C:\inetpub\mailroot\Drop directory was full of unsent
emails
 Destination email address was incomplete, and default
was not valid.
 Once quota was hit, emails were bounced back to
NYCPFAX, clogging queue on fax server.
RightFax Email Delay: Results
RightFax Email Delay: Solution
 When WW switched from MS Exchange to Office365,





email aliasing was not considered.
With local Exchange server, “TALLYFAX1” could translate
to TALLYFAX1@weightwatchers.com
By sending to the Office365 using SMTP Relay, the
destination email address was considered invalid, and not
pick up from C:\inetpub\mailroot\Drop directory
Notification address had to be changed to full email
address for each user
Queue of emails in Drop directory on NYAPSMTP03 and
NYCPFAX had to be cleared out
After change was made, all faxes and notifications were
sent on time.
Conclusion
 Wireshark is extremely useful for troubleshooting
network issues from both types of approaches
 Network Layer is in the middle
 Makes it equally accessible from both ends
 Given more experience, a Divide and Conquer
approach may be more beneficial
 Select any layer desired



If layer has no issues, check layer above it.
If issues were found, check layer below it.
The lowest layer with errors is the culprit.