[GRLUG] Routing Weirdness

[Adam Tauno Williams]

On Wed, 2014-01-29 at 03:32 -0500, Dave Chiodo wrote:
> I think if we want to get to the bottom of this, the following info
> will help:
> - a network diagram showing each router, and its
> relationship/interconnections with other routers

That is contained in the description.

  Site-A[INET<-->[96-35-167-37/192.168.2]]
    <--> Site-B[[192.168.1.254/24-182-144-137]<-->INET]

with a server @ Site-B.

At least, that is my reading.

> - for each router, the output of "show ip route" (for cisco IOS
> devices - or the equivalent for any other brand of router) (Feel free
> to mask the first two octets of any public IP addresses, replace them
> with "A" or "B" etc, if desired - make sure the physical interfaces on
> the network diagram use the same convention, if so)

Ideally, provided the Cisco devices are actual Cisco IOS devices and not
rinky-dink rebranded turd sculptures with power bricks you want a
configuration of

  snmp-server host $My.NMS.HOST.IP  $My-Community-Name
  access-list 99 permit $My.NMS.HOST.IP
  snmp-server community $My-Community-Name ro 99
  snmp-server ifindex persist
  snmp-server enable traps snmp authentication coldstart linkdown linkup
warmstart
  snmp-server enable traps envmon

That will allow the NMS to model the device via SNMP, and cause the
device to send the basic traps to the NMS.  If your NMS is not lame and
retarded there are other traps types you can enable.
  
Getting at least these snmp type traps is 101.
  coldStart -  you're router rebooted!
  linkDown - an interface went down
  linkUp - an interface came up
  authentictionFailure  - that was you, right?
  reload - someone rebooted your router, that was you right?

But if you can access the console of the device, and it is an actual
Cisco IOS device, you can manually look at lots of stats with the "show
int" command.

Serial0/0/0 is up, line protocol is up 
  Hardware is GT96K with integrated T1 CSU/DSU
  Description: MPLS: Some Site
  Internet address is X.X.X.X/30
  MTU 1500 bytes, BW 1544 Kbit/sec, DLY 20000 usec, 
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation PPP, LCP Open
  Stopped: CDPCP
  Open: IPCP, loopback not set
  Keepalive set (10 sec)
  Last input 00:00:00, output 00:00:00, output hang never
  Last clearing of "show interface" counters 4d23h
  Input queue: 0/75/0/0 (size/max/...); Total output drops: 4460
  Queueing strategy: Class-based queueing
  Output queue: 0/1000/0 (size/max total/drops)
  5 minute input rate 3000 bits/sec, 3 packets/sec
  5 minute output rate 3000 bits/sec, 4 packets/sec
     9264796 packets input, 451012271 bytes, 0 no buffer
     Received 0 broadcasts (0 IP multicasts)
     0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     9973301 packets output, 3482128267 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 output buffer failures, 0 output buffers swapped out
     0 carrier transitions
     DCD=up  DSR=up  DTR=up  RTS=up  CTS=up

Depending on the interface type the stats may mean different things - if
your "carrier transitions" is greater than ZERO then your interface is
going up and down [at least at some point].

You can clear the counters with "clear counters" to have a point in time
to baseline from.  You can show the router's perception of time with
"show clock".

doing & cut-n-pasting this into your ticket is like a poor man's NMS.

router#clear counters
Clear "show interface" counters on all interfaces [confirm]y
router#show clock
*07:49:54.694 EASTERN Wed Jan 29 2014

Then at the end of the day go back and look at the counters again.  Note
that the interface records the last time it's counters were cleared.

  ....Last clearing of "show interface" counters 4d23h....[from above]


-- 
Adam Tauno Williams <mailto:awilliam at whitemice.org> GPG D95ED383
Systems Administrator, Python Developer, LPI / NCLA