[GRLUG] Rebooting linux server

[Adam Tauno Williams]

On Fri, 2012-07-06 at 10:13 -0400, Michael Mol wrote: 
> On Fri, Jul 6, 2012 at 9:58 AM, Collin Kidder <collink at kkmfg.com> wrote:
> > On 7/6/2012 9:35 AM, Michael Mol wrote:
> > I've since set up rsyslog to do remote logging to a second machine. Of
> > course, now that I've done this the machine has been up for longer than a
> > day so far with no hiccups. But, when it happens again I'll be ready. I have
> > mcelog installed as well so any mce messages will be remote logged.
> My only fear of remote logging is my suspicion it's not quite as
> resilient to things like kernel panics as a serial port. But it's a
> good step.

It is not very resilient.  You *might* get a few of the first messages
[logging via UDP] but quite likely the network stack will be gone and
syslog inoperable before the kernel does the sysrq thing and posts its
dumps.

> Meanwhile, with remote logging set up, you should play around and see
> if consolidating your networks' logs might evolve some additional
> benefit for you. Perhaps you could shorten your logrotate period on
> the source machines and lengthen it on the destination, for example. A
> consolidated logging point might be a good place to look for systemic
> issues like malicious network scans, too.

Several NMS systems such as ZenOSS support being syslog receivers as
well.  Using those as targets you get a nice event system and the
ability to categorize messages with priorities, etc...  And at least
ZenOSS will automatically age out messages.  I use this and change local
logging to an absolute minimum [and who wants to burn up SAN bandwidth
writing tons of crap to /var/log/messages].

It is useful to keep a Cisco 2511 [they are old and *CHEAP*] in your
rack, that gives you 16 serial ports than can receive data and relay it
over the network to wherever you want.  So you can connect the 2511 to
the server's serial port, enable serial port logging, and tell the 2511
to send it to {host}:{port} where you have a netcat in listen mode
dumping it to somewhere / something.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://shinobu.grlug.org/pipermail/grlug/attachments/20120706/aa96c2ba/attachment-0001.pgp>