Nov 04 2008

nginx 和 syslog-ng 配合

Category: 技术ssmax @ 11:11:10

nginx本来不支持syslog的,虽然有个补丁,试了一下,配置性太差,error log 和access log 不能分开配置。。。

然后就想了一个方案,用syslog-ng ,也不用改nginx的源代码了,syslog-ng可以支持pipe输入

The sources, destinations, and filters available in syslog-ng are listed below. For details, see The syslog-ng Administrator Guide .

Name Description
internal() Messages generated internally in syslog-ng.
unix-stream() Opens the specified unix socket in SOCK_STREAM mode and listens for incoming messages.
unix-dgram() Opens the specified unix socket in SOCK_DGRAM mode and listens for incoming messages.
file() Opens the specified file and reads messages.
pipe(), fifo Opens the specified named pipe and reads messages.
tcp() Listens on the specified TCP port for incoming messages.
udp() Listens on the specified UDP port for incoming messages.
tcp6() Listens on the specified TCP port for incoming messages over IPv6.
udp6() Listens on the specified UDP port for incoming messages over IPv6.
sun-stream(), sun-streams() Opens the specified STREAMS device on Solaris systems and reads incoming messages.

Table 1.1. Source drivers available in syslog-ng

 

开始,先建立一个fifo文件(就是pipe啦)。。。

mkfifo logs/access.pipe

然后把nginx设置为

access_log  logs/access.pipe  main;

syslog-ng配置加上

source s_all {
        # message generated by Syslog-NG
        internal();
        # standard Linux log source (this is the default place for the syslog()
        # function to send logs to)
        unix-stream(“/dev/log”);
        # messages from the kernel
        file(“/proc/kmsg” log_prefix(“kernel: “));
        # use the following line if you want to receive remote UDP logging messages
        # (this is equivalent to the “-r” syslogd flag)
        # udp();
        pipe(“/home/nginx/logs/access.pipe”)
};

这样子就多了一个输入了,syslog-ng用file模式打开管道也可以的,不知道两者性能上面有没有差别,好像是没有什么区别的,另外一种fifo模式,就是pipe了,呵呵。

需要注意一下log_fifo_size的大小,这个是指的缓存多少行,而不是缓存多大。

如果不用pipe,用直接用access.log文件也是可以的,不过就是触发的时间太长,而且重启syslog-ng的时候可能会造成log重复。

设置真实文件的时候就需要注意多一些选项了,最重要就是follow_freq(),多久去tail一次,因为文件是没有poll的:

一般真实文件的例子
source s_tail { file("/var/log/apache/access.log" 
follow_freq(1) flags(no-parse)); };

9.1.1. Options common for every source

Some parameters affecting message parsing are common for all sources:

Name Type Default Description
flags() set of [no-parse,kernel] empty set Specifies log parsing flags. no-parse completely disables syslog message parsing and processes the complete line as the message part of a syslog message. Other information (timestamp, host, etc.) is added automatically. This flag is useful for parsing files not complying to the syslog format. kernel makes the source default to the LOG_KERN | LOG_CRIT priority if not specified otherwise.
follow_freq() number -1 Indicates that the source should be checked periodically instead of being polled. This is useful for files which always indicate readability, even though no new lines were appended. If this value is higher than zero, syslog-ng will not attempt to use poll() on the file, but checks whether the file changed every time the follow_freq() interval (in seconds) has elapsed.
keep_timestamp() yes or no yes Specifies whether syslog-ng should accept the timestamp received from the peer. If disabled, the time of reception will be used instead.
log_fetch_limit() number The value specified by the global log_fetch_limit() option, which defaults to 10. The maximum number of messages fetched from a source during a single poll loop. The destination queues might fill up before flow-control could stop reading if log_fetch_limit() is too high.
log_iw_size() number 100 The size of the initial window, this value is used during flow control.
log_msg_size() number The value specified by the global log_msg_size() option, which defaults to 8192. Specifies the maximum length of incoming log messages. Uses the value of the global option if not specified.
log_prefix() string   A string added to the beginning of every log message. It can be used to add an arbitrary string to any log source, though it is most commonly used for adding kernel: to the kernel messages on Linux.
optional() yes or no   Instruct syslog-ng to ignore the error if a specific source cannot be initialized. No other attempts to initialize the source will be made until the configuration is reloaded. This option currently applies to the pipe(), unix-dgram, and unix-stream drivers.
pad_size() number 0 Specifies input padding. Some operating systems (such as HP-UX) pad all 0 messages to block boundary. This option can be used to specify the block size. (HP-UX uses 2048 bytes). Syslog-ng will pad reads from the associated device to the number of bytes set in pad_size(). Mostly used on HP-UX where /dev/log is a named pipe and every write is padded to 2048 bytes.
time_zone() timezone in the form +/-HH:MM   The default timezone for messages read from the source. Applies only if no timezone is specified within the message itself.

Table 9.1. Common options for source drivers

 

nginx reload配置

 kill -HUP `head -1 /path/to/nginx/pid`

syslog-ng reload 配置,也一样

kill -HUP `head -1 /var/run/syslog-ng.pid`