#log #logger #sqlite #binn

bin+lib ice-age

Simple logging kit for server programs

2 releases

0.2.1 Sep 15, 2018
0.2.0 Sep 14, 2018

#47 in Caching

Download history 22/week @ 2018-09-17

7 downloads per month

Custom license

30KB
515 lines

Ice Age




lib.rs:

Simple logging kit

Project


Design

It uses synchronous channels for communication. Log records are stored in RAM, and will be flushed to disk based on some configurable conditions: a period of time, or when maximum number of records reached.

Backends: either SQLite or Binn format. There's binary kit which can port .ice-age files to .sqlite files.

The crate's own log messages are prefixed with TAG.

Usage

Default features use SQLite via rusqlite crate.

Currently, flushing log records to SQLite database is via a prepared cached statement, inside a transaction. However, from my experiences, Binn backend is often twice as fast. I don't keep the number for SQLite backend, but with Binn backend, some numbers are:

Records Time to flush
1879 3.220934ms
1839 4.975739ms
1932 3.313327ms
1873 5.222861ms
1732 4.771375ms
1957 5.312771ms
2017 3.550235ms
3226 5.561279ms
3137 5.451346ms
2816 8.097975ms

To use Binn backend, you have to turn default features off:

[dependencies]
ice-age = { version = "x.y.z", default-features = false, features = ["with-binn-ir"] }

Examples

use std::{
    path::PathBuf,
    sync::mpsc::TrySendError,
    thread,
    time::{UNIX_EPOCH, Duration, SystemTime},
};
use ice_age::{Config, Cmd, Log, Logger};

let config = Config {
    // Directory to save log files
    work_dir: PathBuf::from("/tmp/"),
    // For this example, max file length is 1 MiB
    max_file_len: 1024 * 1024,
    // Keep log files at most 3 days
    log_files_reserved: Duration::from_secs(3 * 24 * 60 * 60),
    // Maximum log records to be kept in RAM
    buf_len: 5_000,
    // Flush to disk every 30 minutes
    disk_flush_interval: Duration::from_secs(30 * 60),
};

let logger = Logger::make(config).unwrap();
for _ in 0..3 {
    let logger = logger.clone();
    thread::spawn(move || {
        let log = Log {
            time: SystemTime::now().duration_since(UNIX_EPOCH).unwrap().as_secs(),
            remote_ip: String::from("127.0.0.1"),
            url: String::from("/api/statistics"),
            response_size: Some(512),
            code: 200,
            runtime: Duration::from_secs(1),
            notes: None,
        };

        // Use ::try_send() to not block the thread.
        // This example's strategy is to discard failed calls.
        match logger.try_send(Cmd::StoreLog(log)) {
            Ok(()) => (),
            Err(TrySendError::Full(_)) =>
                eprintln!("Log buffer is full, discarding..."),
            Err(TrySendError::Disconnected(_)) =>
                eprintln!("Failed to store log. Perhaps log server is down."),
        };
    });
}

Dependencies

~9.5MB
~171K SLoC