summaryrefslogtreecommitdiff
path: root/src/journal
AgeCommit message (Collapse)Author
2014-07-16Let config_parse open file where applicableZbigniew Jędrzejewski-Szmek
Special care is needed so that we get an error message if the file failed to parse, but not when it is missing. To avoid duplicating the same error check in every caller, add an additional 'warn' boolean to tell config_parse whether a message should be issued. This makes things both shorter and more robust wrt. to error reporting.
2014-07-16test-compress-benchmark: add missing % before PRIu64 formatThomas Hindoe Paaboel Andersen
2014-07-15test-compress-benchmark: silence warningsMarc-Antoine Perennou
and btw make it pass for 32bits where size_t != uint64_t
2014-07-15Clear up confusion wrt. ENTRY_SIZE_MAX and DATA_SIZE_MAXZbigniew Jędrzejewski-Szmek
Define DATA_SIZE_MAX to mean the maximum size of a single field, and ENTRY_SIZE_MAX to mean the size of the whole entry, with some rough calculation of overhead over the payload. Check if entries are not too big when processing native journal messages.
2014-07-15Constify ConfigTableItem tablesZbigniew Jędrzejewski-Szmek
2014-07-15journal-upload: use journal as the sourceZbigniew Jędrzejewski-Szmek
2014-07-15journal-remote: reject fields above maximum sizeZbigniew Jędrzejewski-Szmek
Also fix an infinite loop on E2BIG. Remember what range we already scanned for '\n', to avoid quadratic behaviour on long "text" fields.
2014-07-15Move network-related journal programs to src/journal-remote/Zbigniew Jędrzejewski-Szmek
Directory src/journal has become one of the largest directories, and since systemd-journal-gatewayd, systemd-journal-remote, and forthcoming systemd-journal-upload are all closely related, create a separate directory for them.
2014-07-15microhttp-util: rework gnutls loggingZbigniew Jędrzejewski-Szmek
2014-07-15journal: allow files with no data whatsoeverZbigniew Jędrzejewski-Szmek
If a file was opened for writing, and then closed immediately without actually writing any entries, on subsequent opening, it would be considered "corrupted". This should be totally fine, and even in read mode, an empty file can become non-empty later on.
2014-07-11Fix build without any compression enabledZbigniew Jędrzejewski-Szmek
2014-07-11journald: turn ForwardToSyslog= off by defaultLennart Poettering
After all, rsyslog and friends nowadays read their data directly from the journal, hence the forwarding is unnecessary in most cases.
2014-07-08journal/compress: improve xz compression performanceJon Severinsson
The new lzma2 compression options at the top of compress_blob_xz are equivalent to using preset "0", exept for using a 1 MiB dictionary (the same as preset "1"). This makes the memory usage at most 7.5 MiB in the compressor, and 1 MiB in the decompressor, instead of the previous 92 MiB in the compressor and 8 MiB in the decompressor. According to test-compress-benchmark this commit makes XZ compression 20 times faster, with no increase in compressed data size. Using more realistic test data (an ELF binary rather than repeating ASCII letters 'a' through 'z' in order) it only provides a factor 10 speedup, and at a cost if a 10% increase in compressed data size. But that is still a worthwhile trade-off. According to test-compress-benchmark XZ compression is still 25 times slower than LZ4, but the compressed data is one eighth the size. Using more realistic test data XZ compression is only 18 times slower than LZ4, and the compressed data is only one quarter the size. $ ./test-compress-benchmark XZ: compressed & decompressed 2535300963 bytes in 42.30s (57.15MiB/s), mean compresion 99.95%, skipped 3570 bytes LZ4: compressed & decompressed 2535303543 bytes in 1.60s (1510.60MiB/s), mean compresion 99.60%, skipped 990 bytes
2014-07-08fix #ifdefRonny Chevalier
2014-07-07journal/compress: fix calls to decompress_blobZbigniew Jędrzejewski-Szmek
2014-07-06coredumpctl: show a useful error on permission problemsZbigniew Jędrzejewski-Szmek
2014-07-06compress: add benchmark-style testZbigniew Jędrzejewski-Szmek
This is useful to test the behaviour of the compressor for various buffer sizes. Time is limited to a minute per compression, since otherwise, when LZ4 takes more than a second which is necessary to reduce the noise, XZ takes more than 10 minutes. % build/test-compress-benchmark (without time limit) XZ: compressed & decompressed 2535300963 bytes in 794.57s (3.04MiB/s), mean compresion 99.95%, skipped 3570 bytes LZ4: compressed & decompressed 2535303543 bytes in 1.56s (1550.07MiB/s), mean compresion 99.60%, skipped 990 bytes % build/test-compress-benchmark (with time limit) XZ: compressed & decompressed 174321481 bytes in 60.02s (2.77MiB/s), mean compresion 99.76%, skipped 3570 bytes LZ4: compressed & decompressed 2535303543 bytes in 1.63s (1480.83MiB/s), mean compresion 99.60%, skipped 990 bytes It appears that there's a bug in lzma_end where it leaks 32 bytes.
2014-07-06journal: add LZ4 as optional compressorZbigniew Jędrzejewski-Szmek
Add liblz4 as an optional dependency when requested with --enable-lz4, and use it in preference to liblzma for journal blob and coredump compression. To retain backwards compatibility, XZ is used to decompress old blobs. Things will function correctly only with lz4-119. Based on the benchmarks found on the web, lz4 seems to be the best choice for "quick" compressors atm. For pkg-config status, see http://code.google.com/p/lz4/issues/detail?id=135.
2014-07-06journal/compress: return early in uncompress_startswithZbigniew Jędrzejewski-Szmek
uncompress_startswith would always decode the whole stream, even if it did not start with the given prefix. Reallocation policy was also strange.
2014-07-01coredump: vacuum - fix calculation of 10% of fs size for MaxUseThomas Hindoe Paaboel Andersen
2014-06-27journald: make MaxFileSec really default to 1monthMichał Bartoszkiewicz
journald.conf(5) states that the default for MaxFileSec is one month, but the code didn't respect that.
2014-06-27coredump: simplify compression logic a bitLennart Poettering
This also make sure we remove the original coredump temporary file if we successfully managed to compress the coredump.
2014-06-27coredump: replace Compression= setting by simpler Compress= boolean settingLennart Poettering
Let's move things closer to journald's configuration settings, which knows Compress= already, as a boolean. This makes things more uniform, but also gives us more freedom to possibly swap out the used compression algorithm one day.
2014-06-27coredump: don't expose the compression level as configuration optionLennart Poettering
This sounds overly low-level and implementation-detaily. Let's just use the default level XZ suggests. This gives us more room to possibly swap out the compression algorithm used, as the compression level range will not leak into user configuration.
2014-06-27journald: invoking fstatvfs() is now redundant in the vacuuming codeLennart Poettering
2014-06-27coredump: don't be annoyed if another coredump hook removes our coredump ↵Lennart Poettering
while we work on it
2014-06-27coredump: fix how the compression level is verifiedLennart Poettering
2014-06-27coredump: add simple coredump vacuumingLennart Poettering
When disk space taken up by coredumps grows beyond a configured limit start removing the oldest coredump of the user with the most coredumps, until we get below the limit again.
2014-06-27coredumpctl: fix potential deref of null pointerThomas Hindoe Paaboel Andersen
2014-06-27coredump: make sure variable is set if uncompressedThomas Hindoe Paaboel Andersen
reorder the code so the fstat is done before we can jump to uncompressed
2014-06-26coredumpctl: remove unused variableThomas Hindoe Paaboel Andersen
2014-06-26coredump: fix debug messageThomas Hindoe Paaboel Andersen
typo from 347272731e15d3c4a70fad7ccd7185e8e8059d01
2014-06-26coredumpctl: fix HAVE_XZ checkDaniel Mack
2014-06-26coredumpctl: fix build with !HAVE_XZDaniel Mack
Fixes the following build error: CCLD coredumpctl src/journal/coredumpctl.o: In function `save_core': /src/systemd-master/src/journal/coredumpctl.c:656: undefined reference to `decompress_stream' collect2: error: ld returned 1 exit status make[2]: *** [coredumpctl] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2
2014-06-26man: add coredump.conf(5)Zbigniew Jędrzejewski-Szmek
2014-06-26coredump+coredumpctl: add COREDUMP_FILENAME, use in coredumpctlZbigniew Jędrzejewski-Szmek
2014-06-26coredump: make compression configurableZbigniew Jędrzejewski-Szmek
Add Compression={none,xz} and CompressionLevel=0-9 settings. Defaults are xz/6. Compression=filesystem may be added later. I picked "xz" for the compression "type", since we might want to add different compressors later on. XZ is fairly memory and CPU intensive, and embedded users will likely want to use LZO or some other lightweight compression mechanism.
2014-06-26coredump: compress core filesZbigniew Jędrzejewski-Szmek
Unfortunately the core is first written uncompressed, then compressed by reading from disk and writing to the output file. This is ugly and slow, but I don't see a way around, if we want to get the backtrace without keeping everything in memory.
2014-06-26journal/compress: add stream compression/decompression functionsZbigniew Jędrzejewski-Szmek
2014-06-26journal/compress: simplify compress_blobZbigniew Jędrzejewski-Szmek
2014-06-25tests: add test-compressRonny Chevalier
2014-06-23coredump: log coredump even if core dumping failsZbigniew Jędrzejewski-Szmek
Journal might be functional even if we cannot write to /var/lib/systemd/coredump.
2014-06-23coredump: retrieve comm information from /procZbigniew Jędrzejewski-Szmek
Kernel mangles comm information in an irreversible way when comm constains repeated spaces. Retrieve comm information from /proc, and only fallback to the information provided on the commandline when retrieving information from /proc fails. Add exe information to the list of saved xattr. https://bugs.freedesktop.org/show_bug.cgi?id=62043
2014-06-23coredump: never write more than the configured processing size limit to diskLennart Poettering
2014-06-23coredumpctl: show comm name next to PIDLennart Poettering
2014-06-23coredump: quit early if we cannot store ay coredump to diskLennart Poettering
2014-06-23coredumpctl: include timestamp information in "coredumpctl info" outputLennart Poettering
2014-06-23coredump: also escape spaces in comm fields when formatting file namesLennart Poettering
2014-06-22consistently order cleanup attribute before typeThomas Hindoe Paaboel Andersen
2014-06-20coredump: fix include of xattr.hFilipe Brandenburger
The correct path is now <sys/xattr.h> (from glibc-headers) and no longer <attr/xattr.h> (from libattr-devel.) Fixes: 34c10968cbe3b5591b3c0ce225b8694edd9709d0