summaryrefslogtreecommitdiff
path: root/src/journal/journal-file.h
AgeCommit message (Collapse)Author
2014-07-06journal: add LZ4 as optional compressorZbigniew Jędrzejewski-Szmek
Add liblz4 as an optional dependency when requested with --enable-lz4, and use it in preference to liblzma for journal blob and coredump compression. To retain backwards compatibility, XZ is used to decompress old blobs. Things will function correctly only with lz4-119. Based on the benchmarks found on the web, lz4 seems to be the best choice for "quick" compressors atm. For pkg-config status, see http://code.google.com/p/lz4/issues/detail?id=135.
2014-01-11journald: do not free space when disk space runs lowZbigniew Jędrzejewski-Szmek
Before, journald would remove journal files until both MaxUse= and KeepFree= settings would be satisfied. The first one depends (if set automatically) on the size of the file system and is constant. But the second one depends on current use of the file system, and a spike in disk usage would cause journald to delete journal files, trying to reach usage which would leave 15% of the disk free. This behaviour is surprising for the user who doesn't expect his logs to be purged when disk usage goes above 85%, which on a large disk could be some gigabytes from being full. In addition attempting to keep 15% free provides an attack vector where filling the disk sufficiently disposes of almost all logs. Instead, obey KeepFree= only as a limit on adding additional files. When replacing old files with new, ignore KeepFree=. This means that if journal disk usage reached some high point that at some later point start to violate the KeepFree= constraint, journald will not add files to go above this point, but it will stay (slightly) below it. When journald is restarted, it forgets the previous maximum usage value, and sets the limit based on the current usage, so if disk remains to be filled, journald might use one journal-file-size less on each restart, if restarts happen just after rotation. This seems like a reasonable compromise between implementation complexity and robustness.
2014-01-11journal: fix access to munmapped memory in sd_journal_enumerate_uniqueZbigniew Jędrzejewski-Szmek
sd_j_e_u needs to keep a reference to an object while comparing it with possibly duplicate objects in other files. Because the size of mmap cache is limited, with enough files and object to compare to, at some point the object being compared would be munmapped, resulting in a segmentation fault. Fix this issue by turning keep_always into a reference count that can be increased and decreased. Other callers which set keep_always=true are unmodified: their references are never released but are ignored when the whole file is closed, which happens at some point. keep_always is increased in sd_j_e_u and later on released.
2013-11-26journal: make table constLennart Poettering
2013-11-08Remove dead code and unexport some callsLennart Poettering
"make check-api-unused" informs us about code that is not used anymore or that is exported but only used internally. Fix these all over the place.
2013-06-10journal: remember last direction of search and keep offset cacheZbigniew Jędrzejewski-Szmek
The fields in JournalFile are moved around to avoid wasting 7 bytes because of alignment.
2013-05-02Add __attribute__((const, pure, format)) in various placesZbigniew Jędrzejewski-Szmek
I'm assuming that it's fine if a _const_ or _pure_ function calls assert. It is assumed that the assert won't trigger, and even if it does, it can only trigger on the first call with a given set of parameters, and we don't care if the compiler moves the order of calls.
2013-03-25journal: Add sync timer to journal serverOleksii Shevchuk
Add option to force journal sync with fsync. Default timeout is 5min. Interval configured via SyncIntervalSec option at journal.conf. Synced journal files will be marked as OFFLINE. Manual sync can be performed via sending SIGUSR1.
2012-10-26journal: introduce entry array chain cacheLennart Poettering
When traversing entry array chains for a bisection or for retrieving an item by index we previously always started at the beginning of the chain. Since we tend to look at the same chains repeatedly, let's cache where we have been the last time, and maybe we can skip ahead with this the next time. This turns most bisections and index lookups from O(log(n)*log(n)) into O(log(n)). More importantly however, we seek around on disk much less, which is good to reduce buffer cache and seek times on rotational disks.
2012-10-18journal: add ability to list values a specified field can take in all ↵Lennart Poettering
entries of the journal The new 'unique' API allows listing all unique field values that a field specified by a field name can take in all entries of the journal. This allows answering queries such as "What units logged to the journal?", "What hosts have logged into the journal?", "Which boot IDs have logged into the journal?". Ultimately this allows implementation of tools similar to lastlog based on journal data. Note that listing these field values will not work for journal files created with older journald, as the field values are not indexed in older files.
2012-10-16journal: implement time-based rotation/vacuumingLennart Poettering
This also enables time-based rotation (but not vacuuming) after 1month, so that not more one month of journal is lost at a time per vacuuming.
2012-08-20journal: use a macro to check for file header flagsLennart Poettering
2012-08-19journal: validate timestamps as wellLennart Poettering
2012-08-18journal: even more simple static object testsLennart Poettering
2012-08-18journald: add additional simple static tests to verifierLennart Poettering
2012-08-17journal: rework terminologyLennart Poettering
Let's clean up our terminology a bit. New terminology: FSS = Forward Secure Sealing FSPRG = Forward Secure Pseudo-Random Generator FSS is the combination of FSPRG and a HMAC. Sealing = process of adding authentication tags to the journal. Verification = process of checking authentication tags to the journal. Sealing Key = The key used for adding authentication tags to the journal. Verification Key = The key used for checking authentication tags of the journal. Key pair = The pair of Sealing Key and Verification Key Internally, the Sealing Key is the combination of the FSPRG State plus change interval/start time. Internally, the Verification Key is the combination of the FSPRG Seed plus change interval/start time.
2012-08-16journal: parse fsprg seedLennart Poettering
2012-08-16journal: rename 'mmap' to 'mmap_cache' to appease gccZbigniew Jędrzejewski-Szmek
warning: declaration of 'mmap' shadows a global declaration [-Wshadow]
2012-08-16journal: split up journal-file.cLennart Poettering
2012-08-16journal: implement basic journal file verification logicLennart Poettering
2012-08-16journal: implement generic sharable mmap caching logicLennart Poettering
instead of having one simple per-file cache implement an more comprehensive one that works for multiple files and can actually maintain multiple maps per file and per object type.
2012-08-13journal: add all objects we add to HMACLennart Poettering
2012-08-13journald: initial version of FSPRG hookupLennart Poettering
This adds forward-secure authentication of journal files. This patch includes key generation as well as tagging of journal files, Verification of journal files will be added in a later patch.
2012-07-19use #pragma once instead of foo*foo #define guardsShawn Landden
#pragma once has been "un-deprecated" in gcc since 3.3, and is widely supported in other compilers. I've been using and maintaining (rebasing) this patch for a while now, as it annoyed me to see #ifndef fooblahfoo, etc all over the place, almost arrogant about the annoyance of having to define all these names to perform a commen but neccicary functionality, when a completely superior alternative exists. I havn't sent it till now, cause its kindof a style change, and it is bad voodoo to mess with style that has been established by more established editors. So feel free to lambast me as a crazy bafoon. v2 - preserve externally used headers
2012-07-17journal: size journal data hash table based on maximum file size metricsLennart Poettering
The default of 2047 hash table entries turned out to result in way too many collisions for bigger files, hence scale the hash table size by the estimated maximum file size.
2012-07-17journal: automatically rotate journal files if the data hash table is full > 75%Lennart Poettering
Previously, when the main data hash table grows too full the performance simply started to decrease drastically. Instead, now simply rotate to a new journal file as the hash table gets to full, so that we can start with a new fresh empty hash table.
2012-07-13journal: beef up journal matches considerablyLennart Poettering
we now can take multiple matches, and they will apply as AND if they apply to different fields and OR if they apply to the same fields. Also, terms of this kind can be combined with an overreaching OR.
2012-06-17journal: expose and make use of cutoff times of journalLennart Poettering
This helps explaining when the log output of "systemctl status" is incomplete because the logs got rotated since the service was started.
2012-04-12relicense to LGPLv2.1 (with exceptions)Lennart Poettering
We finally got the OK from all contributors with non-trivial commits to relicense systemd from GPL2+ to LGPL2.1+. Some udev bits continue to be GPL2+ for now, but we are looking into relicensing them too, to allow free copy/paste of all code within systemd. The bits that used to be MIT continue to be MIT. The big benefit of the relicensing is that closed source code may now link against libsystemd-login.so and friends.
2012-03-22add sparse support to detect endianness bugFrederic Crozat
le16/32/64_t type should be used when storing little-endian value header to integrate with sparse from Josh Triplett <josh@joshtriplett.org>
2012-03-15journal: if we encounter a corrupted file, rotate and go onLennart Poettering
2012-01-11journal: fix more 32/64 bit issuesLennart Poettering
2012-01-05build-sys: move public header files into a dir of their ownLennart Poettering
2011-12-31journal: automatically deduce journal metrics from file system sizesLennart Poettering
2011-12-31journal: move max_use into metrics structureLennart Poettering
2011-12-29journald: flush /run to /var as soon as it becomes availableLennart Poettering
2011-12-21journal: add inline compression support with XZLennart Poettering
2011-12-20journal: fix space reservation limit enforcementLennart Poettering
2011-11-08journal: implement multiple field matchesLennart Poettering
2011-10-15journal: unify code for up and for down traversalLennart Poettering
2011-10-14journal: move field index from file into journal objectLennart Poettering
2011-10-14journal: synchronize seqnum across filesLennart Poettering
2011-10-14journal: implement parallel file traversalLennart Poettering
2011-10-13journal: implementation rotationLennart Poettering
2011-10-08journal: implement parallel traversal in clientLennart Poettering