summaryrefslogtreecommitdiff
path: root/src/journal/journal-file.c
AgeCommit message (Collapse)Author
2015-10-02journal: rework vacuuming logicLennart Poettering
Implement a maximum limit on number of journal files to keep around. Enforcing a limit is useful on this since our performance when viewing pays a heavy penalty for each journal file to interleve. This setting is turned on now by default, and set to 100. Also, actully implement what 348ced909724a1331b85d57aede80a102a00e428 promised: use whatever we find on disk at startup as lower bound on how much disk space we can use. That commit introduced some provisions to implement this, but actually never did. This also adds "journalctl --vacuum-files=" to vacuum files on disk by their number explicitly.
2015-10-02journal: improve some messagesLennart Poettering
Indicate that we are ignoring errors, when we ignore them, and log that at LOG_WARNING level. Use the right error code for the log message.
2015-10-02journal: simplify things by using the LESS_BY() macroLennart Poettering
2015-10-02journal: make journal_file_close() return NULLLennart Poettering
The way it is customary everywhere else in our sources.
2015-09-10tree-wide: never use the off_t unless glibc makes us use itLennart Poettering
off_t is a really weird type as it is usually 64bit these days (at least in sane programs), but could theoretically be 32bit. We don't support off_t as 32bit builds though, but still constantly deal with safely converting from off_t to other types and back for no point. Hence, never use the type anymore. Always use uint64_t instead. This has various benefits, including that we can expose these values directly as D-Bus properties, and also that the values parse the same in all cases.
2015-08-17Bug #944: Deletion of unnecessary checks before calls of the function "free"Markus Elfring
The function "free" is documented in the way that no action shall occur for a passed null pointer. It is therefore not needed that a function caller repeats a corresponding check. http://stackoverflow.com/questions/18775608/free-a-null-pointer-anyway-or-check-first This issue was fixed by using the software Coccinelle 1.0.1.
2015-07-24journal: avoid mapping empty data and field hash tablesLennart Poettering
When a new journal file is created we write the header first, then sync and only then create the data and field hash tables in them. That means to other processes it might appear that the files have a valid header but not data and field hash tables. Our reader code should be able to deal with this. With this change we'll not map the two hash tables right-away after opening a file for reading anymore (because that will of course fail if the objects are missing), but delay this until the first time we access them. On top of that, when we want to look something up in the hash tables and we notice they aren't initialized yet, we consider them empty. This improves handling of some journal files reported in #487.
2015-04-22journal: don't force FS_NOCOW_FL on new journal files, but warn if it is missingLennart Poettering
This way users have the freedom to set or unset the FS_NOCOW_FL flag on their journal files by setting it on the journal directory. Since our default tmpfiles configuration now sets this flag on the directory the flag is set by default on new files, however people can opt-out of this by masking the tmpfiles file for it.
2015-04-11shared: add random-util.[ch]Ronny Chevalier
2015-04-08util: merge change_attr_fd() and chattr_fd()Lennart Poettering
2015-03-27fix gcc warnings about uninitialized variablesHarald Hoyer
like: src/shared/install.c: In function ‘unit_file_lookup_state’: src/shared/install.c:1861:16: warning: ‘r’ may be used uninitialized in this function [-Wmaybe-uninitialized] return r < 0 ? r : state; ^ src/shared/install.c:1796:13: note: ‘r’ was declared here int r; ^
2015-03-09journal: fix return codeZbigniew Jędrzejewski-Szmek
Introduced in fa6ac76083b8ff. Might be related to CID #1261724, but I don't know if coverity can recurse this deep.
2015-03-09journal-file: update format string to remove castZbigniew Jędrzejewski-Szmek
2015-03-09journal: align comments to make them more legibleZbigniew Jędrzejewski-Szmek
2015-03-02journal: fix Inappropriate ioctl for device on ext4Cristian Rodríguez
Logs constantly show systemd-journald[395]: Failed to set file attributes: Inappropriate ioctl for device This is because ext4 does not support FS_NOCOW_FL. [zj: fold into one conditional as suggested on the ML and fix (preexisting) r/errno confusion in error message.]
2015-02-25journal: make skipping of exhausted journal files effective againMichal Schmidt
Commit 668c965af "journal: skipping of exhausted journal files is bad if direction changed" fixed a correctness issue, but it also significantly limited the cases where the optimization that skips exhausted journal files could apply. As a result, some journalctl queries are much slower in v219 than in v218. (e.g. queries where a "--since" cutoff should have quickly eliminated older journal files from consideration, but didn't.) If already in the initial iteration find_location_with_matches() finds no entry, the journal file's location is not updated. This is fine, except that: - We must update at least f->last_direction. The optimization relies on it. Let's separate that from journal_file_save_location() and update it immediately after the direction checks. - The optimization was conditional on "f->current_offset > 0", but it would always be 0 in this scenario. This check is unnecessary for the optimization.
2015-02-23remove unused includesThomas Hindoe Paaboel Andersen
This patch removes includes that are not used. The removals were found with include-what-you-use which checks if any of the symbols from a header is in use.
2015-02-10journald: don't specify inline in local functionsLennart Poettering
Leave it to the compiler to figure out whether it shall inline stuff or not. Only place where using static inline is OK to use is in in header files, really.
2015-01-22Fix some format strings for enums, they are signedZbigniew Jędrzejewski-Szmek
2015-01-08util: make it easy to initialize the crtime from the current time in ↵Lennart Poettering
fd_setcrtime()
2015-01-08journald: turn off COW for journal files on btrfsLennart Poettering
btrfs' COW logic results in heavily fragment journal files, which is detrimental for perfomance. Hence, turn off COW for journal files as we create them. Turning off COW comes at the cost of data integrity guarantees, but this should be acceptable, given that we do our own checksumming, and generally have a pretty conservative write pattern. Also see discussion on linux-btrfs: http://www.spinics.net/lists/linux-btrfs/msg41001.html
2015-01-06journal: consider file deletion errors a reason for rotationLennart Poettering
2015-01-06journald: whenever we rotate a file, btrfs defrag itLennart Poettering
Our write pattern is quite awful for CoW file systems (btrfs...), as we keep updating file parts in the beginning of the file. This results in fragmented journal files. Hence: when rotating files, defragment them, since at that point we know that no further write accesses will be made.
2015-01-05journald: when we detect the journal file we are about to write to has been ↵Lennart Poettering
deleted, rotate https://bugzilla.redhat.com/show_bug.cgi?id=1171719
2015-01-05journald: add some additional checks before we divide by values read from ↵Lennart Poettering
journal file headers Since the file headers might be replaced by zeroed pages now due to sigbus we should make sure we don't end up dividing by zero because we don't check values read from journal file headers for changes.
2015-01-05journald: process SIGBUS for the memory maps we set upLennart Poettering
Even though we use fallocate() it appears that file systems like btrfs will trigger SIGBUS on certain low-disk-space situation. We should handle that, hence catch the signal, add it to a list of invalidated pages, and replace the page with an empty memory area. After each write check if SIGBUS was triggered, and consider the write invalid if it was. This should make journald a lot more robust with file systems where fallocate() is not reliable, for example all CoW file systems (btrfs...), where changing written data can fail with disk full errors. https://bugzilla.redhat.com/show_bug.cgi?id=1045810
2014-12-24util: make creation time xattr logic more genericLennart Poettering
2014-12-18journal: journal_file_next_entry() does not need pointer to current ObjectMichal Schmidt
The current offset is sufficient information.
2014-12-18journal: compare candidate entries using JournalFiles' locationsMichal Schmidt
When comparing the locations of candidate entries, we can rely on the location information stored in struct JournalFile.
2014-12-18journal: keep per-JournalFile location info during iterationMichal Schmidt
In next_beyond_location() when we find a candidate entry in a journal file, save its location information in struct JournalFile. The purpose of remembering the locations of candidate entries is to be able to save work in the next iteration. This patch does only the remembering part. LOCATION_SEEK means the location identifies a candidate entry. When a winner is picked from among candidates, it becomes LOCATION_DISCRETE. LOCATION_TAIL here signifies we've iterated the file to the end (or the beginning in the case of reversed direction).
2014-12-18journal: abstract the resetting of JournalFile's locationMichal Schmidt
2014-12-18journal: delete unused function journal_file_skip_entry()Michal Schmidt
Its only caller is a test.
2014-12-18journal: delete unused function journal_file_move_to_entry_by_offset()Michal Schmidt
2014-12-13journal: replace contexts hashmap with a plain arrayMichal Schmidt
try_context() is such a hot path that the hashmap lookup is expensive. The number of contexts is small - it is the number of object types. Using a hashmap is overkill. A plain array will do. Before: $ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null real 0m9.445s user 0m9.228s sys 0m0.213s After: $ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null real 0m5.438s user 0m5.266s sys 0m0.170s
2014-12-13journal: push type_to_context conversion down to journal_file_move_to()Michal Schmidt
2014-12-13journal: have a named enum ObjectTypeMichal Schmidt
2014-12-13journal: consistently use OBJECT_<type> names instead of numbersMichal Schmidt
Note that numbers 0 and -1 are both replaced with OBJECT_UNUSED, because they are treated the same everywhere (e.g. type_to_context() translates them both to 0).
2014-12-13journal: move type_to_context() to journal-file.cMichal Schmidt
It has no other callers. It does not need to be in the header file.
2014-12-13journal: remove journal_file_object_keep/release functionsMichal Schmidt
The only user is sd_journal_enumerate_unique() and, as explained in the previous commit (fed67c38e3 "journal: map objects to context set by caller, not by actual object type"), the use of them there is now superfluous. Let's remove them. This reverts major parts of commits: ae97089d49 journal: fix access to munmapped memory in sd_journal_enumerate_unique 06cc69d44c sd-journal: fix sd_journal_enumerate_unique skipping values Tested with an "--enable-debug" build and "journalctl --list-boots". It gives the expected number of results. Additionally, if I then revert the previous commit ("journal: map objects to context set by caller, not to actual object type"), it crashes with SIGSEGV, as expected.
2014-12-13journal: map objects to context set by caller, not by actual object typeMichal Schmidt
When the caller of journal_file_move_to_object() specifies type==0, the object header is at first mapped in context 0. Then after the header is checked, the whole object is mapped in a context determined by the actual object type (which is not even range-checked using type_to_context()). This looks wrong. It should map in the caller-specified context. An old comment in sd_journal_enumerate_unique() supports this view: /* We do not use the type context here, but 0 instead, * so that we can look at this data object at the same * time as one on another file */ Clearly the expectation was that the data object will remain mapped in context 0 without being pushed away by mapping other objects in context OBJECT_DATA. I suspect that this was the real bug that got fixed by ae97089d49 "journal: fix access to munmapped memory in sd_journal_enumerate_unique". In other words, journal_file_object_keep/release are superfluous after applying this patch.
2014-12-11journald: correct spacing near eol code commentsTorstein Husebø
2014-12-08journal: Fix navigating backwards missing entriesOlivier Brunel
With DIRECTION_UP (i.e. navigating backwards) in generic_array_bisect() when the needle was found as the last item in the array, it wasn't actually processed as match, resulting in entries being missed. https://bugs.freedesktop.org/show_bug.cgi?id=86855
2014-11-28treewide: use log_*_errno whenever %m is in the format stringMichal Schmidt
If the format string contains %m, clearly errno must have a meaningful value, so we might as well use log_*_errno to have ERRNO= logged. Using: find . -name '*.[ch]' | xargs sed -r -i -e \ 's/log_(debug|info|notice|warning|error|emergency)\((".*%m.*")/log_\1_errno(errno, \2/' Plus some whitespace, linewrap, and indent adjustments.
2014-10-27journald: be nice to coverity, add an extra assertLennart Poettering
coverity otherwise assumes that the chain object might be NULL.
2014-10-23journal: make JournalFile::chain_cache an OrderedHashmapMichal Schmidt
The order of entries may matter here. Oldest entries are evicted first when the cache is full. (Though I don't see anything to rejuvenate entries on cache hits.)
2014-10-09sd-journal: fix sd_journal_enumerate_unique skipping valuesJan Janssen
sd_journal_enumerate_unique will lock its mmap window to prevent it from being released by calling mmap_cache_get with keep_always=true. This call may return windows that are wider, but compatible with the parameters provided to it. This can result in a mismatch where the window to be released cannot properly be selected, because we have more than one window matching the parameters of mmap_cache_release. Therefore, introduce a release_cookie to be used when releasing the window. https://bugs.freedesktop.org/show_bug.cgi?id=79380
2014-09-15hashmap: introduce hash_ops to make struct Hashmap smallerMichal Schmidt
It is redundant to store 'hash' and 'compare' function pointers in struct Hashmap separately. The functions always comprise a pair. Store a single pointer to struct hash_ops instead. systemd keeps hundreds of hashmaps, so this saves a little bit of memory.
2014-08-27sd-journal: verify that object start with the field nameZbigniew Jędrzejewski-Szmek
If the journal is corrupted, we might return an object that does not start with the expected field name and/or is shorter than it should.
2014-08-03Fix misuse of uint64_t as size_tZbigniew Jędrzejewski-Szmek
They have different size on 32 bit, so they are really not interchangable.
2014-07-15journal: allow files with no data whatsoeverZbigniew Jędrzejewski-Szmek
If a file was opened for writing, and then closed immediately without actually writing any entries, on subsequent opening, it would be considered "corrupted". This should be totally fine, and even in read mode, an empty file can become non-empty later on.