summaryrefslogtreecommitdiff
path: root/src/journal
AgeCommit message (Collapse)Author
2015-01-05journald: process SIGBUS for the memory maps we set upLennart Poettering
Even though we use fallocate() it appears that file systems like btrfs will trigger SIGBUS on certain low-disk-space situation. We should handle that, hence catch the signal, add it to a list of invalidated pages, and replace the page with an empty memory area. After each write check if SIGBUS was triggered, and consider the write invalid if it was. This should make journald a lot more robust with file systems where fallocate() is not reliable, for example all CoW file systems (btrfs...), where changing written data can fail with disk full errors. https://bugzilla.redhat.com/show_bug.cgi?id=1045810
2014-12-30tree-wide: spelling fixesVeres Lajos
https://github.com/vlajos/misspell_fixer https://github.com/torstehu/systemd/commit/b6fdeb618cf2f3ce1645b3315f15f482710c7ffa Thanks to Torstein Husebo <torstein@huseboe.net>.
2014-12-26journald: always allocate space for object fieldsZbigniew Jędrzejewski-Szmek
If OBJECT_PID= came as the last field, we would not reallocate the iovec to bigger size, and fail the assertion later on in dispatch_message_real().
2014-12-26journald: fix off by one in native transportZbigniew Jędrzejewski-Szmek
https://bugzilla.redhat.com/show_bug.cgi?id=1177184
2014-12-24util: make creation time xattr logic more genericLennart Poettering
2014-12-24util: fix strict aliasing violations in use of struct inotify_event v5Shawn Paul Landden
There is alot of cleanup that will have to happen to turn on -fstrict-aliasing, but I think our code should be "correct" to the rule.
2014-12-19journal: skipping of exhausted journal files is bad if direction changedMichal Schmidt
EOF is meaningless if the direction of iteration changes. Move the EOF optimization under the direction check. This fixes test-journal-interleaving for me. Thanks to Filipe Brandenburger for telling me about the failure.
2014-12-19journal: make next_with_matches() always use f->current_offsetMichal Schmidt
next_with_matches() is odd in that its "unit64_t *offset" parameter is both input and output. In other it's purely for output. The function is called from two places in next_beyond_location(). In both of them "&cp" is used as the argument and in both cases cp is guaranteed to equal f->current_offset. Let's just have next_with_matches() ignore "*offset" on input and operate with f->current_offset. I did not investigate why it is, but it makes my usual benchmark run reproducibly faster: $ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null real 0m4.032s user 0m3.896s sys 0m0.135s (Compare to preceding commit, where real was 4.4s.)
2014-12-19journal: fix skipping of duplicate entries in iterationMichal Schmidt
I accidentally broke the detection of duplicate entries in 7943f42275 "journal: optimize iteration by returning previously found candidate entry". When we have a known location of a candidate entry, we must not return from next_beyond_location() immediately. We must go through the duplicates detection to make sure the candidate differs from the already iterated entry. This fix slows down iteration a bit, but it's still faster than it was before the rework. $ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null real 0m4.448s user 0m4.298s sys 0m0.149s (Compare with results from commit 7943f42275, where real was 5.3s before the rework.)
2014-12-18journal: next_with_matches() now does not need a mapped object as inputMichal Schmidt
Now that journal_file_next_entry() does not need a pointer to the current object, next_with_matches() does not need it either.
2014-12-18journal: journal_file_next_entry() does not need pointer to current ObjectMichal Schmidt
The current offset is sufficient information.
2014-12-18journal: optimize iteration by returning previously found candidate entryMichal Schmidt
In next_beyond_location() when the JournalFile's location type is LOCATION_SEEK, it means there's nothing to do, because we already have the location of the candidate entry. Do an early return. Note that now next_beyond_location() does not anymore guarantee on return that the entry is mapped, but previous patches made sure the caller does not care. This optimization is at least as good as "journal: optimize iteration: skip files that cannot improve current candidate entry" was. Timing results on my workstation, using: $ time ./journalctl -q --since=2014-06-01 --until=2014-07-01 > /dev/null Before "Revert "journal: optimize iteration: skip files that cannot improve current candidate entry": real 0m5.349s user 0m5.166s sys 0m0.181s Now: real 0m3.901s user 0m3.724s sys 0m0.176s
2014-12-18journal: optimize iteration by skipping exhausted filesMichal Schmidt
If from a previous iteration we know we are at the end of a journal file, don't bother looking into the file again. This is complicated by the fact that the EOF does not have to be permanent (think of "journalctl -f"). So we also check if the number of entries in the journal file changed. This optimization has a similar effect as "journal: optimize iteration: skip whole files behind current location" had.
2014-12-18journal: drop unnecessary parameters of next_beyond_location()Michal Schmidt
offset is redundant, because the caller can rely on f->current_offset. The object pointer the function saves in *ret is thrown away by the caller.
2014-12-18journal: remove redundant variable new_offsetMichal Schmidt
The file's current_offset is already updated at this point, so let's use it.
2014-12-18journal: compare candidate entries using JournalFiles' locationsMichal Schmidt
When comparing the locations of candidate entries, we can rely on the location information stored in struct JournalFile.
2014-12-18journal: simplify set_location()Michal Schmidt
set_location() is called from real_journal_next() when a winning entry has been picked from among the candidates in journal files. The location type is always set to LOCATION_DISCRETE. No need to pass it as a parameter. The per-JournalFile location information is already updated at this point. No need for having the direction and offset here.
2014-12-18journal: keep per-JournalFile location info during iterationMichal Schmidt
In next_beyond_location() when we find a candidate entry in a journal file, save its location information in struct JournalFile. The purpose of remembering the locations of candidate entries is to be able to save work in the next iteration. This patch does only the remembering part. LOCATION_SEEK means the location identifies a candidate entry. When a winner is picked from among candidates, it becomes LOCATION_DISCRETE. LOCATION_TAIL here signifies we've iterated the file to the end (or the beginning in the case of reversed direction).
2014-12-18journal: abstract the resetting of JournalFile's locationMichal Schmidt
2014-12-18journal: move definition of LocationType to journal-file.hMichal Schmidt
In preparation for individual JournalFiles maintaining a location of their own.
2014-12-18Revert "journal: optimize iteration: skip whole files behind current location"Michal Schmidt
This reverts commit b7c88ab8cc7d55a43450bf3dea750f95f2e910d6. This optimization will be made redundant by the following patches.
2014-12-18Revert "journal: optimize iteration: skip files that cannot improve current ↵Michal Schmidt
candidate entry" This reverts commit f8b5a3b75fb55f0acb85c21424b3893c822742e9. This optimization will be made redundant by the following patches.
2014-12-18journal: delete unused function journal_file_skip_entry()Michal Schmidt
Its only caller is a test.
2014-12-18journal: delete unused function journal_file_move_to_entry_by_offset()Michal Schmidt
2014-12-13journal: replace contexts hashmap with a plain arrayMichal Schmidt
try_context() is such a hot path that the hashmap lookup is expensive. The number of contexts is small - it is the number of object types. Using a hashmap is overkill. A plain array will do. Before: $ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null real 0m9.445s user 0m9.228s sys 0m0.213s After: $ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null real 0m5.438s user 0m5.266s sys 0m0.170s
2014-12-13journal: delete unused function mmap_cache_close_contextMichal Schmidt
This never had any callers. Contexts are freed when the MMapCache is freed.
2014-12-13journal: push type_to_context conversion down to journal_file_move_to()Michal Schmidt
2014-12-13journal: have a named enum ObjectTypeMichal Schmidt
2014-12-13journal: consistently use OBJECT_<type> names instead of numbersMichal Schmidt
Note that numbers 0 and -1 are both replaced with OBJECT_UNUSED, because they are treated the same everywhere (e.g. type_to_context() translates them both to 0).
2014-12-13journal: consistently allow type==0 to mean "any type"Michal Schmidt
If type==0 and a non-NULL object were given as arguments to journal_file_hmac_put_object(), its object type check would fail and it would return -EBADMSG. All existing callers use either a positive type or -1. Still, for behavior consistency with journal_file_move_to_object() let's allow type 0 to pass.
2014-12-13journal: move type_to_context() to journal-file.cMichal Schmidt
It has no other callers. It does not need to be in the header file.
2014-12-13journal: remove journal_file_object_keep/release functionsMichal Schmidt
The only user is sd_journal_enumerate_unique() and, as explained in the previous commit (fed67c38e3 "journal: map objects to context set by caller, not by actual object type"), the use of them there is now superfluous. Let's remove them. This reverts major parts of commits: ae97089d49 journal: fix access to munmapped memory in sd_journal_enumerate_unique 06cc69d44c sd-journal: fix sd_journal_enumerate_unique skipping values Tested with an "--enable-debug" build and "journalctl --list-boots". It gives the expected number of results. Additionally, if I then revert the previous commit ("journal: map objects to context set by caller, not to actual object type"), it crashes with SIGSEGV, as expected.
2014-12-13journal: map objects to context set by caller, not by actual object typeMichal Schmidt
When the caller of journal_file_move_to_object() specifies type==0, the object header is at first mapped in context 0. Then after the header is checked, the whole object is mapped in a context determined by the actual object type (which is not even range-checked using type_to_context()). This looks wrong. It should map in the caller-specified context. An old comment in sd_journal_enumerate_unique() supports this view: /* We do not use the type context here, but 0 instead, * so that we can look at this data object at the same * time as one on another file */ Clearly the expectation was that the data object will remain mapped in context 0 without being pushed away by mapping other objects in context OBJECT_DATA. I suspect that this was the real bug that got fixed by ae97089d49 "journal: fix access to munmapped memory in sd_journal_enumerate_unique". In other words, journal_file_object_keep/release are superfluous after applying this patch.
2014-12-13journal: add debug mode for mmap-cache (--enable-debug=mmap-cache)Michal Schmidt
This is useful for exposing unsafe access to mmapped objects after the context that they were mapped in was already moved. For example: journal_file_move_to_object(f1, OBJECT_DATA, p1, &o1); journal_file_move_to_object(f2, OBJECT_DATA, p2, &o2); t = o1->object.type; /* this usually works, but is unsafe */
2014-12-12copy: use btrfs reflinking only whe we know we copy full filesLennart Poettering
2014-12-12util: when using basename() for creating temporary files, verify the ↵Lennart Poettering
resulting name is actually valid Also, rename filename_is_safe() to filename_is_valid(), since it actually does a full validation for what the kernel will accept as file name, it's not just a heuristic.
2014-12-11journald: correct spacing near eol code commentsTorstein Husebø
2014-12-09treewide: sanitize loop_writeZbigniew Jędrzejewski-Szmek
loop_write() didn't follow the usual systemd rules and returned status partially in errno and required extensive checks from callers. Some of the callers dealt with this properly, but many did not, treating partial writes as successful. Simplify things by conforming to usual rules.
2014-12-10core: unify how we iterate over inotify eventsLennart Poettering
Let's add some syntactic sugar for iterating through inotify events, and use it everywhere.
2014-12-09journal: optimize iteration: skip files that cannot improve current ↵Michal Schmidt
candidate entry Suppose that while iterating we have already looked into a journal file and got a candidate for the next entry. And we are considering to look into another journal file because it may contain an entry that is nearer to the current location than the candidate. We should skip the whole journal file if we can tell by looking at its header that none of its entries can precede the candidate. Before: $ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null real 0m20.518s user 0m19.989s sys 0m0.328s After: $ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null real 0m9.445s user 0m9.228s sys 0m0.213s
2014-12-09journal: optimize iteration: skip whole files behind current locationMichal Schmidt
Interleaving of entries from many journal files is expensive. But there is room for optimization. We can skip looking into journal files whose entries all lie before the current iterating location. We can tell if that's the case from looking at the journal file header. This saves a huge amount of work if one has many of mostly not interleaved journal files. On my workstation with 90 journal files in /var/log/journal/ID/ totalling 3.4 GB I get these results: Before: $ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null real 5m54.258s user 2m4.263s sys 3m48.965s After: $ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null real 0m20.518s user 0m19.989s sys 0m0.328s The high "sys" time in the original was caused by putting more stress on the mmap-cache than it could handle. With the patch the working set now consists of fewer mmap windows and mmap-cache is not thrashing.
2014-12-09journalctl: respect --after-cursor semantics with --follow in all casesWesley Dawson
In the case where no entries have been added to the journal after the specified cursor, set need_seek before the main loop to prevent display of the entry at said cursor.
2014-12-08journal: Fix navigating backwards missing entriesOlivier Brunel
With DIRECTION_UP (i.e. navigating backwards) in generic_array_bisect() when the needle was found as the last item in the array, it wasn't actually processed as match, resulting in entries being missed. https://bugs.freedesktop.org/show_bug.cgi?id=86855
2014-11-30journald: close passed fds we cannot make sense ofLennart Poettering
This is mostly likely the audit socket, and we really should close it if we cannot make sense of it, since as long as it is open the kernel might disable the kmsg forwarding of audit msgs, and we should avoid that, since audit msgs might get completely lost then. I also downgraded the log message we show a bit, after all things should really work fine, and we proceed fine with it.
2014-11-30tests: use assert_se instead of assertRonny Chevalier
Otherwise they can be optimized away with -DNDEBUG
2014-11-29coredump: Support coredump.conf.d directories in the usual search pathsJosh Triplett
2014-11-29journald: Support journald.conf.d directories in the usual search pathsJosh Triplett
2014-11-28coredump: simplify a few things by allocating small fields on the stack ↵Lennart Poettering
rather than heap
2014-11-28coredump: rework compose_open_fds()Lennart Poettering
Use FOREACH_DIRENT() and FOREACH_LINE() macros instead of manual loops. Don't clobber return parameters on failure. Simplify some other things.
2014-11-28treewide: another round of simplificationsMichal Schmidt
Using the same scripts as in f647962d64e "treewide: yet more log_*_errno + return simplifications".