summaryrefslogtreecommitdiff
path: root/src/journal
AgeCommit message (Collapse)Author
2014-12-18Revert "journal: optimize iteration: skip whole files behind current location"Michal Schmidt
This reverts commit b7c88ab8cc7d55a43450bf3dea750f95f2e910d6. This optimization will be made redundant by the following patches.
2014-12-18Revert "journal: optimize iteration: skip files that cannot improve current ↵Michal Schmidt
candidate entry" This reverts commit f8b5a3b75fb55f0acb85c21424b3893c822742e9. This optimization will be made redundant by the following patches.
2014-12-18journal: delete unused function journal_file_skip_entry()Michal Schmidt
Its only caller is a test.
2014-12-18journal: delete unused function journal_file_move_to_entry_by_offset()Michal Schmidt
2014-12-13journal: replace contexts hashmap with a plain arrayMichal Schmidt
try_context() is such a hot path that the hashmap lookup is expensive. The number of contexts is small - it is the number of object types. Using a hashmap is overkill. A plain array will do. Before: $ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null real 0m9.445s user 0m9.228s sys 0m0.213s After: $ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null real 0m5.438s user 0m5.266s sys 0m0.170s
2014-12-13journal: delete unused function mmap_cache_close_contextMichal Schmidt
This never had any callers. Contexts are freed when the MMapCache is freed.
2014-12-13journal: push type_to_context conversion down to journal_file_move_to()Michal Schmidt
2014-12-13journal: have a named enum ObjectTypeMichal Schmidt
2014-12-13journal: consistently use OBJECT_<type> names instead of numbersMichal Schmidt
Note that numbers 0 and -1 are both replaced with OBJECT_UNUSED, because they are treated the same everywhere (e.g. type_to_context() translates them both to 0).
2014-12-13journal: consistently allow type==0 to mean "any type"Michal Schmidt
If type==0 and a non-NULL object were given as arguments to journal_file_hmac_put_object(), its object type check would fail and it would return -EBADMSG. All existing callers use either a positive type or -1. Still, for behavior consistency with journal_file_move_to_object() let's allow type 0 to pass.
2014-12-13journal: move type_to_context() to journal-file.cMichal Schmidt
It has no other callers. It does not need to be in the header file.
2014-12-13journal: remove journal_file_object_keep/release functionsMichal Schmidt
The only user is sd_journal_enumerate_unique() and, as explained in the previous commit (fed67c38e3 "journal: map objects to context set by caller, not by actual object type"), the use of them there is now superfluous. Let's remove them. This reverts major parts of commits: ae97089d49 journal: fix access to munmapped memory in sd_journal_enumerate_unique 06cc69d44c sd-journal: fix sd_journal_enumerate_unique skipping values Tested with an "--enable-debug" build and "journalctl --list-boots". It gives the expected number of results. Additionally, if I then revert the previous commit ("journal: map objects to context set by caller, not to actual object type"), it crashes with SIGSEGV, as expected.
2014-12-13journal: map objects to context set by caller, not by actual object typeMichal Schmidt
When the caller of journal_file_move_to_object() specifies type==0, the object header is at first mapped in context 0. Then after the header is checked, the whole object is mapped in a context determined by the actual object type (which is not even range-checked using type_to_context()). This looks wrong. It should map in the caller-specified context. An old comment in sd_journal_enumerate_unique() supports this view: /* We do not use the type context here, but 0 instead, * so that we can look at this data object at the same * time as one on another file */ Clearly the expectation was that the data object will remain mapped in context 0 without being pushed away by mapping other objects in context OBJECT_DATA. I suspect that this was the real bug that got fixed by ae97089d49 "journal: fix access to munmapped memory in sd_journal_enumerate_unique". In other words, journal_file_object_keep/release are superfluous after applying this patch.
2014-12-13journal: add debug mode for mmap-cache (--enable-debug=mmap-cache)Michal Schmidt
This is useful for exposing unsafe access to mmapped objects after the context that they were mapped in was already moved. For example: journal_file_move_to_object(f1, OBJECT_DATA, p1, &o1); journal_file_move_to_object(f2, OBJECT_DATA, p2, &o2); t = o1->object.type; /* this usually works, but is unsafe */
2014-12-12copy: use btrfs reflinking only whe we know we copy full filesLennart Poettering
2014-12-12util: when using basename() for creating temporary files, verify the ↵Lennart Poettering
resulting name is actually valid Also, rename filename_is_safe() to filename_is_valid(), since it actually does a full validation for what the kernel will accept as file name, it's not just a heuristic.
2014-12-11journald: correct spacing near eol code commentsTorstein Husebø
2014-12-09treewide: sanitize loop_writeZbigniew Jędrzejewski-Szmek
loop_write() didn't follow the usual systemd rules and returned status partially in errno and required extensive checks from callers. Some of the callers dealt with this properly, but many did not, treating partial writes as successful. Simplify things by conforming to usual rules.
2014-12-10core: unify how we iterate over inotify eventsLennart Poettering
Let's add some syntactic sugar for iterating through inotify events, and use it everywhere.
2014-12-09journal: optimize iteration: skip files that cannot improve current ↵Michal Schmidt
candidate entry Suppose that while iterating we have already looked into a journal file and got a candidate for the next entry. And we are considering to look into another journal file because it may contain an entry that is nearer to the current location than the candidate. We should skip the whole journal file if we can tell by looking at its header that none of its entries can precede the candidate. Before: $ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null real 0m20.518s user 0m19.989s sys 0m0.328s After: $ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null real 0m9.445s user 0m9.228s sys 0m0.213s
2014-12-09journal: optimize iteration: skip whole files behind current locationMichal Schmidt
Interleaving of entries from many journal files is expensive. But there is room for optimization. We can skip looking into journal files whose entries all lie before the current iterating location. We can tell if that's the case from looking at the journal file header. This saves a huge amount of work if one has many of mostly not interleaved journal files. On my workstation with 90 journal files in /var/log/journal/ID/ totalling 3.4 GB I get these results: Before: $ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null real 5m54.258s user 2m4.263s sys 3m48.965s After: $ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null real 0m20.518s user 0m19.989s sys 0m0.328s The high "sys" time in the original was caused by putting more stress on the mmap-cache than it could handle. With the patch the working set now consists of fewer mmap windows and mmap-cache is not thrashing.
2014-12-09journalctl: respect --after-cursor semantics with --follow in all casesWesley Dawson
In the case where no entries have been added to the journal after the specified cursor, set need_seek before the main loop to prevent display of the entry at said cursor.
2014-12-08journal: Fix navigating backwards missing entriesOlivier Brunel
With DIRECTION_UP (i.e. navigating backwards) in generic_array_bisect() when the needle was found as the last item in the array, it wasn't actually processed as match, resulting in entries being missed. https://bugs.freedesktop.org/show_bug.cgi?id=86855
2014-11-30journald: close passed fds we cannot make sense ofLennart Poettering
This is mostly likely the audit socket, and we really should close it if we cannot make sense of it, since as long as it is open the kernel might disable the kmsg forwarding of audit msgs, and we should avoid that, since audit msgs might get completely lost then. I also downgraded the log message we show a bit, after all things should really work fine, and we proceed fine with it.
2014-11-30tests: use assert_se instead of assertRonny Chevalier
Otherwise they can be optimized away with -DNDEBUG
2014-11-29coredump: Support coredump.conf.d directories in the usual search pathsJosh Triplett
2014-11-29journald: Support journald.conf.d directories in the usual search pathsJosh Triplett
2014-11-28coredump: simplify a few things by allocating small fields on the stack ↵Lennart Poettering
rather than heap
2014-11-28coredump: rework compose_open_fds()Lennart Poettering
Use FOREACH_DIRENT() and FOREACH_LINE() macros instead of manual loops. Don't clobber return parameters on failure. Simplify some other things.
2014-11-28treewide: another round of simplificationsMichal Schmidt
Using the same scripts as in f647962d64e "treewide: yet more log_*_errno + return simplifications".
2014-11-28treewide: use log_*_errno whenever %m is in the format stringMichal Schmidt
If the format string contains %m, clearly errno must have a meaningful value, so we might as well use log_*_errno to have ERRNO= logged. Using: find . -name '*.[ch]' | xargs sed -r -i -e \ 's/log_(debug|info|notice|warning|error|emergency)\((".*%m.*")/log_\1_errno(errno, \2/' Plus some whitespace, linewrap, and indent adjustments.
2014-11-28treewide: more log_*_errno + return simplificationsMichal Schmidt
2014-11-28treewide: simplify log_*_errno(r,...) immediately followed by "return r"Michal Schmidt
2014-11-28treewide: more log_*_errno() conversions, multiline callsMichal Schmidt
Basically: find . -name '*.[ch]' | while read f; do perl -i.mmm -e \ 'local $/; local $_=<>; s/log_(debug|info|notice|warning|error|emergency)\("([^"]*)%s"([^;]*),\s*strerror\(-?([->a-zA-Z_]+)\)\);/log_\1_errno(\4, "\2%m"\3);/gms;print;' \ $f; done Plus manual indentation fixups.
2014-11-28treewide: more log_*_errno() conversionsMichal Schmidt
2014-11-28treewide: no need to negate errno for log_*_errno()Michal Schmidt
It corrrectly handles both positive and negative errno values.
2014-11-28treewide: auto-convert the simple cases to log_*_errno()Michal Schmidt
As a followup to 086891e5c1 "log: add an "error" parameter to all low-level logging calls and intrdouce log_error_errno() as log calls that take error numbers", use sed to convert the simple cases to use the new macros: find . -name '*.[ch]' | xargs sed -r -i -e \ 's/log_(debug|info|notice|warning|error|emergency)\("(.*)%s"(.*), strerror\(-([a-zA-Z_]+)\)\);/log_\1_errno(-\4, "\2%m"\3);/' Multi-line log_*() invocations are not covered. And we also should add log_unit_*_errno().
2014-11-28log: fix order of log_unit_struct() to match other logging callsLennart Poettering
Also, while we are at it, introduce some syntactic sugar for creating ERRNO= and MESSAGE= structured logging fields.
2014-11-27log: rearrange log function namingLennart Poettering
- Rename log_meta() → log_internal(), to follow naming scheme of most other log functions that are usually invoked through macros, but never directly. - Rename log_info_object() to log_object_info(), simply because the object should be before any other parameters, to follow OO-style programming style.
2014-11-27coredump: use openatZbigniew Jędrzejewski-Szmek
2014-11-27journalctl: print all possible lines immediately with --follow + --sinceAndrej Manduch
When I tryed to run journalctl with --follow and --since arguments it behaved very strangely. First It prints logs from what I specified in --since argument, then printed 10 lines (as is default in --follow) and when app put something new in to log journalctl printed everithing from the last printed line. How to reproduce: 1. run: journalctl -m --since 14:00 --follow Then you'll see 10 lines of logs since 14:00. After that wait until some app add something in the journal or just run `systemd-cat echo test` 2. After that journalctl will print every single line since 14:00 and will follow as expected. As long as --since and --follow will eventually print all relevant lines, I seen no reason why not to print them right away and not after first new message in journal. Relevant bugzillas: https://bugs.freedesktop.org/show_bug.cgi?id=71546 https://bugs.freedesktop.org/show_bug.cgi?id=64291
2014-11-27coredump: collect all /proc data useful for bug reportingJakub Filak
/proc/[pid]: - status - maps - limits - cgroup - cwd - root - environ - fd/ & fdinfo/ joined in open_fds
2014-11-26journald: proceed even if some sockets are unknownZbigniew Jędrzejewski-Szmek
systemd-journald would refuse to start if it received an unknown socket from systemd. This is annoying, because the failure more for systemd-journald is unpleasant: systemd will keep restarting journald, but most likely the same error will occur every time. It is better to continue. journald will try to open missing sockets on its own, so things should mostly work. One question is whether to close the sockets which cannot be parsed or to keep them open. Either way we might lose some messages. This failure is most likely for the audit socket (selinux issues), which can be opened multiple times so this not a problem, so I decided to keep them open because it makes it easier to debug the issue after the system is fully started.
2014-11-07util: simplify proc_cmdline() to reuse get_process_cmdline()Lennart Poettering
Also, make all parsing of the kernel cmdline non-fatal.
2014-11-07copy: change error code when hitting copy limit to EFBIGLennart Poettering
After all, this is about files, not arguments, hence EFBIG is more appropriate than E2BIG
2014-11-06s/commandline/command line/gHarald Hoyer
2014-11-06journal: adjust audit log messages a bitLennart Poettering
2014-11-04journald: include audit message type number in MESSAGE= stringLennart Poettering
2014-11-04journal: also consider audit fields with '-' validLennart Poettering
2014-11-04journald: don't pass around SO_TIMESTAMP timestamp for audit, which we don't ↵Lennart Poettering
have anyway