| Age | Commit message (Collapse) | Author |
|---|
|
try_context() is such a hot path that the hashmap lookup is expensive.
The number of contexts is small - it is the number of object types.
Using a hashmap is overkill. A plain array will do.
Before:
$ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null
real 0m9.445s
user 0m9.228s
sys 0m0.213s
After:
$ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null
real 0m5.438s
user 0m5.266s
sys 0m0.170s
|
|
This never had any callers. Contexts are freed when the MMapCache is
freed.
|
|
|
|
|
|
Note that numbers 0 and -1 are both replaced with OBJECT_UNUSED,
because they are treated the same everywhere (e.g. type_to_context()
translates them both to 0).
|
|
If type==0 and a non-NULL object were given as arguments to
journal_file_hmac_put_object(), its object type check would fail and it
would return -EBADMSG.
All existing callers use either a positive type or -1. Still, for
behavior consistency with journal_file_move_to_object() let's allow
type 0 to pass.
|
|
It has no other callers. It does not need to be in the header file.
|
|
The only user is sd_journal_enumerate_unique() and, as explained in
the previous commit (fed67c38e3 "journal: map objects to context set by
caller, not by actual object type"), the use of them there is now
superfluous. Let's remove them.
This reverts major parts of commits:
ae97089d49 journal: fix access to munmapped memory in
sd_journal_enumerate_unique
06cc69d44c sd-journal: fix sd_journal_enumerate_unique skipping values
Tested with an "--enable-debug" build and "journalctl --list-boots".
It gives the expected number of results. Additionally, if I then revert
the previous commit ("journal: map objects to context set by caller, not
to actual object type"), it crashes with SIGSEGV, as expected.
|
|
When the caller of journal_file_move_to_object() specifies type==0,
the object header is at first mapped in context 0. Then after the header
is checked, the whole object is mapped in a context determined by
the actual object type (which is not even range-checked using
type_to_context()). This looks wrong. It should map in the
caller-specified context.
An old comment in sd_journal_enumerate_unique() supports this view:
/* We do not use the type context here, but 0 instead,
* so that we can look at this data object at the same
* time as one on another file */
Clearly the expectation was that the data object will remain mapped
in context 0 without being pushed away by mapping other objects in
context OBJECT_DATA.
I suspect that this was the real bug that got fixed by ae97089d49
"journal: fix access to munmapped memory in sd_journal_enumerate_unique".
In other words, journal_file_object_keep/release are superfluous after
applying this patch.
|
|
This is useful for exposing unsafe access to mmapped objects after
the context that they were mapped in was already moved.
For example:
journal_file_move_to_object(f1, OBJECT_DATA, p1, &o1);
journal_file_move_to_object(f2, OBJECT_DATA, p2, &o2);
t = o1->object.type; /* this usually works, but is unsafe */
|
|
|
|
resulting name is actually valid
Also, rename filename_is_safe() to filename_is_valid(), since it
actually does a full validation for what the kernel will accept as file
name, it's not just a heuristic.
|
|
|
|
loop_write() didn't follow the usual systemd rules and returned status
partially in errno and required extensive checks from callers. Some of
the callers dealt with this properly, but many did not, treating
partial writes as successful. Simplify things by conforming to usual rules.
|
|
Let's add some syntactic sugar for iterating through inotify events, and
use it everywhere.
|
|
candidate entry
Suppose that while iterating we have already looked into a journal file
and got a candidate for the next entry. And we are considering to look
into another journal file because it may contain an entry that is nearer
to the current location than the candidate.
We should skip the whole journal file if we can tell by looking at its
header that none of its entries can precede the candidate.
Before:
$ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null
real 0m20.518s
user 0m19.989s
sys 0m0.328s
After:
$ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null
real 0m9.445s
user 0m9.228s
sys 0m0.213s
|
|
Interleaving of entries from many journal files is expensive. But there
is room for optimization.
We can skip looking into journal files whose entries all lie before the
current iterating location. We can tell if that's the case from looking
at the journal file header. This saves a huge amount of work if one has
many of mostly not interleaved journal files.
On my workstation with 90 journal files in /var/log/journal/ID/
totalling 3.4 GB I get these results:
Before:
$ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null
real 5m54.258s
user 2m4.263s
sys 3m48.965s
After:
$ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null
real 0m20.518s
user 0m19.989s
sys 0m0.328s
The high "sys" time in the original was caused by putting more stress on
the mmap-cache than it could handle. With the patch the working set
now consists of fewer mmap windows and mmap-cache is not thrashing.
|
|
In the case where no entries have been added to the journal after the specified
cursor, set need_seek before the main loop to prevent display of the entry at
said cursor.
|
|
With DIRECTION_UP (i.e. navigating backwards) in generic_array_bisect() when the
needle was found as the last item in the array, it wasn't actually processed as
match, resulting in entries being missed.
https://bugs.freedesktop.org/show_bug.cgi?id=86855
|
|
This is mostly likely the audit socket, and we really should close it
if we cannot make sense of it, since as long as it is open the kernel
might disable the kmsg forwarding of audit msgs, and we should avoid
that, since audit msgs might get completely lost then.
I also downgraded the log message we show a bit, after all things should
really work fine, and we proceed fine with it.
|
|
Otherwise they can be optimized away with -DNDEBUG
|
|
|
|
|
|
rather than heap
|
|
Use FOREACH_DIRENT() and FOREACH_LINE() macros instead of manual loops.
Don't clobber return parameters on failure.
Simplify some other things.
|
|
Using the same scripts as in f647962d64e "treewide: yet more log_*_errno
+ return simplifications".
|
|
If the format string contains %m, clearly errno must have a meaningful
value, so we might as well use log_*_errno to have ERRNO= logged.
Using:
find . -name '*.[ch]' | xargs sed -r -i -e \
's/log_(debug|info|notice|warning|error|emergency)\((".*%m.*")/log_\1_errno(errno, \2/'
Plus some whitespace, linewrap, and indent adjustments.
|
|
|
|
|
|
Basically:
find . -name '*.[ch]' | while read f; do perl -i.mmm -e \
'local $/;
local $_=<>;
s/log_(debug|info|notice|warning|error|emergency)\("([^"]*)%s"([^;]*),\s*strerror\(-?([->a-zA-Z_]+)\)\);/log_\1_errno(\4, "\2%m"\3);/gms;print;' \
$f; done
Plus manual indentation fixups.
|
|
|
|
It corrrectly handles both positive and negative errno values.
|
|
As a followup to 086891e5c1 "log: add an "error" parameter to all
low-level logging calls and intrdouce log_error_errno() as log calls
that take error numbers", use sed to convert the simple cases to use
the new macros:
find . -name '*.[ch]' | xargs sed -r -i -e \
's/log_(debug|info|notice|warning|error|emergency)\("(.*)%s"(.*), strerror\(-([a-zA-Z_]+)\)\);/log_\1_errno(-\4, "\2%m"\3);/'
Multi-line log_*() invocations are not covered.
And we also should add log_unit_*_errno().
|
|
Also, while we are at it, introduce some syntactic sugar for creating
ERRNO= and MESSAGE= structured logging fields.
|
|
- Rename log_meta() → log_internal(), to follow naming scheme of most
other log functions that are usually invoked through macros, but never
directly.
- Rename log_info_object() to log_object_info(), simply because the
object should be before any other parameters, to follow OO-style
programming style.
|
|
|
|
When I tryed to run journalctl with --follow and --since arguments it
behaved very strangely.
First It prints logs from what I specified in --since argument, then
printed 10 lines (as is default in --follow) and when app put something
new in to log journalctl printed everithing from the last printed line.
How to reproduce:
1. run: journalctl -m --since 14:00 --follow
Then you'll see 10 lines of logs since 14:00. After that wait until some
app add something in the journal or just run `systemd-cat echo test`
2. After that journalctl will print every single line since 14:00 and will
follow as expected.
As long as --since and --follow will eventually print all relevant
lines, I seen no reason why not to print them right away and not after
first new message in journal.
Relevant bugzillas:
https://bugs.freedesktop.org/show_bug.cgi?id=71546
https://bugs.freedesktop.org/show_bug.cgi?id=64291
|
|
/proc/[pid]:
- status
- maps
- limits
- cgroup
- cwd
- root
- environ
- fd/ & fdinfo/ joined in open_fds
|
|
systemd-journald would refuse to start if it received an unknown
socket from systemd. This is annoying, because the failure more for
systemd-journald is unpleasant: systemd will keep restarting journald,
but most likely the same error will occur every time. It is better
to continue. journald will try to open missing sockets on its own,
so things should mostly work.
One question is whether to close the sockets which cannot be parsed or
to keep them open. Either way we might lose some messages. This
failure is most likely for the audit socket (selinux issues), which
can be opened multiple times so this not a problem, so I decided to
keep them open because it makes it easier to debug the issue after the
system is fully started.
|
|
Also, make all parsing of the kernel cmdline non-fatal.
|
|
After all, this is about files, not arguments, hence EFBIG is more
appropriate than E2BIG
|
|
|
|
|
|
|
|
|
|
have anyway
|
|
Let's make the log output more readable, and the header can be
reconstructed in full from the other fields
|
|
|
|
Similar to auditd actually turn on auditing as we are starting. This way
we can operate entirely without auditd around.
|
|
audit doesn't support timestamps anyway
|
