| Age | Commit message (Collapse) | Author |
|---|
|
Let's add some syntactic sugar for iterating through inotify events, and
use it everywhere.
|
|
candidate entry
Suppose that while iterating we have already looked into a journal file
and got a candidate for the next entry. And we are considering to look
into another journal file because it may contain an entry that is nearer
to the current location than the candidate.
We should skip the whole journal file if we can tell by looking at its
header that none of its entries can precede the candidate.
Before:
$ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null
real 0m20.518s
user 0m19.989s
sys 0m0.328s
After:
$ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null
real 0m9.445s
user 0m9.228s
sys 0m0.213s
|
|
Interleaving of entries from many journal files is expensive. But there
is room for optimization.
We can skip looking into journal files whose entries all lie before the
current iterating location. We can tell if that's the case from looking
at the journal file header. This saves a huge amount of work if one has
many of mostly not interleaved journal files.
On my workstation with 90 journal files in /var/log/journal/ID/
totalling 3.4 GB I get these results:
Before:
$ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null
real 5m54.258s
user 2m4.263s
sys 3m48.965s
After:
$ time ./journalctl --since=2014-06-01 --until=2014-07-01 > /dev/null
real 0m20.518s
user 0m19.989s
sys 0m0.328s
The high "sys" time in the original was caused by putting more stress on
the mmap-cache than it could handle. With the patch the working set
now consists of fewer mmap windows and mmap-cache is not thrashing.
|
|
If the format string contains %m, clearly errno must have a meaningful
value, so we might as well use log_*_errno to have ERRNO= logged.
Using:
find . -name '*.[ch]' | xargs sed -r -i -e \
's/log_(debug|info|notice|warning|error|emergency)\((".*%m.*")/log_\1_errno(errno, \2/'
Plus some whitespace, linewrap, and indent adjustments.
|
|
Basically:
find . -name '*.[ch]' | while read f; do perl -i.mmm -e \
'local $/;
local $_=<>;
s/log_(debug|info|notice|warning|error|emergency)\("([^"]*)%s"([^;]*),\s*strerror\(-?([->a-zA-Z_]+)\)\);/log_\1_errno(\4, "\2%m"\3);/gms;print;' \
$f; done
Plus manual indentation fixups.
|
|
It corrrectly handles both positive and negative errno values.
|
|
As a followup to 086891e5c1 "log: add an "error" parameter to all
low-level logging calls and intrdouce log_error_errno() as log calls
that take error numbers", use sed to convert the simple cases to use
the new macros:
find . -name '*.[ch]' | xargs sed -r -i -e \
's/log_(debug|info|notice|warning|error|emergency)\("(.*)%s"(.*), strerror\(-([a-zA-Z_]+)\)\);/log_\1_errno(-\4, "\2%m"\3);/'
Multi-line log_*() invocations are not covered.
And we also should add log_unit_*_errno().
|
|
Anything that uses hashmap_next() almost certainly cares about the order
and needs to be an OrderedHashmap.
|
|
string_is_safe()
After all, we know have this as generic validator, so let's be correct
and use it wherver applicable.
|
|
|
|
sd_journal_enumerate_unique will lock its mmap window to prevent it
from being released by calling mmap_cache_get with keep_always=true.
This call may return windows that are wider, but compatible with the
parameters provided to it.
This can result in a mismatch where the window to be released cannot
properly be selected, because we have more than one window matching the
parameters of mmap_cache_release. Therefore, introduce a release_cookie
to be used when releasing the window.
https://bugs.freedesktop.org/show_bug.cgi?id=79380
|
|
systemctl would call sd_j_enumerate_unique() interleaved with
sd_j_next(). But the latter can remove a file if it detects an
error in it. In those circumstances sd_j_enumerate_unique would
restart with the first file in hashmap. With many corrupted files
sd_j_enumerate_unique might iterate over the list multiple times.
Avoid this by jumping to the next file in unique list if possible,
or setting a flag that tells sd_j_enumerate_unique that it is done
otherwise.
|
|
It is redundant to store 'hash' and 'compare' function pointers in
struct Hashmap separately. The functions always comprise a pair.
Store a single pointer to struct hash_ops instead.
systemd keeps hundreds of hashmaps, so this saves a little bit of
memory.
|
|
If the journal is corrupted, we might return an object that does
not start with the expected field name and/or is shorter than it
should.
|
|
|
|
They have different size on 32 bit, so they are really not interchangable.
|
|
String which ended in an unfinished quote were accepted, potentially
with bad memory accesses.
Reject anything which ends in a unfished quote, or contains
non-whitespace characters right after the closing quote.
_FOREACH_WORD now returns the invalid character in *state. But this return
value is not checked anywhere yet.
Also, make 'word' and 'state' variables const pointers, and rename 'w'
to 'word' in various places. Things are easier to read if the same name
is used consistently.
mbiebl_> am I correct that something like this doesn't work
mbiebl_> ExecStart=/usr/bin/encfs --extpass='/bin/systemd-ask-passwd "Unlock EncFS"'
mbiebl_> systemd seems to strip of the quotes
mbiebl_> systemctl status shows
mbiebl_> ExecStart=/usr/bin/encfs --extpass='/bin/systemd-ask-password Unlock EncFS $RootDir $MountPoint
mbiebl_> which is pretty weird
|
|
Also modify the function itself to be a bit simpler to read.
|
|
|
|
|
|
Add liblz4 as an optional dependency when requested with --enable-lz4,
and use it in preference to liblzma for journal blob and coredump
compression. To retain backwards compatibility, XZ is used to
decompress old blobs.
Things will function correctly only with lz4-119.
Based on the benchmarks found on the web, lz4 seems to be the best
choice for "quick" compressors atm.
For pkg-config status, see http://code.google.com/p/lz4/issues/detail?id=135.
|
|
|
|
No functional change expected :)
|
|
safe_close() automatically becomes a NOP when a negative fd is passed,
and returns -1 unconditionally. This makes it easy to write lines like
this:
fd = safe_close(fd);
Which will close an fd if it is open, and reset the fd variable
correctly.
By making use of this new scheme we can drop a > 200 lines of code that
was required to test for non-negative fds or to reset the closed fd
variable afterwards.
|
|
If we encounter an inconsistency in a file, let's just
ignore it. Otherwise, after previous patch, we would try,
and fail, to use this file in every invocation of sd_journal_next
or sd_journal_previous that happens afterwards.
|
|
As pointed-out by clang -Wunreachable-code.
No behaviour changes.
|
|
gcc (4.8.2, arm) does not understand that next_beyond_location() will
always set 'p' when it returns > 0. Initialize p in order to fix this.
|
|
sd_journal_get_cutoff_realtime_usec() on failure
|
|
If -flto is used then gcc will generate a lot more warnings than before,
among them a number of use-without-initialization warnings. Most of them
without are false positives, but let's make them go away, because it
doesn't really matter.
|
|
Resolve spotted issues related to missing or extraneous commas, dashes.
|
|
sd_j_e_u needs to keep a reference to an object while comparing it
with possibly duplicate objects in other files. Because the size of
mmap cache is limited, with enough files and object to compare to,
at some point the object being compared would be munmapped, resulting
in a segmentation fault.
Fix this issue by turning keep_always into a reference count that can
be increased and decreased. Other callers which set keep_always=true
are unmodified: their references are never released but are ignored
when the whole file is closed, which happens at some point. keep_always
is increased in sd_j_e_u and later on released.
|
|
This commit also adds error handling for failures during
directory reading.
|
|
log message
|
|
show messages from host too
|
|
This adds the new library call sd_journal_open_container() and a new
"-M" switch to journalctl. Particular care is taken that journalctl's
"-b" switch resolves to the current boot ID of the container, not the
host.
|
|
|
|
|
|
|
|
each invocation
We can determine the list entry type via the typeof() gcc construct, and
so we should to make the macros much shorter to use.
|
|
|
|
|
|
A few asserts are replaced with 'return -EINVAL'. I think that
assert should not be used to check argument in public functions.
Fields in struct sd_journal are rearranged to make it less
swiss-cheesy.
|
|
The order was different in various places, which makes it harder to
read to code. Also consistently use ternany for all direction checks.
Remove one free(NULL).
|
|
The fields in JournalFile are moved around to avoid wasting
7 bytes because of alignment.
|
|
This allows the caller to explicitly specify which journal files
should be opened. The same functionality could be achieved before
by creating a directory and playing around with symlinks. It
is useful to debug stuff and explore the journal, and has been
requested before.
Waiting is supported, the journal will notice modifications on
the files supplied when opening the journal, but will not add
any new files.
|
|
Casts are visually heavy, and can obscure unwanted truncations.
|
|
AND term usually don't have many subterms (4 seems to be the maximum
sensible number, e.g. _BOOT_ID && _SYSTEMD_UNIT && _PID && MESSAGE_ID).
Nevertheless, the cost of checking each subterm can be relatively
high, especially when the nested terms are compound, and it
makes sense to minimize the number of checks.
Instead of looping to the end and then again over the whole list once
again after at least one term changed the offset, start the loop at
the term which caused the change. This way ½ terms in the AND match
are not checked unnecessarily again.
|
|
This is the just the library part.
SD_JOURNAL_CURRENT_USER flags is added to sd_j_open(), to open
files from current user.
SD_JOURNAL_SYSTEM_ONLY is renamed to SD_JOURNAL_SYSTEM,
and changed to mean to (also) open system files. This way various
flags can be combined, which gives them nicer semantics, especially
if other ones are added later.
Backwards compatibility is kept, because SD_JOURNAL_SYSTEM_ONLY
is equivalent to SD_JOURNAL_SYSTEM if used alone, and before there
we no other flags.
|
|
|
|
Also reworded a few debug messages for brevity, and added a log
statement which prints out the filter at debug level:
Journal filter: (((UNIT=sys-module-configfs.device AND _PID=1) OR (COREDUMP_UNIT=sys-module-configfs.device AND MESSAGE_ID=fc2e22bc6ee647b6b90729ab34a250b1) OR _SYSTEMD_UNIT=sys-module-configfs.device) AND _BOOT_ID=4e3c518ab0474c12ac8de7896fe6b154)
|
