| Age | Commit message (Collapse) | Author |
|---|
|
Let's make sure we handle compression errors properly, and don't
misunderstand an error for success.
Also, let's actually compress things if lz4 is enabled.
Fixes #1662.
|
|
Limit test-compress-benchmark to approx. 12 s of runtime
|
|
We were compressing unitialized memory, which should not result in
any problems, but is inelegant.
|
|
If both lz4 and xz are enabled, this results in a limit of
2×3×2 s ~= 12 s runtime.
Previous implementation started with really small buffer sizes. When
combined with a short time limit this resulteded in abysmal results for xz.
It seems that the initialization overead is really significant for small
buffers. Since xz will not be used by default anymore, this does not
seem worth fixing. Instead buffer sizes are changed to run a
pseudo-random non-repeating pattern. This should allow reasonable testing
for all buffer sizes. For testing, both runtime and the buffer size seed
can be specified on the command line. Sufficiently large runtime allows
all buffer sizes up to 1MB to be tested.
|
|
-q suppresses info messages too
|
|
|
|
|
|
|
|
Using lz4 frame api for coredump files
|
|
Logging for compression and decompression is assymetrical on purpose:
if compiled without some type of compression, those compression code
paths should never be invoked. OTOH, it is possible to encounter
unsupported format on decompression, so leave those log_debug statements
in, to make it easier to diagnose stuff.
|
|
|
|
Fix journalctl --dump-catalog, journalctl --list-catalog
|
|
Make journald audit socket maskable
|
|
Fixes #1514.
|
|
`journalctl --dump-catalog ID1 ID2 ...` works fine.
|
|
|
|
Adding them to the documentation makes it easier to find
the right man page for people who are trying to understand
where some socket in the filesystem is coming from.
|
|
If we were given some sockets through socket activation, and audit
socket is not among them, do not try to open it. This way, if the
socket unit is disabled, we will not receive audit events.
https://bugzilla.redhat.com/show_bug.cgi?id=1227379
|
|
Existing test would use highly-compressible repeatable
input. Two types of input are added:
- zeros
- random blocks interspersed with zeros
The idea is to get more information about behaviour in various cases.
On Intel Xeon the results are:
% ./test-compress-benchmark
XZ/zeros: compressed & decompressed 2535301373 bytes in 32.56s (74.26MiB/s), mean compresion 99.96%, skipped 3160 bytes
LZ4/zeros: compressed & decompressed 2535304362 bytes in 1.16s (2088.69MiB/s), mean compresion 99.60%, skipped 171 bytes
XZ/simple: compressed & decompressed 2535300963 bytes in 30.42s (79.48MiB/s), mean compresion 99.95%, skipped 3570 bytes
LZ4/simple: compressed & decompressed 2535303543 bytes in 1.22s (1978.86MiB/s), mean compresion 99.60%, skipped 990 bytes
XZ/random: compressed & decompressed 381756649 bytes in 60.02s (6.07MiB/s), mean compresion 39.64%, skipped 27813723 bytes
LZ4/random: compressed & decompressed 2507385036 bytes in 0.97s (2477.52MiB/s), mean compresion 54.77%, skipped 27919497 bytes
If someone has ideas for more realistic test cases, they can be easily
added to this framework.
|
|
This converts the stream compression to use the new lz4frame api,
compatible with lz4cat. Previous code used custom headers, so the
compressed file was not compatible with lz4 command line tools.
I considered this the last blocker to using lz4 by default.
Speed seems to be reasonable, although a bit (a few percent) slower
than the lz4 binary, even though compression is the same. I don't
consider this important. It could be caused by the overhead of library
calls, but is probably caused by slightly different buffer sizes or
such. The code in this patch uses mmap, since since this allows the
buffer to be reused while not making the code more complicated at all.
In my testing, this version is noticably faster (~20%) than a naive
single-buffered version. mmap can cause the program to be killed with
SIGBUS, if the underlying file is truncated or a disk error occurs. We
only use this from within coredump and coredumpctl, so I don't
consider this an issue.
Old decompression code is retained and is used if the new code fails
indicating a format error. There have been reports of various smaller
distributions using previous lz4 code, i.e. the old format, and it is
nice to provide backwards compatibility. We can remove the legacy code
in a few versions.
The way that blobs are compressed in the journal is not affected.
|
|
Make the API of the new helpers more similar to the old wrapper.
In particular we now return the hash as a byte string to avoid
any endianness problems.
|
|
hashmap/siphash24: refactor hash functions
|
|
All our hash functions are based on siphash24(), factor out
siphash_init() and siphash24_finalize() and pass the siphash
state to the hash functions rather than the hash key.
This simplifies the hash functions, and in particular makes
composition simpler as calling siphash24_compress() repeatedly
on separate chunks of input has the same effect as first
concatenating the input and then calling siphash23_compress()
on the result.
|
|
Implement a maximum limit on number of journal files to keep around.
Enforcing a limit is useful on this since our performance when viewing
pays a heavy penalty for each journal file to interleve. This setting is
turned on now by default, and set to 100.
Also, actully implement what 348ced909724a1331b85d57aede80a102a00e428
promised: use whatever we find on disk at startup as lower bound on how
much disk space we can use. That commit introduced some provisions to
implement this, but actually never did.
This also adds "journalctl --vacuum-files=" to vacuum files on disk by
their number explicitly.
|
|
|
|
Indicate that we are ignoring errors, when we ignore them, and log that
at LOG_WARNING level.
Use the right error code for the log message.
|
|
|
|
Let's try to use O_NOATIME if we can when vacuuming old journal files,
if we have the permissions for it, so that vacuuming doesn't count as
proper journal read access.
|
|
|
|
The way it is customary everywhere else in our sources.
|
|
Let's use fd_getcrtime_at(), since that *at() family of calls is how we
read the rest of the file metadata, too.
|
|
journald: add syslog fields for driver messages
|
|
logging fixes and more
|
|
|
|
Let's clean up our tree a bit, and reduce invocations of the
thread-unsafe strerror() by replacing it with printf()'s %m specifier.
|
|
shortcut for `systemctl kill --kill-who main --signal SIGUSR2 systemd-journald`
|
|
In sd-bus, the sd_bus_open_xyz() family of calls allocates a new bus,
while sd_bus_default_xyz() family tries to reuse the thread's default
bus. bus_open_transport() sometimes internally uses the former,
sometimes the latter family, but suggests it only calls the former via
its name. Hence, let's avoid this confusion, and generically rename the
call to bus_connect_transport().
Similar for all related calls.
And while we are at it, also change cgls + cgtop to do direct systemd
connections where possible, since all they do is talk to systemd itself.
|
|
|
|
|
|
This also allows us to drop build.h from a ton of files, hence do so.
Since we touched the #includes of those files, let's order them properly
according to CODING_STYLE.
|
|
Also, make it slightly more powerful, by accepting a flags argument, and
make it safe for handling if more than one cmsg attribute happens to be
attached.
|
|
Let's underline the header line of the table shown by cgtop, how it is
customary for tables. In order to do this, let's introduce new ANSI
underline macros, and clean up the existing ones as side effect.
|
|
off_t is a really weird type as it is usually 64bit these days (at least
in sane programs), but could theoretically be 32bit. We don't support
off_t as 32bit builds though, but still constantly deal with safely
converting from off_t to other types and back for no point.
Hence, never use the type anymore. Always use uint64_t instead. This has
various benefits, including that we can expose these values directly as
D-Bus properties, and also that the values parse the same in all cases.
|
|
util: introduce safe_fclose() and port everything over to it
|
|
Adds a coccinelle script to port things over automatically.
|
|
Let's also clean up single-line while and for blocks.
|
|
The previous coccinelle semantic patch that improved usage of
log_error_errno()'s return value, only looked for log_error_errno()
invocations with a single parameter after the error parameter. Update
the patch to handle arbitrary numbers of additional arguments.
|
|
Patch via coccinelle.
|
|
Turns this:
r = -errno;
log_error_errno(errno, "foo");
into this:
r = log_error_errno(errno, "foo");
and this:
r = log_error_errno(errno, "foo");
return r;
into this:
return log_error_errno(errno, "foo");
|
|
|
