summaryrefslogtreecommitdiff
path: root/src/journal
AgeCommit message (Collapse)Author
2016-12-21util-lib: various improvements to kernel command line parsingLennart Poettering
This improves kernel command line parsing in a number of ways: a) An kernel option "foo_bar=xyz" is now considered equivalent to "foo-bar-xyz", i.e. when comparing kernel command line option names "-" and "_" are now considered equivalent (this only applies to the option names though, not the option values!). Most of our kernel options used "-" as word separator in kernel command line options so far, but some used "_". With this change, which was a source of confusion for users (well, at least of one user: myself, I just couldn't remember that it's systemd.debug-shell, not systemd.debug_shell). Considering both as equivalent is inspired how modern kernel module loading normalizes all kernel module names to use underscores now too. b) All options previously using a dash for separating words in kernel command line options now use an underscore instead, in all documentation and in code. Since a) has been implemented this should not create any compatibility problems, but normalizes our documentation and our code. c) All kernel command line options which take booleans (or are boolean-like) have been reworked so that "foobar" (without argument) is now equivalent to "foobar=1" (but not "foobar=0"), thus normalizing the handling of our boolean arguments. Specifically this means systemd.debug-shell and systemd_debug_shell=1 are now entirely equivalent. d) All kernel command line options which take an argument, and where no argument is specified will now result in a log message. e.g. passing just "systemd.unit" will no result in a complain that it needs an argument. This is implemented in the proc_cmdline_missing_value() function. e) There's now a call proc_cmdline_get_bool() similar to proc_cmdline_get_key() that parses booleans (following the logic explained in c). f) The proc_cmdline_parse() call's boolean argument has been replaced by a new flags argument that takes a common set of bits with proc_cmdline_get_key(). g) All kernel command line APIs now begin with the same "proc_cmdline_" prefix. h) There are now tests for much of this. Yay!
2016-12-17test-compress: fix warning about LZ4_compress_limitedOutputZbigniew Jędrzejewski-Szmek
691b90d465 fixed one spot, but missed the other one.
2016-12-14journalctl: improve wording in an errors messageLennart Poettering
Fixes: #4660
2016-12-10journal: fix warning about LZ4_compress_limitedOutputZbigniew Jędrzejewski-Szmek
2016-12-02journal: make sure to initially populate the space info cache (#4807)Franck Bui
Make sure to populate the cache in cache_space_refresh() at least once otherwise it's possible that the system boots fast enough (and the journal flush service is finished) before the invalidate cache timeout (30 us) has expired. Fixes: #4790
2016-12-01fs-util: add flags parameter to chase_symlinks()Lennart Poettering
Let's remove chase_symlinks_prefix() and instead introduce a flags parameter to chase_symlinks(), with a flag CHASE_PREFIX_ROOT that exposes the behaviour of chase_symlinks_prefix().
2016-12-01tree-wide: stop using canonicalize_file_name(), use chase_symlinks() insteadLennart Poettering
Let's use chase_symlinks() everywhere, and stop using GNU canonicalize_file_name() everywhere. For most cases this should not change behaviour, however increase exposure of our function to get better tested. Most importantly in a few cases (most notably nspawn) it can take the correct root directory into account when chasing symlinks.
2016-11-25fix journald startup problem when code is compiled with -DNDEBUG (#4735)Waldemar Brodkorb
Similar to this patch from here: http://systemd-devel.freedesktop.narkive.com/AvfCbi6c/patch-0-3-using-assert-se-on-actions-with-side-effects-on-test-cases If the code is compiled with -DNDEBUG which is the default for some embedded buildsystems, systemd-journald does not startup and silently fails.
2016-11-07Rename formats-util.h to format-util.hZbigniew Jędrzejewski-Szmek
We don't have plural in the name of any other -util files and this inconsistency trips me up every time I try to type this file name from memory. "formats-util" is even hard to pronounce.
2016-11-03Merge pull request #4510 from keszybz/tree-wide-cleanupsLennart Poettering
Tree wide cleanups
2016-10-31test-compression: allow the file to compress to be specifiedZbigniew Jędrzejewski-Szmek
I'm seeing strange decompression errors with lz4, which might be content-dependent. Extend test-compression to allow testing specific content. (Edit: PEBKAC: lzcat and lz4cat are not the same beast. Nevertheless, the test might still be useful in the future.)
2016-10-24Merge pull request #4459 from keszybz/commandline-parsingLennart Poettering
Commandline parsing simplification and udev fix
2016-10-23tree-wide: drop NULL sentinel from strjoinZbigniew Jędrzejewski-Szmek
This makes strjoin and strjoina more similar and avoids the useless final argument. spatch -I . -I ./src -I ./src/basic -I ./src/basic -I ./src/shared -I ./src/shared -I ./src/network -I ./src/locale -I ./src/login -I ./src/journal -I ./src/journal -I ./src/timedate -I ./src/timesync -I ./src/nspawn -I ./src/resolve -I ./src/resolve -I ./src/systemd -I ./src/core -I ./src/core -I ./src/libudev -I ./src/udev -I ./src/udev/net -I ./src/udev -I ./src/libsystemd/sd-bus -I ./src/libsystemd/sd-event -I ./src/libsystemd/sd-login -I ./src/libsystemd/sd-netlink -I ./src/libsystemd/sd-network -I ./src/libsystemd/sd-hwdb -I ./src/libsystemd/sd-device -I ./src/libsystemd/sd-id128 -I ./src/libsystemd-network --sp-file coccinelle/strjoin.cocci --in-place $(git ls-files src/*.c) git grep -e '\bstrjoin\b.*NULL' -l|xargs sed -i -r 's/strjoin\((.*), NULL\)/strjoin(\1)/' This might have missed a few cases (spatch has a really hard time dealing with _cleanup_ macros), but that's no big issue, they can always be fixed later.
2016-10-22tree-wide: make parse_proc_cmdline() strip "rd." prefix automaticallyZbigniew Jędrzejewski-Szmek
This stripping is contolled by a new boolean parameter. When the parameter is true, it means that the caller does not care about the distinction between initrd and real root, and wants to act on both rd-dot-prefixed and unprefixed parameters in the initramfs, and only on the unprefixed parameters in real root. If the parameter is false, behaviour is the same as before. Changes by caller: log.c (systemd.log_*): changed to accept rd-dot-prefix params pid1: no change, custom logic cryptsetup-generator: no change, still accepts rd-dot-prefix params debug-generator: no change, does not accept rd-dot-prefix params fsck: changed to accept rd-dot-prefix params fstab-generator: no change, custom logic gpt-auto-generator: no change, custom logic hibernate-resume-generator: no change, does not accept rd-dot-prefix params journald: changed to accept rd-dot-prefix params modules-load: no change, still accepts rd-dot-prefix params quote-check: no change, does not accept rd-dot-prefix params udevd: no change, still accepts rd-dot-prefix params I added support for "rd." params in the three cases where I think it's useful: logging, fsck options, journald forwarding options.
2016-10-22journald: convert journald to use parse_proc_cmdlineZbigniew Jędrzejewski-Szmek
This makes journald use the common option parsing functionality. One behavioural change is implemented: "systemd.journald.forward_to_syslog" is now equivalent to "systemd.journald.forward_to_syslog=1". I think it's nicer to use this way.
2016-10-22journal: remove unused variableThomas Hindoe Paaboel Andersen
2016-10-21journald: systemd.journald.max_level_* kernel command line options (#4427)Umut Tezduyar Lindskog
The log forward levels can be configured through kernel command line.
2016-10-20journald,core: add short comments we we keep reopening /dev/console all the timeLennart Poettering
Just to make sure the next one reading this isn't surprised that the fd isn't kept open. SAK and stuff... Fix suggested: https://github.com/systemd/systemd/pull/4366#issuecomment-253659162
2016-10-19journal: rename determine_space_for() into cache_space_refresh()Franck Bui
Now that determine_space_for() only deals with storage space (cached) values, rename it so it reflects the fact that only the cached storage space values are updated.
2016-10-19journal: introduce patch_min_use() helperFranck Bui
Updating min_use is rather an unusual operation that is limited when we first open the journal files, therefore extracts it from determine_space_for() and create a function of its own and call this new function when needed. determine_space_for() is now dealing with storage space (cached) values only. There should be no functional changes.
2016-10-19journal: introduce cache_space_invalidate()Franck Bui
Introduce a dedicated helper in order to reset the storage space cache.
2016-10-19journal: cache used vfs stats as wellFranck Bui
The set of storage space values we cache are calculated according to a couple of filesystem statistics (free blocks, block size). This patch caches the vfs stats we're interested in so these values are available later and coherent with the rest of the space cached values.
2016-10-19journal: don't emit space usage message when opening the journal (#4190)Franck Bui
This patch makes system_journal_open() stop emitting the space usage message. The caller is now free to emit this message when appropriate. When restarting the journal, we can now emit the message *after* flushing the journal (if required) so that all flushed log entries are written in the persistent journal *before* the status message. This is required since the status message is always younger than the flushed entries. Fixes #4190.
2016-10-19journal: introduce server_space_usage_message()Franck Bui
This commit simply extracts from determine_space_for() the code which emits the storage usage message and put it into a function of its own so it can be reused by others paths later. No functional changes.
2016-10-19journal: introduce JournalStorage and JournalStorageSpace structuresFranck Bui
This structure keeps track of specificities for a given journal type (persistent or volatile) such as metrics, name, etc... The cached space values are now moved in this structure so that each journal has its own set of cached values. Previously only one set existed and we didn't know if the cached values were for the runtime journal or the persistent one. When doing: determine_space_for(s, runtime_metrics, ...); determine_space_for(s, system_metrics, ...); the second call returned the cached values for the runtime metrics.
2016-10-19journal: introduce determine_path_usage()Franck Bui
This commit simply extracts from determine_space_for() the code which determines the FS usage where the passed path lives (statvfs(3)) and put it into a function of its own so it can be reused by others paths later. No functional changes.
2016-10-16tree-wide: use mfree moreZbigniew Jędrzejewski-Szmek
2016-10-13Merge pull request #4362 from poettering/journalbootlistfixZbigniew Jędrzejewski-Szmek
2016-10-12journalctl: say in which directory we vacuum stuffLennart Poettering
Fixes: #4060
2016-10-12journalctl: don't claim the journal was stored on diskLennart Poettering
Let's just say that the journal takes up space in the file system, not on disk, as tmpfs is definitely a file system, but not a disk. Fixes: #4059
2016-10-12journal: refuse opening journal files from the future for writingLennart Poettering
Never permit that we write to journal files that have newer timestamps than our local wallclock has. If we'd accept that, then the entries in the file might end up not being ordered strictly. Let's refuse this with ETXTBSY, and then immediately rotate to use a new file, so that each file remains strictly ordered also be wallclock internally.
2016-10-12journald: automatically rotate journal files when the clock jumps backwardsLennart Poettering
As soon as we notice that the clock jumps backwards, rotate journal files. This is beneficial, as this makes sure that the entries in journal files remain strictly ordered internally, and thus the bisection algorithm applied on it is not confused. This should help avoiding borked wallclock-based bisection on journal files as witnessed in #4278.
2016-10-12journald: use the event loop dispatch timestamp for journal entriesLennart Poettering
Let's use the earliest linearized event timestamp for journal entries we have: the event dispatch timestamp from the event loop, instead of requerying the timestamp at the time of writing. This makes the time a bit more accurate, allows us to query the kernel time one time less per event loop, and also makes sure we always use the same timestamp for both attempts to write an entry to a journal file.
2016-10-12journal: when iterating through entry arrays and we hit an invalid one keep ↵Lennart Poettering
going When iterating through partially synced journal files we need to be prepared for hitting with invalid entries (specifically: non-initialized). Instead of generated an error and giving up, let's simply try to preceed with the next one that is valid (and debug log about this). This reworks the logic introduced with caeab8f626e709569cc492b75eb7e119076059e7 to iteration in both directions, and tries to look for valid entries located after the invalid one. It also extends the behaviour to both iterating through the global entry array and per-data object entry arrays. Fixes: #4088
2016-10-12journal: add an explicit check for uninitialized objectsLennart Poettering
Let's make dissecting of borked journal files more expressive: if we encounter an object whose first 8 bytes are all zeroes, then let's assume the object was simply never initialized, and say so. Previously, this would be detected as "overly short object", which is true too in a away, but it's a lot more helpful printing different debug options for the case where the size is not initialized at all and where the size is initialized to some bogus value. No function behaviour change, only a different log messages for both cases.
2016-10-12journal: also check that our entry arrays are properly orderedLennart Poettering
Let's and extra check, reusing check_properly_ordered() also for journal_file_next_entry_for_data().
2016-10-12journal: split out check for properly ordered arrays into its own functionLennart Poettering
This adds a new call check_properly_ordered(), which we can reuse later, and makes the code a bit more readable.
2016-10-12journal: split out array index inc/dec code into a new call bump_array_index()Lennart Poettering
This allows us to share a bit more code between journal_file_next_entry() and journal_file_next_entry_for_data().
2016-10-12journal: when we encounter a broken journal file, add some debug loggingLennart Poettering
Let's make it easier to figure out when we see an invalid journal file, why we consider it invalid, and add some minimal debug logging for it. This log output is normally not seen (after all, this all is library code), unless debug logging is exlicitly turned on.
2016-10-12Avoid forever loop for journalctl --list-boots command (#4278)hese10
When date is changed in system to future and normal user logs to new journal file, and then date is changed back to present time, the "journalctl --list-boot" command goes to forever loop. This commit tries to fix this problem by checking first the boot id list if the found boot id was already in that list. If it is found, then stopping the boot id find loop.
2016-10-07core: add "invocation ID" concept to service managerLennart Poettering
This adds a new invocation ID concept to the service manager. The invocation ID identifies each runtime cycle of a unit uniquely. A new randomized 128bit ID is generated each time a unit moves from and inactive to an activating or active state. The primary usecase for this concept is to connect the runtime data PID 1 maintains about a service with the offline data the journal stores about it. Previously we'd use the unit name plus start/stop times, which however is highly racy since the journal will generally process log data after the service already ended. The "invocation ID" kinda matches the "boot ID" concept of the Linux kernel, except that it applies to an individual unit instead of the whole system. The invocation ID is passed to the activated processes as environment variable. It is additionally stored as extended attribute on the cgroup of the unit. The latter is used by journald to automatically retrieve it for each log logged message and attach it to the log entry. The environment variable is very easily accessible, even for unprivileged services. OTOH the extended attribute is only accessible to privileged processes (this is because cgroupfs only supports the "trusted." xattr namespace, not "user."). The environment variable may be altered by services, the extended attribute may not be, hence is the better choice for the journal. Note that reading the invocation ID off the extended attribute from journald is racy, similar to the way reading the unit name for a logging process is. This patch adds APIs to read the invocation ID to sd-id128: sd_id128_get_invocation() may be used in a similar fashion to sd_id128_get_boot(). PID1's own logging is updated to always include the invocation ID when it logs information about a unit. A new bus call GetUnitByInvocationID() is added that allows retrieving a bus path to a unit by its invocation ID. The bus path is built using the invocation ID, thus providing a path for referring to a unit that is valid only for the current runtime cycleof it. Outlook for the future: should the kernel eventually allow passing of cgroup information along AF_UNIX/SOCK_DGRAM messages via a unique cgroup id, then we can alter the invocation ID to be generated as hash from that rather than entirely randomly. This way we can derive the invocation race-freely from the messages.
2016-10-07journal: fix format string used for usec_tLennart Poettering
2016-10-07journal: complete slice info in journal metadataLennart Poettering
We are already attaching the system slice information to log messages, now add theuser slice info too, as well as the object slice info.
2016-10-06journald, ratelimit: fix inaccurate message suppression in ↵Yuki Inoguchi
journal_rate_limit_test() (#4291) Currently, the ratelimit does not handle the number of suppressed messages accurately. Even though the number of messages reaches the limit, it still allows to add one extra messages to journal. This patch fixes the problem.
2016-09-28journal: add stdout_stream_scan() comment (#4102)Vito Caputo
When s->length is zero this function doesn't do anything, note that in a comment.
2016-09-26journald,ratelimit: fix wrong calculation of burst_modulate() (#4218)HATAYAMA Daisuke
This patch fixes wrong calculation of burst_modulate(), which now calculates the values smaller than really expected ones if available disk space is strictly more than 1MB. In particular, if available disk space is strictly more than 1MB and strictly less than 16MB, the resulted value becomes smaller than its original one. >>> (math.log2(1*1024**2)-16) / 4 1.0 >>> (math.log2(16*1024**2)-16) / 4 2.0 >>> (math.log2(256*1024**2)-16) / 4 3.0 → This matches the comment in the function.
2016-09-23journal: fix HMAC calculation when appending a data objectFranck Bui
Since commit 5996c7c295e073ce21d41305169132c8aa993ad0 (v190 !), the calculation of the HMAC is broken because the hash for a data object including a field is done in the wrong order: the field object is hashed before the data object is. However during verification, the hash is done in the opposite order as objects are scanned sequentially.
2016-09-23journal: warn when we fail to append a tag to a journalFranck Bui
We shouldn't silently fail when appending the tag to a journal file since FSS protection will simply be disabled in this case.
2016-09-18journal: fix typo in comment (#4176)Felix Zhang
2016-09-17Merge pull request #4123 from keszybz/network-file-dropinsMartin Pitt
Network file dropins