summaryrefslogtreecommitdiff
path: root/src/journal
AgeCommit message (Collapse)Author
2016-04-29journal-file: when rotating a journal file, fsync directory tooLennart Poettering
As suggested by: https://github.com/systemd/systemd/pull/3126#discussion_r61125474
2016-04-29journal: when creating a new journal file, fsync() the directory it is ↵Lennart Poettering
created in too Fixes: #2831
2016-04-27Correctly parse OBJECT_PID in journald messages (#3129)Nalin Dahyabhai
The parse_pid() function doesn't succeed if we don't zero-terminate after the last digit in the buffer.
2016-04-27journal: set STATE_ARCHIVED as part of offlining (#2740)Vito Caputo
The only code path which makes a journal durable is via journal_file_set_offline(). When we perform a rotate the journal's header->state is being set to STATE_ARCHIVED prior to journal_file_set_offline() being called. In journal_file_set_offline(), we short-circuit the entire offline when f->header->state != STATE_ONLINE. This all results in none of the journal_file_set_offline() fsync() calls being reached when rotate archives a journal, so archived journals are never explicitly made durable. What we do now is instead of setting the f->header->state to STATE_ARCHIVED directly in journal_file_rotate() prior to journal_file_close(), we set an archive flag in f->archive for the journal_file_set_offline() machinery to honor by committing STATE_ARCHIVED instead of STATE_OFFLINE when set. Prior to this, rotated journals were never getting fsync() explicitly performed on them, since journal_file_set_offline() short-circuited. Obviously this is undesirable, and depends entirely on the underlying filesystem as to how much durability was achieved when simply closing the file. Note that this problem existed prior to the recent asynchronous fsync changes, but those changes do facilitate our performing this durable offline on rotate without blocking, regardless of the underlying filesystem sync-on-close semantics.
2016-04-26journal-file: make seeking in corrupted files workLennart Poettering
Previously, when we used a bisection table for seeking through a corrupted file, and the end of the bisection table was corrupted we'd most likely fail the entire seek operation. Improve the situation: if we encounter invalid entries in a bisection table, linearly go backwards until we find a working entry again.
2016-04-26journal-file: when iterating through a partly corruped journal file, treat ↵Lennart Poettering
error like EOF When we linearly iterate through a corrupted journal file, and we encounter a read error, don't consider this fatal, but merely as EOF condition (and log about it).
2016-04-26journal-file: always generate the same error when encountering corrupted filesLennart Poettering
Let's make sure EBADMSG is the one error we throw when we encounter corrupted data, so that we can neatly test for it.
2016-04-26sd-journal: detect earlier if we try to read an object from an invalid offsetLennart Poettering
Specifically, detect early if we try to read from offset 0, i.e. are using uninitialized offset data.
2016-04-25Merge pull request #3109 from poettering/journal-by-fdZbigniew Jędrzejewski-Szmek
rework "journalctl -M"
2016-04-25Merge pull request #3114 from poettering/journalctl-bZbigniew Jędrzejewski-Szmek
Fix endless loops in journalctl --list-boots (closes #617).
2016-04-25journal: fix already offline check and thread leak (#2810)Vito Caputo
Early in journal_file_set_offline() f->header->state is tested to see if it's != STATE_ONLINE, and since there's no need to do anything if the journal isn't online, the function simply returned here. Since moving part of the offlining process to a separate thread, there are two problems here: 1. We can't simply check f->header->state, because if there is an offline thread active it may modify f->header->state. 2. Even if the journal is deemed offline, the thread responsible may still need joining, so a bare return may leak the thread's resources like its stack. To address #1, the helper journal_file_is_offlining() is called prior to accessing f->header->state. If journal_file_is_offlining() returns true, f->header->state isn't even checked, because an offlining journal is obviously online, and we'll just continue with the normal set offline code path. If journal_file_is_offlining() returns false, then it's safe to check f->header->state, because the offline_state is beyond the point of modifying f->header->state, and there's a memory barrier in the helper. If we find f->header->state is != STATE_ONLINE, then we call the idempotent journal_file_set_offline_thread_join() on the way out of the function, to join a potential lingering offline thread.
2016-04-25journalctl: turn --unit= in combination with --user into --user-unit=Lennart Poettering
Let's be nice to users, and let's turn the nonsensical "--unit=… --user" into "--user-unit=…" which the user more likely meant. Fixes #1621
2016-04-25sd-journal: "soft" deprecate sd_journal_open_container()Lennart Poettering
Let's document the call as deprecated, since it doesn't cover containers with directories that aren#t visible to the host properly.
2016-04-25journalctl: port --machine= switch to use machined's OpenMachineRootDirectory()Lennart Poettering
This way, the switch becomes compatible with nspawn containers using --image=, and those which only store journal data in /run (i.e. have persistant logs off). Fixes: #49
2016-04-25journalctl: don't trust the per-field entry tables when looking for boot IDsLennart Poettering
When appending to a journal file, journald will: a) first, append the actual entry to the end of the journal file b) second, add an offset reference to it to the global entry array stored at the beginning of the file c) third, add offset references to it to the per-field entry array stored at various places of the file The global entry array, maintained by b) is used when iterating through the journal without matches applied. The per-field entry array maintained by c) is used when iterating through the journal with a match for that specific field applied. In the wild, there are journal files where a) and b) were completed, but c) was not before the files were abandoned. This means, that in some cases log entries are at the end of these files that appear in the global entry array, but not in the per-field entry array of the _BOOT_ID= field. Now, the "journalctl --list-boots" command alternatingly uses the global entry array and the per-field entry array of the _BOOT_ID= field. It seeks to the last entry of a specific _BOOT_ID=field by having the right match installed, and then jumps to the next following entry with no match installed anymore, under the assumption this would bring it to the next boot ID. However, if the per-field entry wasn't written fully, it might actually turn out that the global entry array might know one more entry with the same _BOOT_ID, thus resulting in a indefinite loop around the same _BOOT_ID. This patch fixes that, by updating the boot search logic to always continue reading entries until the boot ID actually changed from the previous. Thus, the per-field entry array is used as quick jump index (i.e. as an optimization), but not trusted otherwise. Only the global entry array is trusted. This replaces PR #1904, which is actually very similar to this one. However, this one actually reads the boot ID directly from the entry header, and doesn't try to read it at all until the read pointer is actually really located on the first item to read. Fixes: #617 Replaces: #1904
2016-04-25journalctl: improve output of --header a bitLennart Poettering
Show the various timestamps in hexadecimal too. This is useful for matching the timestamps included in cursor strings (which are encoded in hex, too), with the references in the journal header.
2016-04-25journalctl: simplify discover_next_boot() a bitLennart Poettering
Drop the "read_realtime" parameter. Getting the realtime timestamp from an entry is cheap, as it is a normal header field, hence let's just get this unconditionally, and simplify our code a bit.
2016-04-25journalctl: simplify get_boots() a bit, by getting rid of one BootId objectLennart Poettering
Let's store the reference as simple sd_id128_t, since we don't actually need a BootId for it.
2016-04-25journalctl: add some explanatory comments to get_boots()Lennart Poettering
2016-04-25sd-journal: add logic to open journal files of a specific OS treeLennart Poettering
With this change a new flag SD_JOURNAL_OS_ROOT is introduced. If specified while opening the journal with the per-directory calls (specifically: sd_journal_open_directory() and sd_journal_open_directory_fd()) the passed directory is assumed to be the root directory of an OS tree, and the journal files are searched for in /var/log/journal, /run/log/journal relative to it. This is useful to allow usage of sd-journal on file descriptors returned by the OpenRootDirectory() call of machined.
2016-04-25sd-journal: add API for opening journal files or directories by fdLennart Poettering
Also, expose this via the "journalctl --file=-" syntax for STDIN. This feature remains undocumented though, as it is probably not too useful in real-life as this still requires fds that support mmaping and seeking, i.e. does not work for pipes, for which reading from STDIN is most commonly used.
2016-04-25sd-journal: minor simplificationLennart Poettering
2016-04-25journalctl: improve error message when we have trouble reading journal filesLennart Poettering
Let's output the actual error code encountered, and let's not claim this was purely triggered by files, because it can also be triggered by directories.
2016-04-25sd-journal: properly collect errors from readdir()Lennart Poettering
Let's also collect errors returned by readdir() into our set of errors, like we do this for all other errors from journal files.
2016-04-25journal: add inotify watches by-fd instead of by-pathLennart Poettering
This is slightly nicer, since we actually watch the directories we opened and enumerate. However, primarily this is preparation for adding support for opening journal files by fd without specifying any path, to be added in a later commit.
2016-04-22journalctl: add --no-hostname switchLennart Poettering
This suppresses output of the hostname for messages from the local system. Fixes: #2342
2016-04-22coredump,basic: generalize O_TMPFILE handling a bitLennart Poettering
This moves the O_TMPFILE handling from the coredumping code into common library code, and generalizes it as open_tmpfile_linkable() + link_tmpfile(). The existing open_tmpfile() function (which creates an unlinked temporary file that cannot be linked into the fs) is renamed to open_tmpfile_unlinkable(), to make the distinction clear. Thus, code may now choose between: a) open_tmpfile_linkable() + link_tmpfile() b) open_tmpfile_unlinkable() Depending on whether they want a file that may be linked back into the fs later on or not. In a later commit we should probably convert fopen_temporary() to make use of open_tmpfile_linkable(). Followup for: #3065
2016-04-21tree-wide: use mdash instead of a two minusesZbigniew Jędrzejewski-Szmek
2016-04-16journald: rewrite function with switch, fix handling of -ESHUTDOWNZbigniew Jędrzejewski-Szmek
The comments and the log messages are next to one another, so it's easier to check that the messages match the comments. The sign was omitted in the check for -ESHUTDOWN, so it was never matched.
2016-04-08journal: assert gcry_mpi_scan succeededZbigniew Jędrzejewski-Szmek
It might be nicer to propagate the error to the caller, but that'd be a bigger refactoring. This shouldn't really fail, so just add an assert. CID #1349697.
2016-04-02compress: fix gcc warnings about void* used in arithmeticZbigniew Jędrzejewski-Szmek
src/journal/compress.c: In function ‘compress_blob_lz4’: src/journal/compress.c:115:49: warning: pointer of type ‘void *’ used in arithmetic [-Wpointer-arith] r = LZ4_compress_limitedOutput(src, dst + 8, src_size, (int) dst_alloc_size - 8); ^ src/journal/compress.c: In function ‘decompress_blob_xz’: src/journal/compress.c:179:35: warning: pointer of type ‘void *’ used in arithmetic [-Wpointer-arith] s.next_out = *dst + used; ^ src/journal/compress.c: In function ‘decompress_blob_lz4’: src/journal/compress.c:218:37: warning: pointer of type ‘void *’ used in arithmetic [-Wpointer-arith] r = LZ4_decompress_safe(src + 8, out, src_size - 8, size); ^ src/journal/compress.c: In function ‘decompress_startswith_xz’: src/journal/compress.c:294:38: warning: pointer of type ‘void *’ used in arithmetic [-Wpointer-arith] s.next_out = *buffer + *buffer_size - s.avail_out; ^ src/journal/compress.c:294:53: warning: pointer of type ‘void *’ used in arithmetic [-Wpointer-arith] s.next_out = *buffer + *buffer_size - s.avail_out; ^ src/journal/compress.c: In function ‘decompress_startswith_lz4’: src/journal/compress.c:327:45: warning: pointer of type ‘void *’ used in arithmetic [-Wpointer-arith] r = LZ4_decompress_safe_partial(src + 8, *buffer, src_size - 8, ^ LZ4 and XZ functions use char* and unsigned char*, respectively, so keep void* in our internal APIs and add casts.
2016-04-02test-compress-benchmark: fix argument parsing on 32bitZbigniew Jędrzejewski-Szmek
The patch is not minimal, but a function to parse size_t is probably going to come in handy in other places, so I think it's nicer to define a proper parsing function than to open-code the cast.
2016-03-04test-compress-benchmark: skip loop iteration if size is 0Zbigniew Jędrzejewski-Szmek
Otherwise we would hit an assert in the compression code.
2016-02-29Use `PRIu64` to print `uint64_t` in log msgsElias Probst
2016-02-25Merge pull request #2671 from 0xAX/move-pager-open-to-one-placeZbigniew Jędrzejewski-Szmek
tree-wide: merge pager_open_if_enabled() to the pager_open()
2016-02-26tree-wide: merge pager_open_if_enabled() to the pager_open()Alexander Kuleshov
Many subsystems define own pager_open_if_enabled() function which checks '--no-pager' command line argument and open pager depends on its value. All implementations of pager_open_if_enabled() are the same. Let's merger this function with pager_open() from the shared/pager.c and remove pager_open_if_enabled() from all subsytems to prevent code duplication.
2016-02-23tree-wide: minor formatting inconsistency cleanupsVito Caputo
2016-02-23Merge pull request #2708 from vcaputo/journal-restore-offline-state-on-errorLennart Poettering
journal: restore offline state on error
2016-02-22tree-wide: make ++/-- usage consistent WRT spacingVito Caputo
Throughout the tree there's spurious use of spaces separating ++ and -- operators from their respective operands. Make ++ and -- operator consistent with the majority of existing uses; discard the spaces.
2016-02-22journal: restore offline state on errorVito Caputo
If we fail to create the thread, technically we should leave the offline_state as OFFLINE_JOINED, not OFFLINE_SYNCING.
2016-02-19journal: defer journal closes on rotateVito Caputo
When we rotate journals, we must set offline and close the current one, but don't generally need to wait for this to complete. Instead, we'll initiate an asynchronous offline via journal_file_set_offline(oldfile, false), and add the file to a per-server set of deferred closes to be closed later when they won't block. There's one complication however; journal_file_open() via journal_file_verify_header() assumes that any writable journal in the online state is the product of an unclean shutdown or other form of corruption. Thus there's a need for journal_file_open() to be aware of deferred closes and synchronize with their completion when opening preexisting journals for writing. To facilitate this the deferred closes set is supplied to the journal_file_open() function where the deferred closes may be closed synchronously before verifying the header in such circumstances.
2016-02-19journal: asynchronous journal_file_set_offline()Vito Caputo
This adds a wait flag to journal_file_set_offline(), when false the offline is performed asynchronously in a separate thread. When wait is true, if an asynchronous offline is already in-progress it is restarted and waited for. Otherwise the offline is performed synchronously without the use of a thread. journal_file_set_online() cancels or waits for the asynchronous offline to complete if in-flight, depending on where in the offline process the thread happens to be. If the thread is in the fsync() phase, it is cancelled and waiting is unnecessary. Otherwise, the thread is joined before proceeding. A new offline_state member is added to JournalFile which is used via atomic operations for communicating between the offline thread and the journal_file_set_{offline,online}() functions.
2016-02-19journal: add void cast to journal_file_close() callsVito Caputo
2016-02-19journal: add void cast to fsync() callsVito Caputo
2016-02-18journal/catalog: fix memory leaksZbigniew Jędrzejewski-Szmek
Various buffers were lost because finish_item() either consumed the buffer or allocated a new one (if an entry with the same key existed). The caller would simply forget the buffer in either case. Also add a check for the case when a valid identifier is followed by an empty body. We should not allow this. Also be more consistent in error handling and always print an error message.
2016-02-13Merge pull request #2589 from keszybz/resolve-tool-2Lennart Poettering
Better support of OPENPGPKEY, CAA, TLSA packets and tests
2016-02-11Move initialize_libgcrypt to separate fileZbigniew Jędrzejewski-Szmek
It's annoying to have the exact same function in three places. It's stored in src/shared, but it's not added to the library to avoid the dependency on libgcrypt.
2016-02-11Add memcpy_safeZbigniew Jędrzejewski-Szmek
ISO/IEC 9899:1999 §7.21.1/2 says: Where an argument declared as size_t n specifies the length of the array for a function, n can have the value zero on a call to that function. Unless explicitly stated otherwise in the description of a particular function in this subclause, pointer arguments on such a call shall still have valid values, as described in 7.1.4. In base64_append_width memcpy was called as memcpy(x, NULL, 0). GCC 4.9 started making use of this and assumes This worked fine under -O0, but does something strange under -O3. This patch fixes a bug in base64_append_width(), fixes a possible bug in journal_file_append_entry_internal(), and makes use of the new function to simplify the code in other places.
2016-02-10build-sys: move coredump logic into subdir of its ownLennart Poettering
2016-02-10tree-wide: remove Emacs lines from all filesDaniel Mack
This should be handled fine now by .dir-locals.el, so need to carry that stuff in every file.