summaryrefslogtreecommitdiff
path: root/src/journal-remote
AgeCommit message (Collapse)Author
2016-07-12Various fixes for typos found by lintian (#3705)Michael Biebl
2016-04-22journal-remote : Ensure reallocation of source->buf does not result in ↵Klearchos Chaloulos
source->size < source->filled (#3086) While the function journal-remote-parse.c:get_line() enforces an assertion that source->filled <= source->size, in function journal-remote-parse.c:process_source() there is a chance that source->size will be decreased to a lower value than source->filled, when source->buf is reallocated. Therefore a check is added that ensures that source->buf is reallocated only when source->filled is smaller than target / 2.
2016-04-22coredump,basic: generalize O_TMPFILE handling a bitLennart Poettering
This moves the O_TMPFILE handling from the coredumping code into common library code, and generalizes it as open_tmpfile_linkable() + link_tmpfile(). The existing open_tmpfile() function (which creates an unlinked temporary file that cannot be linked into the fs) is renamed to open_tmpfile_unlinkable(), to make the distinction clear. Thus, code may now choose between: a) open_tmpfile_linkable() + link_tmpfile() b) open_tmpfile_unlinkable() Depending on whether they want a file that may be linked back into the fs later on or not. In a later commit we should probably convert fopen_temporary() to make use of open_tmpfile_linkable(). Followup for: #3065
2016-04-05journal-upload: make watchdog state non-staticZbigniew Jędrzejewski-Szmek
Also parse watchdog config when creating the Uploader object.
2016-04-05journal-upload: Update watchdog while in curl_easy_performKlearchos Chaloulos
It is observed that a combination of high log throughput, low I/O speed on journal remote side and many nodes uploading simultaneously caused the journal-upload process to dump core because of watchdog starvation. This is caused because journal-upload stays in curl_easy_perform(), because it cannot upload fast enough to reach the end of the journal. Currently journal-upload will return from curl_easy_perform() only when the end of the journal is reached. Therefore a check is added in journal_input_callback(), which will update the watchdog if the elapsed time since the start of the uploading process is greater than WATCHDOG_USEC/2.
2016-02-23tree-wide: minor formatting inconsistency cleanupsVito Caputo
2016-02-22tree-wide: make ++/-- usage consistent WRT spacingVito Caputo
Throughout the tree there's spurious use of spaces separating ++ and -- operators from their respective operands. Make ++ and -- operator consistent with the majority of existing uses; discard the spaces.
2016-02-22Merge pull request #2685 from poettering/lldp-fixes2Tom Gundersen
lldp fixes, second iteration
2016-02-20tree-wide: place #pragma once at the same place everywhereLennart Poettering
Usually, we place the #pragma once before the copyright blurb in header files, but in a few cases we didn't. Move those around, so that we do the same thing everywhere.
2016-02-19journal: defer journal closes on rotateVito Caputo
When we rotate journals, we must set offline and close the current one, but don't generally need to wait for this to complete. Instead, we'll initiate an asynchronous offline via journal_file_set_offline(oldfile, false), and add the file to a per-server set of deferred closes to be closed later when they won't block. There's one complication however; journal_file_open() via journal_file_verify_header() assumes that any writable journal in the online state is the product of an unclean shutdown or other form of corruption. Thus there's a need for journal_file_open() to be aware of deferred closes and synchronize with their completion when opening preexisting journals for writing. To facilitate this the deferred closes set is supplied to the journal_file_open() function where the deferred closes may be closed synchronously before verifying the header in such circumstances.
2016-02-10tree-wide: remove Emacs lines from all filesDaniel Mack
This should be handled fine now by .dir-locals.el, so need to carry that stuff in every file.
2016-02-04journal-remote: decrease microhttpd memory limitKlearchos Chaloulos
Set the MHD_OPTION_CONNECTION_MEMORY_LIMIT to 128KB. The precious value was DATA_SIZE_MAX, which was defined as 1024*1024*768. This caused journal-remote to allocate 756MB for each journal-upload connection, thus exhausting the available memory.
2016-01-24Merge pull request #2425 from yuwata/journal-remote-v4Zbigniew Jędrzejewski-Szmek
ZJS: remove unnecessary oom check after strdupa().
2016-01-24journal-remote: fix broken --getter optionYu Watanabe
This commit fixes the following broken --getter option: when systemd-journal-remote is called with --getter option, it causes the error meesage "Zero sources specified" and the getter command will not be called.
2016-01-24journal-remote: output file name is determined by the remote hostnameYu Watanabe
When --url option is specified, e.g. --url='http://some.host:19531/entries' retrieved remote journal entries will be stored to /var/log/journal/remote/remote-some.host.journal
2016-01-24journal-remote: make --url option support arbitrary urlYu Watanabe
Currently, --url option supports the only form like http(s)://some.host:19531. This commit adds support to call systemd-journal-remote as follwos: systemd-journal-remote --url='http://some.host:19531' systemd-journal-remote --url='http://some.host:19531/' systemd-journal-remote --url='http://some.host:19531/entries' systemd-journal-remote --url='http://some.host:19531/entries?boot&follow' The first three example result the same and retrieve all entries. The last example retrieves only current boot entries and wait new events.
2016-01-22Merge pull request #2080 from ↵Daniel Mack
chaloulo/split-mode-host-remove-port-from-journal-filename journal-remote: split-mode=host, remove port from journal filename
2016-01-22systemd-activate: Add support for datagram socketsSusant Sahani
core: Add flexible way to provide socket type the socket type should be a diffrent argumet in make_socket_fd . In this way we can set the socket type like SOCK_STREAM SOCK_DGRAM in the address. journal-remote: modify make_socket_fd
2016-01-21Merge pull request #2071 from chaloulo/journal-upload-miss-logsDaniel Mack
journal-upload : Ignore journal event when already in uploading state.
2016-01-20journal-gatewayd: fix offsetZbigniew Jędrzejewski-Szmek
I was checking something when writing the patch and committed this by mistake.
2016-01-18journal-remote: update to 64bit µhttp api, provide fallbackZbigniew Jędrzejewski-Szmek
64 bit offset is now accepted, which is nice. The old function is deprecated, and generates a compile time warning when used. We only use an offset of 0, so we really don't care. Adapt to use the new function, but fall back to the old one on older versions.
2016-01-18journal-remote: fix warning about deprecated µhttpd macroZbigniew Jędrzejewski-Szmek
src/journal-remote/journal-remote.c:590:13: warning: Value MHD_HTTP_METHOD_NOT_ACCEPTABLE is deprecated, use MHD_HTTP_NOT_ACCEPTABLE return mhd_respond(connection, MHD_HTTP_METHOD_NOT_ACCEPTABLE, ^ The new define was added in 0.9.38. Instead of requiring the new libmicrohttpd version, provide the fallback, it is trivial.
2016-01-08Merge pull request #2287 from dandedrick/journal-gatewayd-timeout-fixDaniel Mack
journal-gatewayd: timeout journal wait to allow thread cleanup
2015-12-20Add Seal option in the configuration file for journald-remoteMichael Scherer
While journal received remotely can be sealed, it can only be done on the command line using --seal, so for consistency, we will also permit to set it in the configuration file.
2015-12-14journal-gatewayd: timeout journal wait to allow thread cleanupDan Dedrick
When a client connects with follow=1 and then disconnects we can get stuck in sd_journal_wait indefinitely if no journal messages are logged. Every time a client does this another thread is allocated and these continue to stack until either a journal message is logged or we run out of mapping to put a stack in. By adding a timeout if we don't see any journal messages in that timeout we will simply pop back out to microhttpd which will sanity check the connection for us and if it is still connected pop us back into the wait for more journal messages.
2015-12-09journal-upload : Ignore journal event when already in uploading state.Klearchos Chaloulos
When the log rate is high, it is possible that the callback dispatch_journal_input() will be called twice, while the program is in uploading state. There is a guard for this in dispatch_journal_input(). However it is not enough, as it is possible that the uploading state is not set when the code is in dispatch_journal_input(). The result of the above is that a log would be skipped, as sd_journal_next_skip() would be called twice. Adding a new check in process_journal_input(), just before the code to sd_journal_next_skip(), makes sure that the code ignores a duplicate callback, when the first callback is in uploading state. Also, removed the warning log from dispatch_journal_input(), as this occurence is normal.
2015-12-01journal-remote: split-mode=host, remove port from journal filenameKlearchos Chaloulos
When constructing the journal filename to store logs from a remote host, remove the port of the tcp connection, as the port will change with every reboot/connection loss between sender/reveiver machines. Having the port in the filename will cause a new journal file to be created for every reboot or connection loss. For the implementation, a new argument "bool include_port" is added to the getpeername_pretty() function. This is passed to the sockaddr_pretty() function. The value of the include_port argument is set to true in all calls of getpeername_pretty(), except for 2 calls in journal-remote.c, where it is set to false.
2015-11-27tree-wide: expose "p"-suffix unref calls in public APIs to make gcc cleanup easyLennart Poettering
GLIB has recently started to officially support the gcc cleanup attribute in its public API, hence let's do the same for our APIs. With this patch we'll define an xyz_unrefp() call for each public xyz_unref() call, to make it easy to use inside a __attribute__((cleanup())) expression. Then, all code is ported over to make use of this. The new calls are also documented in the man pages, with examples how to use them (well, I only added docs where the _unref() call itself already had docs, and the examples, only cover sd_bus_unrefp() and sd_event_unrefp()). This also renames sd_lldp_free() to sd_lldp_unref(), since that's how we tend to call our destructors these days. Note that this defines no public macro that wraps gcc's attribute and makes it easier to use. While I think it's our duty in the library to make our stuff easy to use, I figure it's not our duty to make gcc's own features easy to use on its own. Most likely, client code which wants to make use of this should define its own: #define _cleanup_(function) __attribute__((cleanup(function))) Or similar, to make the gcc feature easier to use. Making this logic public has the benefit that we can remove three header files whose only purpose was to define these functions internally. See #2008.
2015-11-18tree-wide: sort includes in *.hThomas Hindoe Paaboel Andersen
This is a continuation of the previous include sort patch, which only sorted for .c files.
2015-11-16tree-wide: sort includesThomas Hindoe Paaboel Andersen
Sort the includes accoding to the new coding style.
2015-11-10defs: rework CONF_DIRS_NULSTR() macroLennart Poettering
The macro is generically useful for putting together search paths, hence let's make it truly generic, by dropping the implicit ".d" appending it does, and leave that to the caller. Also rename it from CONF_DIRS_NULSTR() to CONF_PATHS_NULSTR(), since it's not strictly about dirs that way, but any kind of file system path. Also, mark CONF_DIR_SPLIT_USR() as internal macro by renaming it to _CONF_PATHS_SPLIT_USR() so that the leading underscore indicates that it's internal.
2015-11-05treewide: use the negative error codes returned by our functionsMichal Schmidt
Our functions return negative error codes. Do not rely on errno being set after calling our own functions.
2015-11-03util-lib: move CONF_DIRS_NULSTR definition to def.hLennart Poettering
After all, this is not some compiler or C magic, but something very specific to how systemd works, hence let's move it into def.h, and out of macro.h
2015-11-03log-generator: add option to generate easily compressible dataZbigniew Jędrzejewski-Szmek
This is useful to check that compression actually works, and how compression influences file size in the best-case-scenario for compression. (The answer is that not as much as one would hope: there's still a big overhead of the indexing and since every field is compressed separately, even fields that compress very well contribute to the file size. This overhead becomes negligible only for very big fields.)
2015-11-01journal-remote: remove unused variable warning when building without GNUTLS.Lennart Poettering
2015-10-30journal-remote: increase memory limit per connection to avoid errors with HTTPSDongsu Park
Explicitly set MHD_OPTION_CONNECTION_MEMORY_LIMIT to a larger value, when setting up microhttpd, to give more memory per HTTP(S) connection. This way systemd-journal-remote can now prevent microhttpd from failing in creating response headers with messages like "Not enough memory for write", especially when lots of HTTPS requests arrive. That's precisely because MHD_OPTION_CONNECTION_MEMORY_LIMIT in libmicrohttpd defaults to 32768, which is in practice insufficient in this case. See also https://gnunet.org/bugs/view.php?id=4007 for more details. Fixes: https://github.com/coreos/bugs/issues/927
2015-10-28journal-remote: port to extract_first_wordSusant Sahani
2015-10-27util-lib: split out allocation calls into alloc-util.[ch]Lennart Poettering
2015-10-27util-lib: split out globbing related calls into glob-util.[ch]Lennart Poettering
2015-10-27util-lib: split out printf() helpers to stdio-util.hLennart Poettering
2015-10-27util-lib: move string table stuff into its own string-table.[ch]Lennart Poettering
2015-10-27util-lib: split stat()/statfs()/stavfs() related calls into stat-util.[ch]Lennart Poettering
2015-10-27util-lib: split string parsing related calls from util.[ch] into parse-util.[ch]Lennart Poettering
2015-10-25util-lib: split out fd-related operations into fd-util.[ch]Lennart Poettering
There are more than enough to deserve their own .c file, hence move them over.
2015-10-24util-lib: split our string related calls from util.[ch] into its own file ↵Lennart Poettering
string-util.[ch] There are more than enough calls doing string manipulations to deserve its own files, hence do something about it. This patch also sorts the #include blocks of all files that needed to be updated, according to the sorting suggestions from CODING_STYLE. Since pretty much every file needs our string manipulation functions this effectively means that most files have sorted #include blocks now. Also touches a few unrelated include files.
2015-10-24util: split out escaping code into escape.[ch]Lennart Poettering
This really deserves its own file, given how much code this is now.
2015-10-05Merge pull request #1406 from blaskovic/journal-remote-typoTom Gundersen
journal-remote: typo in log_error when no sources are specified [tomegun: this was a pun, but let's not do that]
2015-09-30journal: make sure to set MHD_USE_PIPE_FOR_SHUTDOWN for libmicrohttpd serversLennart Poettering
This makes sure libmicrohttpd won't call shutdown() on our listening sockets, which make sure socket activation and re-activation will work cleanly. See: https://github.com/systemd/systemd/pull/1286 https://lists.gnu.org/archive/html/libmicrohttpd/2015-09/msg00014.html Fixes #1286
2015-09-29util: introduce common version() implementation and use it everywhereLennart Poettering
This also allows us to drop build.h from a ton of files, hence do so. Since we touched the #includes of those files, let's order them properly according to CODING_STYLE.
2015-09-29journal-remote: typo in log_error when no sources are specifiedBranislav Blaskovic