summaryrefslogtreecommitdiff
path: root/src/journal-remote
AgeCommit message (Collapse)Author
2016-10-10journal-remote: make the child pipe non-blockingDan Dedrick
We are going to add this child as a source to our event loop so we don't want to block when reading data from it as this will prevent us from processing other events. Specifically this will block the signalfds which means if we are waiting for data from curl we won't handle SIGTERM or SIGINT until we happen to get more data.
2016-09-24journal-remote: fix error format stringZbigniew Jędrzejewski-Szmek
Bug introduced in 1b4cd64683.
2016-09-17Merge pull request #4123 from keszybz/network-file-dropinsMartin Pitt
Network file dropins
2016-09-16tree-wide: rename config_parse_many to …_nulstrZbigniew Jędrzejewski-Szmek
In preparation for adding a version which takes a strv.
2016-09-13microhttpd-util: add the trailing newline automaticallyZbigniew Jędrzejewski-Szmek
It's prone to error and annoying to have to add it manually. It was missing from a few places.
2016-09-13journal-remote: implement %m support in mhd_respondfZbigniew Jędrzejewski-Szmek
errno value is not protected (it is undefined after this function returns). Various mhd_* functions are not documented to protect errno, so this could not guaranteed anyway.
2016-08-09journal-gatewayd: fix typo; s/Certificat/Certificate/ (#3931)Vito Caputo
2016-08-06journal-gatewayd: fix segfault with certain request (#3893)Yi EungJun
When client requests to get logs with `follow` and `KEY=match` that doesn't match any log entry, journal-gatewayd segfaulted. Make request_reader_entries to return zero in such case to wait for matching entries. This fixes https://github.com/systemd/systemd/issues/3873.
2016-08-06journal-gatewayd: add --directory option (#3913)Yi EungJun
Serve journals in the specified directory instead of default journals.
2016-08-05journal-remote: remove unnecessary gnutls includes (#3895)Vito Caputo
journal-(gatewayd,remote).c don't actually utilize libgnutls even when HAVE_GNUTLS is defined.
2016-07-31Add enable_disable() helperZbigniew Jędrzejewski-Szmek
In this patch "enabled" and "disabled" is used exclusively, but "enable" and "disable" forms are need for the following patch.
2016-07-12Various fixes for typos found by lintian (#3705)Michael Biebl
2016-04-22journal-remote : Ensure reallocation of source->buf does not result in ↵Klearchos Chaloulos
source->size < source->filled (#3086) While the function journal-remote-parse.c:get_line() enforces an assertion that source->filled <= source->size, in function journal-remote-parse.c:process_source() there is a chance that source->size will be decreased to a lower value than source->filled, when source->buf is reallocated. Therefore a check is added that ensures that source->buf is reallocated only when source->filled is smaller than target / 2.
2016-04-22coredump,basic: generalize O_TMPFILE handling a bitLennart Poettering
This moves the O_TMPFILE handling from the coredumping code into common library code, and generalizes it as open_tmpfile_linkable() + link_tmpfile(). The existing open_tmpfile() function (which creates an unlinked temporary file that cannot be linked into the fs) is renamed to open_tmpfile_unlinkable(), to make the distinction clear. Thus, code may now choose between: a) open_tmpfile_linkable() + link_tmpfile() b) open_tmpfile_unlinkable() Depending on whether they want a file that may be linked back into the fs later on or not. In a later commit we should probably convert fopen_temporary() to make use of open_tmpfile_linkable(). Followup for: #3065
2016-04-05journal-upload: make watchdog state non-staticZbigniew Jędrzejewski-Szmek
Also parse watchdog config when creating the Uploader object.
2016-04-05journal-upload: Update watchdog while in curl_easy_performKlearchos Chaloulos
It is observed that a combination of high log throughput, low I/O speed on journal remote side and many nodes uploading simultaneously caused the journal-upload process to dump core because of watchdog starvation. This is caused because journal-upload stays in curl_easy_perform(), because it cannot upload fast enough to reach the end of the journal. Currently journal-upload will return from curl_easy_perform() only when the end of the journal is reached. Therefore a check is added in journal_input_callback(), which will update the watchdog if the elapsed time since the start of the uploading process is greater than WATCHDOG_USEC/2.
2016-02-23tree-wide: minor formatting inconsistency cleanupsVito Caputo
2016-02-22tree-wide: make ++/-- usage consistent WRT spacingVito Caputo
Throughout the tree there's spurious use of spaces separating ++ and -- operators from their respective operands. Make ++ and -- operator consistent with the majority of existing uses; discard the spaces.
2016-02-22Merge pull request #2685 from poettering/lldp-fixes2Tom Gundersen
lldp fixes, second iteration
2016-02-20tree-wide: place #pragma once at the same place everywhereLennart Poettering
Usually, we place the #pragma once before the copyright blurb in header files, but in a few cases we didn't. Move those around, so that we do the same thing everywhere.
2016-02-19journal: defer journal closes on rotateVito Caputo
When we rotate journals, we must set offline and close the current one, but don't generally need to wait for this to complete. Instead, we'll initiate an asynchronous offline via journal_file_set_offline(oldfile, false), and add the file to a per-server set of deferred closes to be closed later when they won't block. There's one complication however; journal_file_open() via journal_file_verify_header() assumes that any writable journal in the online state is the product of an unclean shutdown or other form of corruption. Thus there's a need for journal_file_open() to be aware of deferred closes and synchronize with their completion when opening preexisting journals for writing. To facilitate this the deferred closes set is supplied to the journal_file_open() function where the deferred closes may be closed synchronously before verifying the header in such circumstances.
2016-02-10tree-wide: remove Emacs lines from all filesDaniel Mack
This should be handled fine now by .dir-locals.el, so need to carry that stuff in every file.
2016-02-04journal-remote: decrease microhttpd memory limitKlearchos Chaloulos
Set the MHD_OPTION_CONNECTION_MEMORY_LIMIT to 128KB. The precious value was DATA_SIZE_MAX, which was defined as 1024*1024*768. This caused journal-remote to allocate 756MB for each journal-upload connection, thus exhausting the available memory.
2016-01-24Merge pull request #2425 from yuwata/journal-remote-v4Zbigniew Jędrzejewski-Szmek
ZJS: remove unnecessary oom check after strdupa().
2016-01-24journal-remote: fix broken --getter optionYu Watanabe
This commit fixes the following broken --getter option: when systemd-journal-remote is called with --getter option, it causes the error meesage "Zero sources specified" and the getter command will not be called.
2016-01-24journal-remote: output file name is determined by the remote hostnameYu Watanabe
When --url option is specified, e.g. --url='http://some.host:19531/entries' retrieved remote journal entries will be stored to /var/log/journal/remote/remote-some.host.journal
2016-01-24journal-remote: make --url option support arbitrary urlYu Watanabe
Currently, --url option supports the only form like http(s)://some.host:19531. This commit adds support to call systemd-journal-remote as follwos: systemd-journal-remote --url='http://some.host:19531' systemd-journal-remote --url='http://some.host:19531/' systemd-journal-remote --url='http://some.host:19531/entries' systemd-journal-remote --url='http://some.host:19531/entries?boot&follow' The first three example result the same and retrieve all entries. The last example retrieves only current boot entries and wait new events.
2016-01-22Merge pull request #2080 from ↵Daniel Mack
chaloulo/split-mode-host-remove-port-from-journal-filename journal-remote: split-mode=host, remove port from journal filename
2016-01-22systemd-activate: Add support for datagram socketsSusant Sahani
core: Add flexible way to provide socket type the socket type should be a diffrent argumet in make_socket_fd . In this way we can set the socket type like SOCK_STREAM SOCK_DGRAM in the address. journal-remote: modify make_socket_fd
2016-01-21Merge pull request #2071 from chaloulo/journal-upload-miss-logsDaniel Mack
journal-upload : Ignore journal event when already in uploading state.
2016-01-20journal-gatewayd: fix offsetZbigniew Jędrzejewski-Szmek
I was checking something when writing the patch and committed this by mistake.
2016-01-18journal-remote: update to 64bit µhttp api, provide fallbackZbigniew Jędrzejewski-Szmek
64 bit offset is now accepted, which is nice. The old function is deprecated, and generates a compile time warning when used. We only use an offset of 0, so we really don't care. Adapt to use the new function, but fall back to the old one on older versions.
2016-01-18journal-remote: fix warning about deprecated µhttpd macroZbigniew Jędrzejewski-Szmek
src/journal-remote/journal-remote.c:590:13: warning: Value MHD_HTTP_METHOD_NOT_ACCEPTABLE is deprecated, use MHD_HTTP_NOT_ACCEPTABLE return mhd_respond(connection, MHD_HTTP_METHOD_NOT_ACCEPTABLE, ^ The new define was added in 0.9.38. Instead of requiring the new libmicrohttpd version, provide the fallback, it is trivial.
2016-01-08Merge pull request #2287 from dandedrick/journal-gatewayd-timeout-fixDaniel Mack
journal-gatewayd: timeout journal wait to allow thread cleanup
2015-12-20Add Seal option in the configuration file for journald-remoteMichael Scherer
While journal received remotely can be sealed, it can only be done on the command line using --seal, so for consistency, we will also permit to set it in the configuration file.
2015-12-14journal-gatewayd: timeout journal wait to allow thread cleanupDan Dedrick
When a client connects with follow=1 and then disconnects we can get stuck in sd_journal_wait indefinitely if no journal messages are logged. Every time a client does this another thread is allocated and these continue to stack until either a journal message is logged or we run out of mapping to put a stack in. By adding a timeout if we don't see any journal messages in that timeout we will simply pop back out to microhttpd which will sanity check the connection for us and if it is still connected pop us back into the wait for more journal messages.
2015-12-09journal-upload : Ignore journal event when already in uploading state.Klearchos Chaloulos
When the log rate is high, it is possible that the callback dispatch_journal_input() will be called twice, while the program is in uploading state. There is a guard for this in dispatch_journal_input(). However it is not enough, as it is possible that the uploading state is not set when the code is in dispatch_journal_input(). The result of the above is that a log would be skipped, as sd_journal_next_skip() would be called twice. Adding a new check in process_journal_input(), just before the code to sd_journal_next_skip(), makes sure that the code ignores a duplicate callback, when the first callback is in uploading state. Also, removed the warning log from dispatch_journal_input(), as this occurence is normal.
2015-12-01journal-remote: split-mode=host, remove port from journal filenameKlearchos Chaloulos
When constructing the journal filename to store logs from a remote host, remove the port of the tcp connection, as the port will change with every reboot/connection loss between sender/reveiver machines. Having the port in the filename will cause a new journal file to be created for every reboot or connection loss. For the implementation, a new argument "bool include_port" is added to the getpeername_pretty() function. This is passed to the sockaddr_pretty() function. The value of the include_port argument is set to true in all calls of getpeername_pretty(), except for 2 calls in journal-remote.c, where it is set to false.
2015-11-27tree-wide: expose "p"-suffix unref calls in public APIs to make gcc cleanup easyLennart Poettering
GLIB has recently started to officially support the gcc cleanup attribute in its public API, hence let's do the same for our APIs. With this patch we'll define an xyz_unrefp() call for each public xyz_unref() call, to make it easy to use inside a __attribute__((cleanup())) expression. Then, all code is ported over to make use of this. The new calls are also documented in the man pages, with examples how to use them (well, I only added docs where the _unref() call itself already had docs, and the examples, only cover sd_bus_unrefp() and sd_event_unrefp()). This also renames sd_lldp_free() to sd_lldp_unref(), since that's how we tend to call our destructors these days. Note that this defines no public macro that wraps gcc's attribute and makes it easier to use. While I think it's our duty in the library to make our stuff easy to use, I figure it's not our duty to make gcc's own features easy to use on its own. Most likely, client code which wants to make use of this should define its own: #define _cleanup_(function) __attribute__((cleanup(function))) Or similar, to make the gcc feature easier to use. Making this logic public has the benefit that we can remove three header files whose only purpose was to define these functions internally. See #2008.
2015-11-18tree-wide: sort includes in *.hThomas Hindoe Paaboel Andersen
This is a continuation of the previous include sort patch, which only sorted for .c files.
2015-11-16tree-wide: sort includesThomas Hindoe Paaboel Andersen
Sort the includes accoding to the new coding style.
2015-11-10defs: rework CONF_DIRS_NULSTR() macroLennart Poettering
The macro is generically useful for putting together search paths, hence let's make it truly generic, by dropping the implicit ".d" appending it does, and leave that to the caller. Also rename it from CONF_DIRS_NULSTR() to CONF_PATHS_NULSTR(), since it's not strictly about dirs that way, but any kind of file system path. Also, mark CONF_DIR_SPLIT_USR() as internal macro by renaming it to _CONF_PATHS_SPLIT_USR() so that the leading underscore indicates that it's internal.
2015-11-05treewide: use the negative error codes returned by our functionsMichal Schmidt
Our functions return negative error codes. Do not rely on errno being set after calling our own functions.
2015-11-03util-lib: move CONF_DIRS_NULSTR definition to def.hLennart Poettering
After all, this is not some compiler or C magic, but something very specific to how systemd works, hence let's move it into def.h, and out of macro.h
2015-11-03log-generator: add option to generate easily compressible dataZbigniew Jędrzejewski-Szmek
This is useful to check that compression actually works, and how compression influences file size in the best-case-scenario for compression. (The answer is that not as much as one would hope: there's still a big overhead of the indexing and since every field is compressed separately, even fields that compress very well contribute to the file size. This overhead becomes negligible only for very big fields.)
2015-11-01journal-remote: remove unused variable warning when building without GNUTLS.Lennart Poettering
2015-10-30journal-remote: increase memory limit per connection to avoid errors with HTTPSDongsu Park
Explicitly set MHD_OPTION_CONNECTION_MEMORY_LIMIT to a larger value, when setting up microhttpd, to give more memory per HTTP(S) connection. This way systemd-journal-remote can now prevent microhttpd from failing in creating response headers with messages like "Not enough memory for write", especially when lots of HTTPS requests arrive. That's precisely because MHD_OPTION_CONNECTION_MEMORY_LIMIT in libmicrohttpd defaults to 32768, which is in practice insufficient in this case. See also https://gnunet.org/bugs/view.php?id=4007 for more details. Fixes: https://github.com/coreos/bugs/issues/927
2015-10-28journal-remote: port to extract_first_wordSusant Sahani
2015-10-27util-lib: split out allocation calls into alloc-util.[ch]Lennart Poettering
2015-10-27util-lib: split out globbing related calls into glob-util.[ch]Lennart Poettering