libfastjson 0.99.8 released
This is a new fork of the json-c library, which is optimized for liblognorm processing.
This release provides several fixes to libfastjson. Most notably is the bugfix for proper handling of constant key names. For more details, please refer to the changelog below.
Changelog:
0.99.8 2017-12-18
– make build under gcc7 with strict settings (warning==error)
– bugfix: constant key names not properly handled
if fjson_object_object_add_ex() is used with option
FJSON_OBJECT_KEY_IS_CONSTANT, fjson_object_object_del() will still
try to delete the key name. Depending on use, this can lead to
double-free, use-after-free or no problem.
see also https://github.com/rsyslog/rsyslog/issues/1839
closes https://github.com/rsyslog/libfastjson/issues/148
– fix potentially invalid return value of fjson_object_iter_begin
this could lead to callers doing improper opreations and thus
could lead to a segfault in callers
detected by Coverity scan, CID 198891
– fix small potential memory leak in json_tokener (unlinkely to occur)
detected by Coverity Scan, CID 198890
Download:
http://download.rsyslog.com/libfastjson/libfastjson-0.99.8.tar.gz
sha256sum: 3544c757668b4a257825b3cbc26f800f59ef3c1ff2a260f40f96b48ab1d59e07
As always, feedback is appreciated.
Best regards,
Florian Riedl
rsyslog 8.30.0 (v8-stable) released
We have released rsyslog 8.30.0.
This release features a large number of changes. First we should mention the new build requirements for libfastjson 0.99.7 and the build recommendation for imjournal being libsystemd-journal >= 234.
Notable changes are that (JSON) variables are now handled case-insensitive by default, imjournal being able to switch to persistent journal in runtime and the complete refactoring of mmanon. Also, a lot of improvements have been added to the error reporting as well as many bugfixes.
For a complete list of changes, fixes and enhancements, please visit the ChangeLog.
The packages will follow when they are finished.
https://github.com/rsyslog/rsyslog/blob/v8-stable/ChangeLog
Download:
http://www.rsyslog.com/downloads/download-v8-stable/
As always, feedback is appreciated.
Best regards,
Florian Riedl
libfastjson 0.99.7 released
This is a new fork of the json-c library, which is optimized for liblognorm processing.
This release adds the option for case-insensitive comparisons and removes userdata and custom-serialization functions. For more details, please refer to the changelog below.
Changelog:
0.99.7 2017-10-17
– added option for case-insensitive comparisons
This permits to search for json keys in a case-sensitive way. The default is “off”, as this is against the JSON spec. However, rsyslog needs this capability to increase usability inside the variable system.
We add a new API call to switch between case-sensitive and case-insensitive comparison, with case-sensitive being the default.
closes https://github.com/rsyslog/libfastjson/issues/142
– Removed userdata and custom-serialization functions
Reasoning (from pull request):
The library uses the concept of “userdata” and “custom serialization functions” that can be set from user space. However, to effectively make use of this feature, a user must have a deep understanding of the internal data representation of the library, which makes this feature not very useful.
But what is worse: internally, the library itself also sometimes assigns data to this userdata member (especially when working with doubles), and it also sometimes assigns alternative serialization functions. This makes the feature even more unusable, because as a user you never can know when the userdata pointer is save to use for your own settings, and when you must leave it alone because it is used by the library.
Long story short. In this pull request I got rid of the userdata pointer completely. The case where the library was using the “userdata” (for storing the original string representation of a parsed double) has been moved into the union that is already used for storing values.
see also: https://github.com/rsyslog/libfastjson/pull/141
Thanks to Emiel Bruijntjes for the patch.
Download:
http://download.rsyslog.com/libfastjson/libfastjson-0.99.7.tar.gz
sha256sum: 373cca41cad90fd288998696cd56cc72df75fb094db82b4e95d78fa61910f690
As always, feedback is appreciated.
Best regards,
Florian Riedl
libfastjson 0.99.6 released
This is a new fork of the json-c library, which is optimized for liblognorm processing.
This release fixes a build issue under Solaris.
Changelog:
0.99.6 2017-05-12
– fix a build issue under Solaris
Download:
http://download.rsyslog.com/libfastjson/libfastjson-0.99.6.tar.gz
sha256sum: 95e3d371c82b5d85c7294d46c923b3fee0384e1541a1556f0ca185170eb27270
As always, feedback is appreciated.
Best regards,
Florian Riedl
libfastjson 0.99.5 released
This is a new fork of the json-c library, which is optimized for liblognorm processing.
This release brings a lot of fixes and changes. Most notably, libfastjson now builds under AIX and under Solaris again.
Changelog:
0.99.5 2017-05-03
– fix tautology comparison in tautology in `fjson_object_iter_equal`
– made build under Solaris again
– made to build under AIX
Thanks to github user purnimam1 for the patch
– fix floating point representation when fractional part is missing
see also https://github.com/rsyslog/libfastjson/issues/126
Thanks to Jan Gerhards for the patch.
– m4: fix detection of atomics
In cross-compilation, it is impossible to run code at configure time to
detect the target specifics.
As such, AC_TRY_RUN fails miserably to detect reliably that atomic
intrisics are present in a toolchain, and decides they are not just
because this is cross-compilation.
Instead of AC_TRY_RUN, use AC_LINK_IFELSE that does not need to actually
run code, since all we’re interested in is whether the intrisics are
present (or not). Fix both the 32- and 64-bit variants, even if the
latter is not used currently.
Fixes build failures detected by the Buildroot autobuilders, like:
http://autobuild.buildroot.org/results/23a/23ac0e742ed3a70ae4d038f8c9eadc23e708f671/build-end.log
http://autobuild.buildroot.org/results/192/1923d0b570adba494f83747a9610ea6ec35f5223/build-end.log
and many other cases, espcially on architectures where such intrsics are
present, but where the toolchain does not have threads (and anyway, it
is much more efficient to use the intrisics rather than use mutexes).
Thanks to Yann E. MORIN for the patch.
– add fjson_object_dump() and fjson_object_write() functions
… that make it possible to dump the json tree without having to
dynamically allocate a string, and to write the tree to a FILE*.
NOTE: right now, most of the code is simply copied from the functions
that use the “printbuf” for writing the data. I have not touched the old
printbuf-implementation, because some other code may still rely on it.
However, in my opinion these printbuf-based functions (if it is desirable
to keep them in the first place) can now be re-implemented to use the more
flexible fjson_object_dump() function.
MAINTAINER NOTE: we need to performance-test any new implementation and will
do so. The results will ultimately decide which parts of the code remain in
the codebase.
Thanks to Emiel Bruijntjes for the patch.
Download:
http://download.rsyslog.com/libfastjson/libfastjson-0.99.5.tar.gz
sha256sum: 8aecdc111da2beebc09e99e2b51e13b70f219c5a095c38db953a75fde51779e9
As always, feedback is appreciated.
Best regards,
Florian Riedl
libfastjson 0.99.3 released
We have released libfastjson 0.99.3.
This is a new fork of the json-c library, which is optimized for liblognorm processing.
The changes are mostly for cleanup purposes. Also some bug have been fixed, like a potential segfault issue, “make distcheck” not working and more.
Changelog:
0.99.3 2016-07-11
– new dependency: autoconf-archive
– exit() is no longer called in unexpected situations
The previous code called exit on some occasions and did not
give the caller a chance to do any cleanup or handling on
it’s own. This has completely been removed. Note that it was
very unlikely that this problem affected a caller, as exit()
was only called under very rare circumstances (e.g. OOM).
– fjson_version now returns configure VERSION
This avoid inconsistency.
– removal of Windows and Android bits
Thanks to Michael Biebl for the patch.
– fixes of the build system
Thanks to Michael Biebl for the patch.
– dropped support for Windows and Android as we do not target
these platforms
– “make distcheck” now works
– fix invalid Unicode representation for some non US-ASCII
characters when printed as string. Note that this could
potentially also lead to a segfault
Download:
http://download.rsyslog.com/libfastjson/libfastjson-0.99.3.tar.gz
sha256sum: be5604a9f61b3b247b45bc6234e8bfb5da5dd1e405d07fb8e52b1ea6e2a27fee
As always, feedback is appreciated.
Best regards,
Florian Riedl
rsyslog 8.17.0 (v8-stable) released
We have released rsyslog 8.17.0.
This release brings, among a few bugfixes, a lot of brand-new features. The most important change is probably the libfastjson requirement, which replaces the json-c dependency. There is a new contributed plugin called omampq1 for AMQP 1.0 compliant brokers, a new experimental lookup table support, dynamic statistics counters and many many more.
http://www.rsyslog.com/changelog-for-8-17-0-v8-stable/
Download:
http://www.rsyslog.com/downloads/download-v8-stable/
As always, feedback is appreciated.
Best regards,
Florian Riedl
Changelog for 8.17.0 (v8-stable)
Version 8.17.0 [v8-stable] 2016-03-08
- NEW REQUIREMENT: libfastjson
see also:
http://blog.gerhards.net/2015/12/rsyslog-and-liblognorm-will-switch-to.html - new testbench requirement: faketime command line tool
This is used to generate a controlled environment for time-based tests; if
not available, tests will gracefully be skipped. - improve json variable performance
We use libfastjson’s alternative hash function, which has been
proven to be much faster than the default one (which stems
back to libjson-c). This should bring an overall performance
improvement for all operations involving variable processing.
closes https://github.com/rsyslog/rsyslog/issues/848 - new experimental feature: lookup table suport
Note that at this time, this is an experimental feature which is not yet
fully supported by the rsyslog team. It is introduced in order to gain
more feedback and to make it available as early as possible because many
people consider it useful.
Thanks to Janmejay Singh for implementing this feature - new feature: dynamic statistics counters
which may be changed during rule processing
Thanks to Janmejay Singh for suggesting and implementing this feature - new contributed plugin: omampq1 for AMQP 1.0-compliant brokers
Thanks to Ken Giusti for this module - new set of UTC-based $now family of variables ($now-utc, $year-utc, …)
- simplified locking when accessing message and local variables
this simlifies the code and slightly increases performance if such
variables are heavily accessed. - new global parameter “debug.unloadModules”
This permits to disable unloading of modules, e.g. to make valgrind
reports more useful (without a need to recompile). - timestamp handling: guard against invalid dates
We do not permit dates outside of the year 1970..2100
interval. Note that network-receivers do already guard
against this, so the new guard only guards against invalid
system time. - imfile: add “trimlineoverbytes” input paramter
Thanks to github user JindongChen for the patch. - ommongodb: add support for extended json format for dates
Thanks to Florian Bücklers for the patch. - omjournal: add support for templates
see also: https://github.com/rsyslog/rsyslog/pull/770
Thanks to github user bobthemighty for the patch - imuxsock: add “ruleset” input parameter
- testbench: framework improvement: configs can be included in test file
they do no longer need to be in a separate file, which saves a bit
of work when working with them. This is supported for simple tests with
a single running rsyslog instance
Thanks to Janmejay Singh for inspiring me with a similar method in
liblognorm testbench. - imptcp: performance improvements
Thanks to Janmejay Singh for implementing this improvement - made build compile (almost) without warnings
still some warnings are suppressed where this is currently required - improve interface definition in some modules, e.g. mmanon, mmsequence
This is more an internal cleanup and should have no actual affect to
the end user. - solaris build: MAXHOSTNAMELEN properly detected
- build system improvement: ability to detect old hiredis libs
This permits to automatically build omhiredis on systems where the
hiredis libs do not provide a pkgconfig file. Previsouly, this
required manual configuration.
Thanks to github user jaymell for the patch. - rsgtutil: dump mode improvements
- auto-detect signature file type
- ability to dump hash chains for log extraction files
- build system: fix build issues with clang
clang builds often failed with a missing external symbol
“rpl_malloc”. This was caused by checks in configure.ac,
which checked for specific GNU semantics. As we do not need
them (we never ask malloc for zero bytes), we can safely
remove the macros.
Note that we routinely run clang static analyer in CI and
it also detects such calls as invalid.
closes https://github.com/rsyslog/rsyslog/issues/834 - bugfix: unixtimestamp date format was incorrectly computed
The problem happened in leap year from March til then end
of year and healed itself at the begining of the next year.
During the problem period, the timestamp was 24 hours too low.
fixes https://github.com/rsyslog/rsyslog/issues/830 - bugfix: date-ordinal date format was incorrectly computed
same root cause aus for unixtimestamp and same triggering
condition. During the affected perido, the ordinal was one
too less. - bugfix: some race when shutting down input module threads
this had little, if at all, effect on real deployments as it resulted
in a small leak right before rsyslog termination. However, it caused
trouble with the testbench (and other QA tools).
Thanks to Peter Portante for the patch and both Peter and Janmejay
Singh for helping to analyze what was going on. - bugfix tcpflood: did not handle connection drops correct in TLS case
note that tcpflood is a testbench too. The bug caused some testbench
instability, but had no effect on deplyments. - bugfix: abort if global parameter value was wrong
If so, the abort happened during startup. Once started,
all was stable. - bugfix omkafka: fix potential NULL pointer addressing
this happened when the topic cache was full and an entry
needed to be evicted - bugfix impstats: @cee cookie was prefixed to wrong fromat (json vs. cee)
Thanks to Volker Fröhlich for the fix. - bugfix imfile: fix race during startup that could lead to some duplication
If a to-be-monitored file was created after inotify was initialized
but before startup was completed, the first chunk of data from this
file could be duplicated. This should have happened very rarely in
practice, but caused occasional testbench failures.
see also: https://github.com/rsyslog/rsyslog/issues/791 - bugfix: potential loss of single message at queue shutdown
see also: https://github.com/rsyslog/rsyslog/issues/262 - bugfix: potential deadlock with heavy variable access
When making havy use of global, local and message variables, a deadlock
could occur. While it is extremly unlikely to happen, we have at least
seen one incarnation of this problem in practice. - bugfix ommysql: on some platforms, serverport parameter had no effect
This was caused by an invalid code sequence which’s outcome depends on
compiler settings. - bugfix omelasticsearch: invalid pointer dereference
The actual practical impact is not clear. This came up when working
on compiler warnings.
Thanks to David Lang for the patch. - bugfix omhiredis: serverport config parameter did not reliably work
depended on environment/compiler used to build - bugfix rsgtutil: -h command line option did not work
Thanks to Henri Lakk for the patch. - bugfix lexer: hex numbers were not properly represented
see: https://github.com/rsyslog/rsyslog/pull/771
Thanks to Sam Hanes for the patch. - bugfix TLS syslog: intermittent errors while sending data
Regression from commit 1394e0b. A symptom often seen was the message
“unexpected GnuTLS error -50 in nsd_gtls.c:530” - bugfix imfile: abort on startup if no slash was present in file name param
Thanks to Brian Knox for the patch. - bugfix rsgtutil: fixed abort when using short command line options
Thanks to Henri Lakk - bugfix rsgtutil: invalid computation of log record extraction file
This caused verification to fail because the hash chain was actually
incorrect. Depended on the input data set.
closes https://github.com/rsyslog/rsyslog/issues/832 - bugfix build system: KSI components could only be build if in default path
libfastjson 0.99.2 released
We have released libfastjson 0.99.2.
This is a new fork of the json-c library, which is optimized for liblognorm processing.
The changes consist of a new API and compatibility with autoconf < 2.64.
Changelog:
0.99.2 2016-03-07
– new API: json_object_get_member_count()
– make comaptible with autoconf < 2.64
Download:
http://download.rsyslog.com/libfastjson/libfastjson-0.99.2.tar.gz
sha256sum: 6ff053d455243a81014f37b4d81c746d9b8d40256a56326c3a7921c8bf458dfd
As always, feedback is appreciated.
Best regards,
Florian Riedl
libfastjson 0.99.0 released
We have released libfastjson 0.99.0.
This is a new fork of the json-c library, which is optimized for liblognorm processing.
More details are available here: rsyslog and liblognorm will switch to libfastjson as replacement for json-c
Changelog:
0.99.0 2015-12-22
– bugfix: reference counting was not thread-safe
Download:
http://download.rsyslog.com/libfastjson/libfastjson-0.99.0.tar.gz
sha256sum: 5d19c39daaedfd9b335f6222b521e7529016bc11382cccebe41a9894d4ab32fd
As always, feedback is appreciated.
Best regards,
Florian Riedl