Commit graph

431 commits

Author SHA1 Message Date
Petr Štetiar
eb216a9524 cmake: use extra compiler warnings only on gcc6+
gcc version 4.8.4 (Ubuntu 14.04) and -Wextra produces following:

 json_script.c:124:3: error: missing initializer for field 'name' of 'struct blobmsg_policy' [-Werror=missing-field-initializers]

Reported-by: Jonas Gorski <jonas.gorski@gmail.com>
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-12-01 12:26:41 +01:00
Petr Štetiar
07413cce72 tests: jshn: add more test cases
In order to cover all command line options.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-11-24 13:26:58 +01:00
Petr Štetiar
26586dae43 jshn: fix missing usage for -p and -o arguments
Add missing usage hints for -p and -o arguments.

Fixes: e16fa068a5 ("jshn: add support for namespaces")
Fixes: eb30a03048 ("libubox, jshn: add option to write output to a file")
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-11-24 13:26:58 +01:00
Petr Štetiar
8e832a771d jshn: fix off by one in jshn_parse_file
Fixes following error:

 Invalid read of size 1
   at 0x4C32D04: strlen
   by 0x5043367: json_tokener_parse_ex
   by 0x5045316: json_tokener_parse_verbose
   by 0x504537D: json_tokener_parse
   by 0x401AB1: jshn_parse (jshn.c:179)
   by 0x40190D: jshn_parse_file (jshn.c:370)
   by 0x40190D: main (jshn.c:434)
 Address 0x5848c4c is 0 bytes after a block of size 1,036 alloc'd
   at 0x4C2FB0F: malloc
   by 0x4018E2: jshn_parse_file (jshn.c:357)
   by 0x4018E2: main (jshn.c:434)

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-11-24 13:26:58 +01:00
Petr Štetiar
cb698e3540 jshn: jshn_parse: fix leaks of memory pointed to by 'obj'
Fixes following leaks of memory:

 352 (72 direct, 280 indirect) bytes in 1 blocks are definitely lost in loss record 3 of 3
   at 0x4C31B25: calloc
   by 0x5042E1F: json_object_new_array
   by 0x5044B02: json_tokener_parse_ex
   by 0x5045316: json_tokener_parse_verbose
   by 0x504537D: json_tokener_parse
   by 0x401AA9: jshn_parse (jshn.c:179)
   by 0x401977: main (jshn.c:378)

 752 (72 direct, 680 indirect) bytes in 1 blocks are definitely lost in loss record 6 of 6
   at 0x4C31B25: calloc
   by 0x50424CF: json_object_new_object
   by 0x5044B38: json_tokener_parse_ex
   by 0x5045316: json_tokener_parse_verbose
   by 0x504537D: json_tokener_parse
   by 0x401AA9: jshn_parse (jshn.c:179)
   by 0x401977: main (jshn.c:380)

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-11-24 13:26:58 +01:00
Petr Štetiar
c42f11cc7c jshn: main: fix leak of memory pointed to by 'vars'
Fixes following leak of memory:

 6,016 bytes in 1 blocks are possibly lost in loss record 1 of 1
    at 0x4C31B25: calloc
    by 0x1098F8: main (jshn.c:353)

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-11-24 13:26:58 +01:00
Petr Štetiar
93848ec96d jshn: refactor main into smaller pieces
Turn longer switch cases into separate functions in order to make it
easier to follow. Don't return from the cases as it makes future
cleaning up harder.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-11-24 13:26:58 +01:00
Petr Štetiar
9b6ede0e53 avl: guard against theoretical null pointer dereference
clang-10 analyzer reports following:

 avl.c:671:25: warning: Access to field 'parent' results in a dereference of a null pointer (loaded from field 'right')
     node->right->parent = parent;
           ~~~~~         ^

Which seems to be impossible to trigger via exported AVL public API, but
it could be probably trigerred by fiddling with the AVL tree node struct
members manually as they are exposed.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-11-24 13:26:58 +01:00
Petr Štetiar
c008294a83 blobmsg_json: fix possible uninitialized struct member
clang-10 analyzer reports following:

 blobmsg_json.c:285:2: warning: The expression is an uninitialized value. The computed value will also be garbage
         s->indent_level++;
         ^~~~~~~~~~~~~~~~~

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-11-24 13:26:58 +01:00
Petr Štetiar
0003ea9c45 base64: fix possible null pointer dereference
clang-10 analyzer reports following:

 base64.c:325:20: warning: Array access (from variable 'target') results in a null pointer dereference
                 target[tarindex] = 0;
                 ~~~~~~           ^

and prepared test case confirms it:

 Invalid write of size 1
    at 0x4E4463F: b64_decode (base64.c:325)
    by 0x40088C: test_invalid_inputs (tests/test-base64.c:26)
    by 0x40088C: main (tests/test-base64.c:32)
  Address 0x1 is not stack'd, malloc'd or (recently) free'd

 Process terminating with default action of signal 11 (SIGSEGV)
  Access not within mapped region at address 0x1
    at 0x4E4463F: b64_decode (base64.c:325)
    by 0x40088C: test_invalid_inputs (tests/test-base64.c:26)
    by 0x40088C: main (tests/test-base64.c:32)

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-11-24 13:26:58 +01:00
Petr Štetiar
8baeeea1f5 add assert.h component
In order to allow seamless assert() usage in release builds without the
need for fiddling with CMake C flags as CMake adds -DNDEBUG switch in
release builds which disable assert().

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-11-24 13:26:58 +01:00
Petr Štetiar
b0a5cd8a28 add cram based unit tests
For improved QA etc. For the start with initial test cases for avl,
base64, jshn and list components. Moved runqueue and blobmsg from
examples to tests.  Converted just a few first test cases from
json-script example into the new cram based unit test, more to come.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-11-24 13:26:58 +01:00
Petr Štetiar
1fefb7c4d7 add initial GitLab CI support
Uses currently proof-of-concept openwrt-ci[1] in order to:

 * improve the quality of the codebase in various areas
 * decrease code review time and help merging contributions faster
 * get automagic feedback loop on various platforms and tools
   - out of tree build with OpenWrt SDK on following targets:
     * ath79-generic
     * imx6-generic
     * malta-be
     * mvebu-cortexa53
   - out of tree native build on x86/64 with GCC (versions 7, 8, 9) and Clang 10
   - out of tree native x86/64 static code analysis with cppcheck and
     scan-build from Clang 10

1. https://gitlab.com/ynezz/openwrt-ci/

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-11-20 14:34:01 +01:00
Petr Štetiar
c955464d7a enable extra compiler checks
Let's enforce additional automatic checks enforced by the compiler in
order to catch possible errors during compilation.

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-11-20 14:34:01 +01:00
Petr Štetiar
6228df9de9 iron out all extra compiler warnings
gcc-9 on x86/64 has reported following issues:

 base64.c:173:17: error: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Werror=sign-compare]
 base64.c:230:18: error: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Werror=sign-compare]
 base64.c:238:18: error: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Werror=sign-compare]
 base64.c:242:22: error: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Werror=sign-compare]
 base64.c:252:18: error: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Werror=sign-compare]
 base64.c:256:22: error: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Werror=sign-compare]
 base64.c:266:18: error: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Werror=sign-compare]
 base64.c:315:27: error: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Werror=sign-compare]
 base64.c:329:15: error: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Werror=sign-compare]
 blob.c:207:11: error: comparison of integer expressions of different signedness: ‘unsigned int’ and ‘int’ [-Werror=sign-compare]
 blob.c:210:11: error: comparison of integer expressions of different signedness: ‘unsigned int’ and ‘int’ [-Werror=sign-compare]
 blob.c:243:31: error: comparison of integer expressions of different signedness: ‘int’ and ‘unsigned int’ [-Werror=sign-compare]
 blob.c:246:31: error: comparison of integer expressions of different signedness: ‘int’ and ‘unsigned int’ [-Werror=sign-compare]
 blob.h:245:37: error: comparison of integer expressions of different signedness: ‘unsigned int’ and ‘int’ [-Werror=sign-compare]
 blob.h:253:37: error: comparison of integer expressions of different signedness: ‘unsigned int’ and ‘int’ [-Werror=sign-compare]
 blobmsg.h:269:37: error: comparison of integer expressions of different signedness: ‘unsigned int’ and ‘int’ [-Werror=sign-compare]
 blobmsg_json.c:155:10: error: comparison of integer expressions of different signedness: ‘int’ and ‘size_t’ {aka ‘long unsigned int’} [-Werror=sign-compare]
 examples/../blob.h:245:37: error: comparison of integer expressions of different signedness: ‘unsigned int’ and ‘int’ [-Werror=sign-compare]
 examples/../blobmsg.h:269:37: error: comparison of integer expressions of different signedness: ‘unsigned int’ and ‘int’ [-Werror=sign-compare]
 json_script.c:590:7: error: this statement may fall through [-Werror=implicit-fallthrough=]

Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-11-20 14:34:01 +01:00
Yousong Zhou
301303911d vlist: add more macros for loop iteration
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2019-10-29 13:07:38 +00:00
Roman Yeryomin
eb30a03048 libubox, jshn: add option to write output to a file
This would allow board_config_flush to run one command instead
of two and would be faster and safer than redirecting output
and moving a file between filesystems.

Originally discussed here:
http://lists.openwrt.org/pipermail/openwrt-devel/2017-December/010127.html

Signed-off-by: Roman Yeryomin <roman@advem.lv>
2019-10-21 14:21:41 +02:00
Hauke Mehrtens
ecf56174da ustream: Add format string checks to ustream_(v)printf()
This tells the compiler that these functions are takeing a format
string, the compiler will now do additional checks and is able to emitt
a compile warning in case the format string is not valid.

Signed-off-by: Hauke Mehrtens <hauke@hauke-m.de>
2019-06-16 16:30:01 +02:00
Kristupas Savickas
9dd2dcff70 libubox: add format string checking to ulog()
This offers an increased level of security, as the arguments will be
checked for validity against the format string at compile time. The
format attribute is supported by both GCC and Clang, so there shouldn't
be any portability issues.

Signed-off-by: Kristupas Savickas <savickas.kristupas@gmail.com>
2019-06-16 16:29:47 +02:00
Yousong Zhou
eeef7b50a0 blobmsg_json: blobmsg_format_string: do not escape '/'
Resolves FS#2147

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2019-02-27 02:50:02 +00:00
John Crispin
c83a84afbe fix segfault when passed blobmsg attr is NULL
Signed-off-by: John Crispin <john@phrozen.org>
2018-07-25 10:30:05 +02:00
Felix Fietkau
3c1b33b7d5 utils: add const_* byteswapping functions
Some gcc versions have issues with the __builtin_choose_expr construct
which attempts to automatically use the const versions of those
functions.

Make it possible to explicitly use const_* versions to avoid that.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2018-06-07 15:18:52 +02:00
Felix Fietkau
6eff829d78 utils: fix build error with g++
g++ does not support __builtin_choose_expr, so we can't support
byte swapping as constant expression there.

Reported-by: Cleynhens Stijn <Stijn.Cleynhens@technicolor.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2018-04-12 11:09:52 +02:00
Felix Fietkau
ace64897d4 switch from typeof to the more portable __typeof__
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2018-04-07 15:21:33 +02:00
Felix Fietkau
92009b7f98 utils: ensure that byte-order conversion functions evaluate the argument only once
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2018-04-07 10:43:49 +02:00
Felix Fietkau
42a8ecd858 jshn: fix format string for int64 type
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2018-04-07 10:36:38 +02:00
Felix Fietkau
eebe3fcd2a utils: use constant byte-order conversion
Simplifies portability and ensures that cpu_to_* can be used in const
declarations. If the architecture has special instructions, the compiler
should be able to detect the pattern and use them.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2018-03-21 17:55:57 +01:00
Rosen Penev
3aad2948eb libubox: Plug a small memory leak.
va_end was not called if calloc fails.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2018-02-11 16:39:35 +01:00
Hans Dedecker
bb0c830b2a sh/jshn.sh: add json_for_each_item()
Function usefull to iterate through the different elements of an
array or object; the provided callback function is called for each
element which is passed the value, key and user provided arguments.
For field types different from array or object the callback is called
with the retrieved value.

Signed-off-by: Hans Dedecker <dedeckeh@gmail.com>
Acked-by: Jo-Philipp Wich <jo@mein.io>
2018-02-08 12:15:20 +01:00
Christian Beier
9c4aeda962 jshn: add functionality to read big JSON
The existing read functionality feeds the complete JSON to jshn as a
cmdline argument, leading to `-ash: jshn: Argument list too long`
errors for JSONs bigger than ca. 100KB.

This commit adds the ability to read the JSON directly from a file if
wanted, removing this shell-imposed size limit.

Tested on x86-64 and ar71xx. An mmap()-based solution was also evaluated,
but found to make no performance difference on either platform.

Signed-off-by: Christian Beier <dontmind@freeshell.org>
2018-01-22 09:21:03 +01:00
Jo-Philipp Wich
1c08e80313 jshn: properly support JSON "null" type
Instead of abort parsing, properly deal with "null" values by implementing
support for reading and formatting such values.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-01-07 15:52:24 +01:00
Christian Beier
729f47fd52 jshn: read and write 64-bit integers
This allows for reading in and writing out bigger JSON Numbers.

Following test script (that fails to print correct values _without_ this
commit) verifies the functionality (tested on x86-64 as well as on ar71xx):

---snip---
# assumes you built jshn and sourced jshn.sh
echo testing reading-in JSON
SHELL_BIGNUM=12147483647
json_init
json_load "{ \"bignum\": $SHELL_BIGNUM }"
json_get_var BIGNUM bignum
echo jshn bignum: $BIGNUM
echo shll bignum: $SHELL_BIGNUM
echo testing writing-out JSON
json_init
json_add_int bigint $SHELL_BIGNUM
json_dump
--snap---

Signed-off-by: Christian Beier <dontmind@freeshell.org>
2017-11-06 10:05:42 +01:00
Stijn Tintel
632688e8d6 utils: nuke bitfield functions and macros
The bitfield functions and macros were committed without explaining
their purpose in the commit message.

As they are only used in uci, and conflict with similar functions added
in hostapd, breaking our hostapd ubus patch, nuke them from libubox and
add them in uci instead.

If we need them anywhere else in the future we can add it to libubox
again, but preferably prefixed with ubox_.

Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
2017-09-29 15:34:36 +03:00
Petar Paradzik
f714be125c uloop: make SIGCHLD signal handling optional
Some programs want to manage their own child life cycle without using
SIGCHLD signal handler. In these cases, uloop is reaping children for
them because they don't have SIGCHLD handler set. This patch makes it
possible to disable reaping children through 'uloop_handle_sigchld'
variable.

Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>
2017-09-29 13:37:17 +02:00
Michal Sojka
7a1057604e uloop: Fix race condition in SIGCHLD handling
When uloop_process_add() is called outside of uloop_run(), i.e. not
from a callback (which is the case of at least utrace and ujail),
child events can be missed. The reason is that when SIGCHILD handler
is installed in uloop_run(), after the uloop_process_add() is called,
then an initial signal could be missed.

Commit 4e3a47a ("uloop: use a waker for notifying sigchld and loop
cancel events", 2016-06-09) solved a similar problem and introduced
uloop_init() but forgot to move a call to uloop_setup_signals() there.
This is what this commit does.

Now, uloop_process_add() can be called any time after uloop_init()
without missing any event.

Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
Acked-by: Yousong Zhou <yszhou4tech@gmail.com>
2017-09-15 15:39:42 +08:00
Felix Fietkau
fd57eea9f3 uloop: allow passing 0 as timeout to uloop_run
Useful for processing only immediate timers and fds

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-06-17 11:47:49 +02:00
Felix Fietkau
4bc3decf87 uloop: fix a regression in timeout handling
Variable confusion was breaking timers

Fixes: 368fd26458 ("uloop: allow specifying a timeout for uloop_run()")
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-06-17 11:40:11 +02:00
Felix Fietkau
6a7fb7d8df runqueue: fix use-after-free bug
Calling t->complete in runqueue_task_complete can free the memory
associated with t. Change the runqueue_start_next accordingly.

Fixes https://github.com/openwrt/openwrt/issues/493

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-06-14 12:08:53 +02:00
Felix Fietkau
368fd26458 uloop: allow specifying a timeout for uloop_run()
This can be useful for cleanup with pending timers, or for hooking into
existing code that does not use uloop

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-06-01 11:24:44 +02:00
Denis Osvald
fa9937cc4f json_script: enable custom expr handler callback
This wires in custom expression handler functionality, which was present
in json script since the original version, but never used.

Signed-off-by: Denis Osvald <denis.osvald@sartura.hr>
Signed-off-by: Felix Fietkau <nbd@nbd.name> [error handling fix]
2017-06-01 11:23:16 +02:00
Yousong Zhou
7237302147 md5: add "const" qualifier to the "file" argument
This is intended to fix the following compiler warning in opkg-lede:

    /home/yousong/git-repo/lede-project/opkg-lede/libopkg/file_util.c: In function ‘file_md5sum_alloc’:
    /home/yousong/git-repo/lede-project/opkg-lede/libopkg/file_util.c:144:2: warning: passing argument 1 of ‘md5sum’ discards ‘const’ qualifier from pointer target type [enabled by default]
    In file included from /home/yousong/git-repo/lede-project/opkg-lede/libopkg/file_util.c:28:0:
    /home/yousong/.usr/include/libubox/md5.h:56:5: note: expected ‘char *’ but argument is of type ‘const char *’

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2017-03-20 12:44:46 +01:00
Ted Hess
96305a3caf libubox: Change calloc_a() to return size_t aligned pointers
Un-aligned pointers were causing seg faults on some targets

Signed-off-by: Ted Hess <thess@kitschensync.net>
2017-02-24 15:00:15 -05:00
Felix Fietkau
de3f14b643 uloop: add uloop_cancelling function
Returns true if uloop_run is still running and uloop_cancelled is set

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-03 16:52:21 +01:00
Felix Fietkau
3b6181b63d utils: fix build on Mac OS X 10.12
clock_gettime and CLOCK_MONOTONIC are supported now

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-20 11:29:53 +01:00
André Gaul
7f671b1e68 blobmsg: add support for double
This adds support for double floating point type to make it more JSON
compatible. For type checking it also adds a stub BLOB_ATTR_DOUBLE type.
If necessary, the accessor functions for blob can be added later

Signed-off-by: André Gaul <andre@gaul.io>
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-04 21:36:31 +01:00
Felix Fietkau
0fe13749d0 utils: add helper functions useful for allocating a ring buffer
This creates a mapping with twice the size of the allocated memory. The
second half of that mapping points at the same memory as the first half.

This is useful for ring buffers, because any read starting in the first
half can overflow into the second half as long as the read size is
smaller than the size of the memory area.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-28 22:44:30 +01:00
Rosen Penev
8fc1c3053e libubox: replace strtok with _r version.
_r is re-entrant. Also happens to silence a cppcheck warning.

Signed-off by: Rosen Penev <rosenp@gmail.com>
2016-12-24 11:03:29 +01:00
Florian Eckert
4a9f74f7ae libubox: allow reading out the pid of uloop process in lua
Add Lua method to get the forked pid of a uloop process

Signed-off-by: Florian Eckert <Eckert.Florian@googlemail.com>
2016-12-13 17:13:20 +01:00
Felix Fietkau
372e1e65ef uloop: remove useless epoll data assignment
ev.data is a union, so setting ev.data.fd is lost after setting
ev.data.ptr

Reported-by: Song Yaofei <songyaofei@joyware.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-12 12:24:14 +01:00
Stijn Cleynhens
f9db1cb918 libubox: allow reading out the remaining time of a uloop timer in Lua
Add Lua method to the uloop wrapper to allow reading out the remaining time of a uloop timer

Signed-off-by: Stijn Cleynhens <stijncleynhens@gmail.com>
2016-12-05 18:11:50 +01:00