Commit graph

455 commits

Author SHA1 Message Date
Felix Fietkau
eebe3fcd2a utils: use constant byte-order conversion
Simplifies portability and ensures that cpu_to_* can be used in const
declarations. If the architecture has special instructions, the compiler
should be able to detect the pattern and use them.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2018-03-21 17:55:57 +01:00
Rosen Penev
3aad2948eb libubox: Plug a small memory leak.
va_end was not called if calloc fails.

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2018-02-11 16:39:35 +01:00
Hans Dedecker
bb0c830b2a sh/jshn.sh: add json_for_each_item()
Function usefull to iterate through the different elements of an
array or object; the provided callback function is called for each
element which is passed the value, key and user provided arguments.
For field types different from array or object the callback is called
with the retrieved value.

Signed-off-by: Hans Dedecker <dedeckeh@gmail.com>
Acked-by: Jo-Philipp Wich <jo@mein.io>
2018-02-08 12:15:20 +01:00
Christian Beier
9c4aeda962 jshn: add functionality to read big JSON
The existing read functionality feeds the complete JSON to jshn as a
cmdline argument, leading to `-ash: jshn: Argument list too long`
errors for JSONs bigger than ca. 100KB.

This commit adds the ability to read the JSON directly from a file if
wanted, removing this shell-imposed size limit.

Tested on x86-64 and ar71xx. An mmap()-based solution was also evaluated,
but found to make no performance difference on either platform.

Signed-off-by: Christian Beier <dontmind@freeshell.org>
2018-01-22 09:21:03 +01:00
Jo-Philipp Wich
1c08e80313 jshn: properly support JSON "null" type
Instead of abort parsing, properly deal with "null" values by implementing
support for reading and formatting such values.

Signed-off-by: Jo-Philipp Wich <jo@mein.io>
2018-01-07 15:52:24 +01:00
Christian Beier
729f47fd52 jshn: read and write 64-bit integers
This allows for reading in and writing out bigger JSON Numbers.

Following test script (that fails to print correct values _without_ this
commit) verifies the functionality (tested on x86-64 as well as on ar71xx):

---snip---
# assumes you built jshn and sourced jshn.sh
echo testing reading-in JSON
SHELL_BIGNUM=12147483647
json_init
json_load "{ \"bignum\": $SHELL_BIGNUM }"
json_get_var BIGNUM bignum
echo jshn bignum: $BIGNUM
echo shll bignum: $SHELL_BIGNUM
echo testing writing-out JSON
json_init
json_add_int bigint $SHELL_BIGNUM
json_dump
--snap---

Signed-off-by: Christian Beier <dontmind@freeshell.org>
2017-11-06 10:05:42 +01:00
Stijn Tintel
632688e8d6 utils: nuke bitfield functions and macros
The bitfield functions and macros were committed without explaining
their purpose in the commit message.

As they are only used in uci, and conflict with similar functions added
in hostapd, breaking our hostapd ubus patch, nuke them from libubox and
add them in uci instead.

If we need them anywhere else in the future we can add it to libubox
again, but preferably prefixed with ubox_.

Signed-off-by: Stijn Tintel <stijn@linux-ipv6.be>
2017-09-29 15:34:36 +03:00
Petar Paradzik
f714be125c uloop: make SIGCHLD signal handling optional
Some programs want to manage their own child life cycle without using
SIGCHLD signal handler. In these cases, uloop is reaping children for
them because they don't have SIGCHLD handler set. This patch makes it
possible to disable reaping children through 'uloop_handle_sigchld'
variable.

Signed-off-by: Petar Paradzik <petar.paradzik@sartura.hr>
2017-09-29 13:37:17 +02:00
Michal Sojka
7a1057604e uloop: Fix race condition in SIGCHLD handling
When uloop_process_add() is called outside of uloop_run(), i.e. not
from a callback (which is the case of at least utrace and ujail),
child events can be missed. The reason is that when SIGCHILD handler
is installed in uloop_run(), after the uloop_process_add() is called,
then an initial signal could be missed.

Commit 4e3a47a ("uloop: use a waker for notifying sigchld and loop
cancel events", 2016-06-09) solved a similar problem and introduced
uloop_init() but forgot to move a call to uloop_setup_signals() there.
This is what this commit does.

Now, uloop_process_add() can be called any time after uloop_init()
without missing any event.

Signed-off-by: Michal Sojka <sojkam1@fel.cvut.cz>
Acked-by: Yousong Zhou <yszhou4tech@gmail.com>
2017-09-15 15:39:42 +08:00
Felix Fietkau
fd57eea9f3 uloop: allow passing 0 as timeout to uloop_run
Useful for processing only immediate timers and fds

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-06-17 11:47:49 +02:00
Felix Fietkau
4bc3decf87 uloop: fix a regression in timeout handling
Variable confusion was breaking timers

Fixes: 368fd26458 ("uloop: allow specifying a timeout for uloop_run()")
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-06-17 11:40:11 +02:00
Felix Fietkau
6a7fb7d8df runqueue: fix use-after-free bug
Calling t->complete in runqueue_task_complete can free the memory
associated with t. Change the runqueue_start_next accordingly.

Fixes https://github.com/openwrt/openwrt/issues/493

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-06-14 12:08:53 +02:00
Felix Fietkau
368fd26458 uloop: allow specifying a timeout for uloop_run()
This can be useful for cleanup with pending timers, or for hooking into
existing code that does not use uloop

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-06-01 11:24:44 +02:00
Denis Osvald
fa9937cc4f json_script: enable custom expr handler callback
This wires in custom expression handler functionality, which was present
in json script since the original version, but never used.

Signed-off-by: Denis Osvald <denis.osvald@sartura.hr>
Signed-off-by: Felix Fietkau <nbd@nbd.name> [error handling fix]
2017-06-01 11:23:16 +02:00
Yousong Zhou
7237302147 md5: add "const" qualifier to the "file" argument
This is intended to fix the following compiler warning in opkg-lede:

    /home/yousong/git-repo/lede-project/opkg-lede/libopkg/file_util.c: In function ‘file_md5sum_alloc’:
    /home/yousong/git-repo/lede-project/opkg-lede/libopkg/file_util.c:144:2: warning: passing argument 1 of ‘md5sum’ discards ‘const’ qualifier from pointer target type [enabled by default]
    In file included from /home/yousong/git-repo/lede-project/opkg-lede/libopkg/file_util.c:28:0:
    /home/yousong/.usr/include/libubox/md5.h:56:5: note: expected ‘char *’ but argument is of type ‘const char *’

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2017-03-20 12:44:46 +01:00
Ted Hess
96305a3caf libubox: Change calloc_a() to return size_t aligned pointers
Un-aligned pointers were causing seg faults on some targets

Signed-off-by: Ted Hess <thess@kitschensync.net>
2017-02-24 15:00:15 -05:00
Felix Fietkau
de3f14b643 uloop: add uloop_cancelling function
Returns true if uloop_run is still running and uloop_cancelled is set

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-02-03 16:52:21 +01:00
Felix Fietkau
3b6181b63d utils: fix build on Mac OS X 10.12
clock_gettime and CLOCK_MONOTONIC are supported now

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-20 11:29:53 +01:00
André Gaul
7f671b1e68 blobmsg: add support for double
This adds support for double floating point type to make it more JSON
compatible. For type checking it also adds a stub BLOB_ATTR_DOUBLE type.
If necessary, the accessor functions for blob can be added later

Signed-off-by: André Gaul <andre@gaul.io>
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2017-01-04 21:36:31 +01:00
Felix Fietkau
0fe13749d0 utils: add helper functions useful for allocating a ring buffer
This creates a mapping with twice the size of the allocated memory. The
second half of that mapping points at the same memory as the first half.

This is useful for ring buffers, because any read starting in the first
half can overflow into the second half as long as the read size is
smaller than the size of the memory area.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-28 22:44:30 +01:00
Rosen Penev
8fc1c3053e libubox: replace strtok with _r version.
_r is re-entrant. Also happens to silence a cppcheck warning.

Signed-off by: Rosen Penev <rosenp@gmail.com>
2016-12-24 11:03:29 +01:00
Florian Eckert
4a9f74f7ae libubox: allow reading out the pid of uloop process in lua
Add Lua method to get the forked pid of a uloop process

Signed-off-by: Florian Eckert <Eckert.Florian@googlemail.com>
2016-12-13 17:13:20 +01:00
Felix Fietkau
372e1e65ef uloop: remove useless epoll data assignment
ev.data is a union, so setting ev.data.fd is lost after setting
ev.data.ptr

Reported-by: Song Yaofei <songyaofei@joyware.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-12-12 12:24:14 +01:00
Stijn Cleynhens
f9db1cb918 libubox: allow reading out the remaining time of a uloop timer in Lua
Add Lua method to the uloop wrapper to allow reading out the remaining time of a uloop timer

Signed-off-by: Stijn Cleynhens <stijncleynhens@gmail.com>
2016-12-05 18:11:50 +01:00
Felix Fietkau
77a629375d blob/blobmsg: add explicit typecasts for attribute iterators
Fixes C++ compatibility.

Reported in https://forum.lede-project.org/t/blobmsg-for-each-attr-from-c/389

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-11-29 14:02:57 +01:00
Felix Fietkau
d0312bd9b1 kvlist: add static initializer macros
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-10-24 12:34:09 +02:00
Felix Fietkau
290c64ef5b libubox: add static initializer macro for runqueues
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-29 12:01:37 +02:00
Felix Fietkau
72613ca2c8 avl: add blob comparator function
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-07-29 10:58:55 +02:00
Matthias Schiffer
d3fa561e5a blobmsg_json: add new functions blobmsg_format_json_value*
The current blobmsg_format_json* functions will return invalid JSON when
the "list" argument is given as false (blobmsg_format_element() will
output the name of the blob_attr as if the value is printed as part of a
JSON object).

To avoid breaking software relying on this behaviour, introduce new
functions which will never print the blob_attr name and thus always
produce valid JSON.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
Acked-by: Jo-Philipp Wich <jo@mein.io>
Signed-off-by: Felix Fietkau <nbd@nbd.name> [cosmetic style fix]
2016-07-02 10:10:08 +02:00
Eyal Birger
d197c8ffa3 uloop: handle waker pipe write() return value
Recent glibc warns if result of read() or write() is unused.

Added a retry in case of EINTR, all other faults are silently discarded.

Signed-off-by: Eyal Birger <eyal.birger@gmail.com>

-----
- I was not able to reproduce the EINTR case, but it seems to be the right
  thing to do
- Retrying on EAGAIN in this case would be weird as there is no one to read
  from the other end of the pipe. We could call waker_consume() directly but
  since the size of the message is just one byte, I think this would be dead
  code
2016-06-26 13:05:54 +02:00
Matthias Schiffer
1ad3d93eb8 loop: make uloop_run() return the cancelling signal
When a process quits in response to a signal it handles, it should to so
be re-sending the signal to itself. This especially important for SIGINT,
as is explained in [1].

uloop currently hides the reason for quitting uloop_run(). Fix this by
returning the signal that caused the loop to quit (or 0 when uloop_end()
was used), so a program using loop an comply with [1].

[1] https://www.cons.org/cracauer/sigint.html

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2016-06-26 12:55:31 +02:00
Matthias Schiffer
1f019ceea1 Fix various memory management issues
Consistently handle allocation failures. Some functions are changed to
return bool or int instead of void to allow returning an error.

Also fix a buffer size miscalculation in lua/uloop and use _exit() instead
of exit() on errors after forking.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2016-06-26 12:53:51 +02:00
Felix Fietkau
c2f2c47f3e uloop: add missing waker_pipe initialization
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-06-15 15:14:51 +02:00
Yousong Zhou
4e3a47a4cb uloop: use a waker for notifying sigchld and loop cancel events
Fix a race condition when do_sigchld, uloop_cancelled were set just
before epoll_wait(timeout=-1), resulting the loop stuck in the syscall
without noticing the events just happened

Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-06-15 11:54:37 +02:00
Felix Fietkau
1257a38a6e uloop: revert signalfd support for now
It hasn't fixed the reported race condition and it introduced some new
issues.

Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-05-19 10:59:48 +02:00
Felix Fietkau
93be9309b8 uloop: add back support for overriding signal handlers when signalfd is in use
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-05-17 13:59:05 +02:00
Felix Fietkau
004be15be4 uloop: fix signal unblocking
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-05-17 13:35:00 +02:00
Felix Fietkau
b06cd8c58e uloop: retry waitpid on signal interrupt
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-05-17 13:27:44 +02:00
Felix Fietkau
6a75b3b643 uloop: try to use signalfd for signal handling if available
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-05-17 13:23:38 +02:00
Felix Fietkau
8ae74b4378 uloop: move epoll code into a separate file
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-05-17 13:23:38 +02:00
Felix Fietkau
4b0aa9ccc2 uloop: move kqueue code into a separate file
Signed-off-by: Felix Fietkau <nbd@nbd.name>
2016-05-17 13:23:38 +02:00
Matthias Schiffer
155bf39896 blobmsg_json: simplify add_separator and fix thread-safety
The hard-coded length limits are replaced with strlen to make the code more
robust.

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2016-04-06 17:43:47 +02:00
Felix Fietkau
dfe446e2a9 jshn: use an avl tree for env variables to speed up processing of bigger json files
Signed-off-by: Felix Fietkau <nbd@openwrt.org>
2016-03-05 18:51:31 +01:00
Yousong Zhou
5326ce1046 examples: add shunit2 tests for json_script
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2016-02-20 13:36:08 +01:00
Yousong Zhou
02db6d71b6 json_script: add "isdir" support
Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
2016-02-20 13:36:08 +01:00
Jo-Philipp Wich
f43da92dd7 blobmsg_json: support json_type_null in blobmsg_add_json_element()
We already serialize BLOBMSG_TYPE_UNSPEC as "null" so represent JSON null
values as blob attribute type unspec with payload size zero.

Signed-off-by: Jo-Philipp Wich <jow@openwrt.org>
2016-02-09 23:30:32 +01:00
Felix Fietkau
0c2fbbca80 usock: implement usock_inet_timeout() with RFC6555 support
Signed-off-by: Felix Fietkau <nbd@openwrt.org>
2016-01-28 12:07:25 +01:00
Felix Fietkau
bc7e277276 ustream-fd: handle ENOTCONN for read/write on not-yet-connected sockets
Signed-off-by: Felix Fietkau <nbd@openwrt.org>
2016-01-15 21:35:48 +01:00
Felix Fietkau
663d7d4b88 lua: add gc/delete support for processes
Signed-off-by: Felix Fietkau <nbd@openwrt.org>
2016-01-15 21:26:43 +01:00
Felix Fietkau
a8d12b22e0 lua: create a common function for userdata with gc
Signed-off-by: Felix Fietkau <nbd@openwrt.org>
2016-01-15 21:19:47 +01:00