When started with a verbose level of 3, asterisk can emit over 1500
verbose message that serve no real purpose other than to fill up
logs. When asterisk shuts down, it emits another 1100 that are of
even less use. Since the testsuite runs asterisk with a verbose
level of 3, and asterisk starts and stops for every one of the 700+
tests, the number of log messages is staggering. Besides taking up
resources, it also makes it hard to debug failing tests.
This commit changes the log level for those verbose messages to 5
instead of 3 which reduces the number of log messages to only a
handful. Of course, NOTICE, WARNING and ERROR message are
unaffected.
There's also one other minor change...
ast_context_remove_extension_callerid2() logs a DEBUG message
instead of an ERROR if the extension you're deleting doesn't exist.
The pjsip_config_wizard calls that function to clean up the config
and has been triggering that annoying error message for years.
Resolves: #582
A sporadic test failure was happening when executing the AEAP
Websocket transport tests. It was originally thought this was
due to things not getting cleaned up fast enough, but upon further
investigation I determined the underlying cause was poll()
getting interrupted and this not being handled in all places.
This change adds EINTR and EAGAIN handling to the Websocket
client connect code as well as the AEAP Websocket transport code.
If either occur then the code will just go back to waiting
for data.
The originally disabled failure test case has also been
re-enabled.
ASTERISK-30099
Change-Id: I1711a331ecf5d35cd542911dc6aaa9acf1e172ad
Previously there was no way to specify a connection timeout when
attempting to connect a websocket client to a server. This patch
makes it possible to now do such.
Change-Id: I5812f6f28d3d13adbc246517f87af177fa20ee9d
Some modules have a different support level documented in their
MODULEINFO XML and Asterisk module definition. This change
brings the two in sync for the modules which were not matching.
ASTERISK-29336
Change-Id: If2f819103d4a271e2e0624ef4db365e897fa3d35
When dealing with a lot of video streams on WebRTC
the resulting SDPs can grow to be quite large. This
effectively doubles the maximum size to allow more
streams to exist.
The res_http_websocket module has also been changed
to use a buffer on the session for reading in packets
to ensure that the stack space usage is not excessive.
Change-Id: I31d4351d70c8e2c11564807a7528b984f3fbdd01
We read beyond the end of the buffer when copying the string out of the
buffer when we used ast_copy_string() because the original string was
not null terminated. Instead switch to ast_strndup() which does not
exhibit the same behavior.
ASTERISK-28975 #close
Change-Id: Ib4a75cffeb1eb8cf01136ef30306bd623e531a2a
Do not return error if the client received ping frame
while looking for a string and just wait for another frame.
ASTERISK-28958 #close
Change-Id: I4d06b4827bd71e56cbaafc011ffdcef9f0332922
When TLS is in use, checking the readiness of the underlying FD is insufficient
for determining if there is data available to be read. So before polling the
FD, check if there is any buffered data in the TLS layer and use that first.
ASTERISK-28562 #close
Reported by: Robert Sutton
Change-Id: I95fcb3e2004700d5cf8e5ee04943f0115b15e10d
Control frames (PING / PONG / CLOSE) can be received in the middle of a
fragmented message. In order to ensure they do not interfere with the
reassembly buffer, we exit early and do not return the payload to the
caller.
ASTERISK-28257 #close
Change-Id: Ia5367144fe08ac6141bba3309517a48ec7f013bc
This ensures that Asterisk responds properly to frames received from a
client with opcode 8 (CLOSE) by echoing back the status code in its own
CLOSE frame.
Handling of the CLOSE opcode is moved up with the rest of the opcodes so
that unmasking gets applied. The payload is no longer returned to the
caller, but neither ARI nor the chan_sip nor pjsip made use of the
payload, which is a good thing since it was masked.
ASTERISK-28231 #close
Change-Id: Icb1b60205fc77ee970ddc91d1f545671781344cf
Replace usage of ao2_container_alloc with ao2_container_alloc_hash or
ao2_container_alloc_list. Remove ao2_container_alloc macro.
Change-Id: I0907d78bc66efc775672df37c8faad00f2f6c088
The HTTP request processing in res_http_websocket allocates additional
space on the stack for various headers received during an Upgrade request.
An attacker could send a specially crafted request that causes this code
to overflow the stack, resulting in a crash.
* No longer allocate memory from the stack in a loop to parse the header
values. NOTE: There is a slight API change when using the passed in
strings as is. We now require the passed in strings to no longer have
leading or trailing whitespace. This isn't a problem as the only callers
have already done this before passing the strings to the affected
function.
ASTERISK-28013 #close
Change-Id: Ia564825a8a95e085fd17e658cb777fe1afa8091a
* acl (named_acl.c)
* cdr
* cel
* ccss
* dnsmgr
* dsp
* enum
* extconfig (config.c)
* features
* http
* indications
* logger
* manager
* plc
* sounds
* udptl
These modules are now loaded at appropriate time by the module loader.
Unlike loadable modules these use AST_MODULE_LOAD_FAILURE on error so
the module loader will abort startup on failure of these modules.
Some of these modules are still initialized or shutdown from outside the
module loader. logger.c is initialized very early and shutdown very
late, manager.c is initialized by the module loader but is shutdown by
the Asterisk core (too much uses it without holding references).
Change-Id: I371a9a45064f20026c492623ea8062d02a1ab97f
In ast_websocket_read() we were not adequately checking that the
payload_len was non-zero before passing it to ws_safe_read(). Calling
ws_safe_read with a len argument of 0 will result in a busy loop until
the underlying socket is closed.
ASTERISK-27658 #close
Change-Id: I9d59f83bc563f711df1a6197c57de473f6b0663a
This removes references that are no longer needed due to automatic
references created by module dependencies.
In addition this removes most calls to ast_module_check as they were
checking modules which are listed as dependencies.
Change-Id: I332a6e8383d4c72c8e89d988a184ab8320c4872e
There are many places in the code base where we ignore the return value
of fcntl() when getting/setting file descriptior flags. This patch
introduces a convenience function that allows setting or clearing file
descriptor flags and will also log an error on failure for later
analysis.
Change-Id: I8b81901e1b1bd537ca632567cdb408931c6eded7
Previously for PJSIP the local address of WebSocket connections
was set to the remote address. For logging purposes this is
not particularly useful.
The WebSocket API has been extended to allow the local
address to be queried and this is used in PJSIP to set the
local address to the correct value.
The PJSIP HEP support has also been tweaked so that reliable
transports always use the local address on the transport
and do not try to (wrongly) guess. As they are connection
based it is impossible for the source to be anything else.
ASTERISK-26758
ASTERISK-27363
Change-Id: Icd305fd038ad755e2682ab2786e381f6bf29e8ca
Once an Optional API module is loaded it should stay loaded. Unloading
an optional API module runs the risk of a crash if something else is
using it. This patch causes all optional API providers to tell the
module loader not to unload except at shutdown.
ASTERISK-27389
Change-Id: Ia07786fe655681aec49cc8d3d96e06483b11f5e6
In all non-pbx modules, AST_MODULE_LOAD_FAILURE has been changed
to AST_MODULE_LOAD_DECLINE. This prevents asterisk from exiting
if a module can't be loaded. If the user wishes to retain the
FAILURE behavior for a specific module, they can use the "require"
or "preload-require" keyword in modules.conf.
A new API was added to logger: ast_is_logger_initialized(). This
allows asterisk.c/check_init() to print to the error log once the
logger subsystem is ready instead of just to stdout. If something
does fail before the logger is initialized, we now print to stderr
instead of stdout.
Change-Id: I5f4b50623d9b5a6cb7c5624a8c5c1274c13b2b25
When doing some WebRTC testing, I found that the websocket would
disconnect whenever I attempted to place a call into Asterisk. After
looking into it, I pinpointed the problem to be due to the iostreams
change being merged in.
Under certain circumstances, a call to ast_iostream_read() can return a
negative value. However, in this circumstance, the websocket code was
treating this negative return as if it were a partial read from the
websocket. The expected length would get adjusted by this negative
value, resulting in the expected length being too large.
This patch simply adds an if check to be sure that we are only updating
the expected length of a read when the return from a read is positive.
ASTERISK-26842 #close
Reported by Mark Michelson
Change-Id: Ib4423239828a013d27d7bc477d317d2f02db61ab
fopencookie/funclose is a non-standard API and should not be used
in portable software. Additionally, the way FILE's fd is used in
non-blocking mode is undefined behaviour and cannot be relied on.
This introduces internal abstraction for io streams, that allows
implementing the desired virtualization of read/write operations
with necessary timeout handling.
ASTERISK-24515 #close
ASTERISK-24517 #close
Change-Id: Id916aef418b665ced6a7489aef74908b6e376e85
Not surprisingly, using Respoke (and possibly other systems) it is
possible to blow past the 16k limit for a WebSocket packet size. This
patch bumps it up to 32k, which, at least for Respoke, is sufficient.
For now.
Because 32k is laughable on a LOW_MEMORY system (as is 16k, for that
matter), this patch adds a LOW_MEMORY directive that sets the buffer to
8k for systems who have asked for their reduced memory availability to
be considered.
Change-Id: Id235902537091b58608196844dc4b045e383cd2e
ASTERISK_REGISTER_FILE no longer has any purpose so this commit removes
all traces of it.
Previously exported symbols removed:
* __ast_register_file
* __ast_unregister_file
* ast_complete_source_filename
This also removes the mtx_prof static variable that was declared when
MTX_PROFILE was enabled. This variable was only used in lock.c so it
is now initialized in that file only.
ASTERISK-26480 #close
Change-Id: I1074af07d71f9e159c48ef36631aa432c86f9966
Updated ast_websocket_write to encode the entire frame in to one
write operation, to ensure that we don't end up with a situation
where the websocket header has been sent, while the body can not
be written.
Previous to August's patch in commit b9bd3c14, certain network
conditions could cause the header to be written, and then the
sub-sequent body to fail - which would cause the next successful
write to contain a new header, and a new body (resulting in
the peer receiving two headers - the second of which would be
read as part of the body for the first header).
This was patched to have both write operations individually fail
by closing the websocket.
In a case available to the submitter of this patch, the same
body which would consistently fail to write, would succeed
if written at the same time as the header.
This update merges the two operations in to one, adds debug messages
indicating the reason for a websocket connection being closed during
a write operation, and clarifies some variable names for code legibility.
Change-Id: I4db7a586af1c7a57184c31d3d55bf146f1a40598
Due to the use of ast_websocket_close in session termination it is
possible for the underlying socket to already be closed when the
session is terminated. This occurs when the close frame is attempted
to be written out but fails.
Change-Id: I7572583529a42a7dc911ea77a974d8307d5c0c8b
The res_http_websocket module will currently attempt to close
the WebSocket connection if fatal cases occur, such as when
attempting to write out data and being unable to. When the
fatal cases occur the code attempts to write a WebSocket close
frame out to have the remote side close the connection. If
writing this fails then the connection is not terminated.
This change forcefully terminates the connection if the
WebSocket is to be closed but is unable to send the close frame.
ASTERISK-25312 #close
Change-Id: I10973086671cc192a76424060d9ec8e688602845
We don't have a compatability function to fill in a missing htobe64; but
we already have one for the identical htonll.
Change-Id: Ic0a95db1c5b0041e14e6b127432fb533b97e4cac
Commit 39cc28f6ea attempted to fix a
test failure observed on 32 bit test agents by ensuring that a cast from
a 32 bit unsigned integer to a 64 bit unsigned integer was happening in
a predictable place. As it turns out, this did not cause test runs to
succeed.
This commit adds several redundant debug messages that print the payload
lengths of websocket frames. The idea here is that this commit will not
cause tests to succeed for the faulty test agent, but we might deduce
where the fault lies more easily this way by observing at what point the
expected value (537) changes to some ungangly huge number.
If you are wondering why something like this is being committed to the
branch, keep in mind that in commit
39cc28f6ea I noted that the observed test
failures only happen when automated tests are run. Attempts to run the
tests by hand manually on the test agent result in the tests passing.
Change-Id: I14a65c19d8af40dadcdbd52348de3b0016e1ae8d
We have seen a rash of test failures on a 32-bit build agent. Commit
48698a5e21 solved an obvious problem where
we were not encoding a 64-bit value correctly over the wire. This
commit, however, did not solve the test failures.
In the failing tests, ARI is attempting to send a 537 byte text frame
over a websocket. When sending a frame this small, 16 bits are all that
is required in order to encode the payload length on the websocket
frame. However, ast_websocket_write() thinks that the payload length is
greater than 65535 and therefore writes out a 64 bit payload length.
Inspecting this payload length, the lower 32 bits are exactly what we
would expect it to be, 537 in hex. The upper 32 bits, are junk values
that are not expected to be there.
In the failure, we are passing the result of strlen() to a function that
expects a uint64_t parameter to be passed in. strlen() returns a size_t,
which on this 32-bit machine is 32 bits wide. Normally, passing a 32-bit
unsigned value to somewhere where a 64-bit unsigned value is expected
would cause no problems. In fact, in manual runs of failing tests, this
works just fine. However, ast_websocket_write() uses the Asterisk
optional API, which means that rather than a simple function call, there
are a series of macros that are used for its declaration and
implementation. These macros may be causing some sort of error to occur
when converting from a 32 bit quantity to a 64 bit quantity.
This commit changes the logic by making existing ast_websocket_write()
calls use ast_websocket_write_string() instead. Within
ast_websocket_write_string(), the 64-bit converted strlen is saved in a
local variable, and that variable is passed to ast_websocket_write()
instead.
Note that this commit message is full of speculation rather than
certainty. This is because the observed test failures, while always
present in automated test runs, never occur when tests are manually
attempted on the same test agent. The idea behind this commit is to fix
a theoretical issue by performing changes that should, at the least,
cause no harm. If it turns out that this change does not fix the failing
tests, then this commit should be reverted.
Change-Id: I4458dd87d785ca322b89c152b223a540a3d23e67
Prior to ASTERISK-24988, the WebSocket handshake was resolved before Stasis
applications were registered. This was done such that the WebSocket would be
ready when an application is registered. However, by creating the WebSocket
first, the client had the ability to make requests for the Stasis application
it thought had been created with the initial handshake request. The inevitable
conclusion of this scenario was the cart being put before the horse.
ASTERISK-24988 resolved half of the problem by ensuring that the applications
were created and registered with Stasis prior to completing the handshake
with the client. While this meant that Stasis was ready when the client
received the green-light from Asterisk, it also meant that the WebSocket was
not yet ready for Stasis to dispatch messages.
This patch introduces a message queuing mechanism for delaying messages from
Stasis applications while the WebSocket is being constructed. When the ARI
event processor receives the message from the WebSocket that it is being
created, the event processor instantiates an event session which contains a
message queue. It then tries to create and register the requested applications
with Stasis. Messages that are dispatched from Stasis between this point and
the point at which the event processor is notified the WebSocket is ready, are
stashed in the queue. Once the WebSocket has been built, the queue's messages
are dispatched in the order in which they were originally received and the
queue is concurrently cleared.
ASTERISK-25181 #close
Reported By: Matt Jordan
Change-Id: Iafef7b85a2e0bf78c114db4c87ffc3d16d671a17
A test agent was continuously failing all ARI tests when run against
Asterisk 13. As it turns out, the reason for this is that on those test
runs, for some reason we decided to use the super extended 64 bit
payload length for websocket text frames instead of the extended 16 bit
payload length. For 64-bit payloads, the expected byte order over the
network is
7, 6, 5, 4, 3, 2, 1, 0
However, we were sending the payload as
3, 2, 1, 0, 7, 6, 5, 4
This meant that we were saying to expect an absolutely MASSIVE payload
to arrive. Since we did not follow through on this expected payload
size, the client would sit patiently waiting for the rest of the payload
to arrive until the test would time out.
With this change, we use the htobe64() function instead of htonl() so
that a 64-bit byte-swap is performed instead of a 32 bit byte-swap.
Change-Id: Ibcd8552392845fbcdd017a8c8c1043b7fe35964a
This change makes it so that when accepting a WebSocket
connection the HTTP response is sent as one packet instead of
fragmented. Browsers don't like it when you send it fragmented.
ASTERISK-25103
Change-Id: I9b82c4ec2949b0bce692ad0bf6f7cea9709e7f69
This patch updates http_websocket and its corresponding implementation
with a pre-session established callback. This callback allows for
WebSocket server consumers to be notified when a WebSocket connection is
attempted, but before we accept it. Consumers can choose to reject the
connection, if their application specific logic allows for it.
As a result, this patch pulls out the previously private
websocket_protocol struct and makes it public, as
ast_websocket_protocol. In order to preserve backwards compatibility
with existing modules, the existing APIs were left as-is, and new APIs
were added for the creation of the ast_websocket_protocol as well as for
adding a sub-protocol to a WebSocket server.
In particular, the following new API calls were added:
* ast_websocket_add_protocol2 - add a protocol to the core WebSocket
server
* ast_websocket_server_add_protocol2 - add a protocol to a specific
WebSocket server
* ast_websocket_sub_protocol_alloc - allocate a sub-protocol object.
Consumers can populate this with whatever callbacks they wish to
support, then add it to the core server or a specified server.
ASTERISK-24988
Reported by: Joshua Colp
Change-Id: Ibe0bbb30c17eec6b578071bdbd197c911b620ab2
Git does not support the ability to replace a token with a version
string during check-in. While it does have support for replacing a
token on clone, this is somewhat sub-optimal: the token is replaced
with the object hash, which is not particularly easy for human
consumption. What's more, in practice, the source file version was often
not terribly useful. Generally, when triaging bugs, the overall version
of Asterisk is far more useful than an individual SVN version of a file. As a
result, this patch removes Asterisk's support for showing source file
versions.
Specifically, it does the following:
* Rename ASTERISK_FILE_VERSION macro to ASTERISK_REGISTER_FILE, and
remove passing the version in with the macro. Other facilities
than 'core show file version' make use of the file names, such as
setting a debug level only on a specific file. As such, the act of
registering source files with the Asterisk core still has use. The
macro rename now reflects the new macro purpose.
* main/asterisk:
- Refactor the file_version structure to reflect that it no longer
tracks a version field.
- Remove the "core show file version" CLI command. Without the file
version, it is no longer useful.
- Remove the ast_file_version_find function. The file version is no
longer tracked.
- Rename ast_register_file_version/ast_unregister_file_version to
ast_register_file/ast_unregister_file, respectively.
* main/manager: Remove value from the Version key of the ModuleCheck
Action. The actual key itself has not been removed, as doing so would
absolutely constitute a backwards incompatible change. However, since
the file version is no longer tracked, there is no need to attempt to
include it in the Version key.
* UPGRADE: Add notes for:
- Modification to the ModuleCheck AMI Action
- Removal of the "core show file version" CLI command
Change-Id: I6cf0ff280e1668bf4957dc21f32a5ff43444a40e
Some WebSocket applications, like [chan_respoke][], require a larger
frame size than the default 8k; this patch bumps the default to 16k.
This patch also fixes some problems exacerbated by large frames.
The sanity counter was decremented on every fread attempt in
ws_safe_read(), regardless of whether data was read from the socket or
not. For large frames, this could result in loss of sanity prior to
reading the entire frame. (16k frame / 1448 bytes per segment = 12
segments).
This patch changes the sanity counter so that it only decrements when
fread() doesn't read any bytes. This more closely matches the original
intention of ws_safe_read(), given that the error message is
"Websocket seems unresponsive".
This patch also properly logs EOF conditions, so disconnects are no
longer confused with unresponsive connections.
[chan_respoke]: https://github.com/respoke/chan_respoke
Review: https://reviewboard.asterisk.org/r/4431/
........
Merged revisions 432236 from http://svn.asterisk.org/svn/asterisk/branches/11
........
Merged revisions 432237 from http://svn.asterisk.org/svn/asterisk/branches/13
git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@432238 65c4cc65-6c06-0410-ace0-fbb531ad65f3
George Joseph
Sean Bright
Joshua C. Colp
Kevin Harwell
Alexander Traud
Nickolay Shmyrev
Moises Silva
Sean Bright
Jeremy Lainé
Corey Farrell
Kevin Harwell
Joshua Colp
George Joseph
Mark Michelson
Badalyan Vyacheslav
Timo Teräs
Matt Jordan
Dade Brandon
Richard Mudgett
David M. Lee
Ashley Sanders
Rodrigo Ramírez Norambuena
David M. Lee