Commit Graph

32 Commits

Author SHA1 Message Date
Joshua Colp 0231dd6ae7 stasis: Improve topic/subscription names and statistics.
Topic names now follow: <subsystem>:<functionality>[/<object>]

This ensures that they are all unique, and also provides better
insight in to what each topic is for.

Subscriber ids now also use the main topic name they are
subscribed to and an incrementing integer as their identifier to
make it easier to understand what the subscription is primarily
responsible for.

Both the CLI commands for listing topic and subscription statistics
now sort to make it a bit easier to see what is going on.

Subscriptions will now show all topics that they are receiving messages
from, not just the main topic they were subscribed to.

ASTERISK-28335

Change-Id: I484e971a38c3640f2bd156282e532eed84bf220d
2019-03-11 11:39:35 -03:00
Joshua Colp 50ac85cb40 stasis: Segment channel snapshot to reduce creation cost.
When a channel snapshot was created it used to be done
from scratch, copying all data (many strings). This incurs
a cost when doing so.

This change segments the channel snapshot into different
components which can be reused if unchanged from the
previous snapshot creation, reducing the cost. In normal
cases this results in some pointers being copied with
reference count being bumped, some integers being set,
and a string or two copied. The other benefit is that it
is now possible to determine if a channel snapshot update
is redundant and thus stop it before a message is published
to stasis.

The specific segments in the channel snapshot were split up
based on whether they are changed together, how often they
are changed, and their general grouping. In practice only
1 (or 0) of the segments actually get changed in normal
operation.

Invalidation is done by setting a flag on the channel when
the segment source is changed, forcing creation of a new
segment when the channel snapshot is created.

ASTERISK-28119

Change-Id: I5d7ef3df963a88ac47bc187d73c5225c315f8423
2018-11-26 12:56:24 -06:00
Joshua Colp d0ccbb3377 stasis: Use an implementation specific channel snapshot cache.
Channels no longer use the Stasis cache for channel snapshots. Instead
they are stored in a hash table in stasis_channels which reduces the
number of Stasis messages created and allows better storage.

As a result the following APIs are no longer available since the stasis
cache is no longer used:
ast_channel_topic_cached()
ast_channel_topic_all_cached()

The ast_channel_cache_all() and ast_channel_cache_by_name() functions
now return an ao2_container of ast_channel_snapshots rather than
a container of stasis_messages therefore you can't (and don't need
to) call stasis_cache functions on it.

The ast_channel_topic_all() function now returns a normal topic not
a cached one so you can't use stasis cache functions on it either.

The ast_channel_snapshot_type() stasis message now has the
ast_channel_snapshot_update structure as it's data. It contains the
last snapshot and the new one.

ast_channel_snapshot_get_latest() still returns the latest snapshot.

The latest snapshot is now stored on the channel itself to eliminate
cache hits when Stasis messages that have the snapshot as a payload
are created.

ASTERISK-28102

Change-Id: I9334febff60a82d7c39703e49059fa3a68825786
2018-11-26 18:43:53 +00:00
Corey Farrell 021ce938ca
astobj2: Remove legacy ao2_container_alloc routine.
Replace usage of ao2_container_alloc with ao2_container_alloc_hash or
ao2_container_alloc_list.  Remove ao2_container_alloc macro.

Change-Id: I0907d78bc66efc775672df37c8faad00f2f6c088
2018-11-21 09:56:16 -05:00
Joshua Colp 3077ad0c24 stasis: Add internal filtering of messages.
This change adds the ability for subscriptions to indicate
which message types they are interested in accepting. By
doing so the filtering is done before being dispatched
to the subscriber, reducing the amount of work that has
to be done.

This is optional and if a subscriber does not add
message types they wish to accept and set the subscription
to selective filtering the previous behavior is preserved
and they receive all messages.

There is also the ability to explicitly force the reception
of all messages for cases such as AMI or ARI where a large
number of messages are expected that are then generically
converted into a different format.

ASTERISK-28103

Change-Id: I99bee23895baa0a117985d51683f7963b77aa190
2018-11-18 15:08:16 -05:00
Corey Farrell bc73337e07 core: Use macros to generate ao2_container callbacks where possible.
This uses AO2_STRING_FIELD_HASH_FN and AO2_STRING_FIELD_CMP_FN where
possible in the Asterisk core.

This removes CMP_STOP from the result of CMP_FN callbacks for the
following structure types:
* ast_bucket_metadata
* ast_bucket_scheme
* generic_monitor_instance_list (ccss.c)
* ast_bucket_file (media_cache.c)
* named_acl

Change-Id: Ide4c1449a894bce70dea1fef664dade9b57578f1
2017-12-30 13:20:16 -05:00
Corey Farrell a6e5bae3ef Remove ASTERISK_REGISTER_FILE.
ASTERISK_REGISTER_FILE no longer has any purpose so this commit removes
all traces of it.

Previously exported symbols removed:
* __ast_register_file
* __ast_unregister_file
* ast_complete_source_filename

This also removes the mtx_prof static variable that was declared when
MTX_PROFILE was enabled.  This variable was only used in lock.c so it
is now initialized in that file only.

ASTERISK-26480 #close

Change-Id: I1074af07d71f9e159c48ef36631aa432c86f9966
2016-10-27 09:53:55 -04:00
George Joseph 22db16fa81 endpoint/stasis: Eliminate duplicate events on endpoint status change
When an endpoint is created, its messages are forwarded to both the tech
endpoint topic and the all endpoints topic. This is done so that various
parties interested in endpoint messages can subscribe to just the tech
endpoint and receive all messages associated with that particular technology,
as opposed to subscribing to the all endpoints topic. Unfortunately, when the
tech endpoint is created, it also forwards all of its messages to the all
topic. This results in duplicate messages whenever an endpoint publishes its
messages.

This patch resolves the duplicate message issue by creating a new function
for Stasis caching topics, stasis_cp_sink_create. In most respects, this acts
as a normal caching topic, save that it no longer forwards messages it receives
to the all endpoints topic. This allows it to act as an aggregation "sink",
while preserving the necessary caching behaviour.

ASTERISK-25137 #close
Reported-by: Vitezslav Novy

ASTERISK-25116 #close
Reported-by: George Joseph <george.joseph@fairview5.com>
Tested-by: George Joseph <george.joseph@fairview5.com>

Change-Id: Ie47784adfb973ab0063e59fc18f390d7dd26d17b
2015-12-28 11:39:26 -06:00
Scott Griepentrog 7691035312 endpoint snapshot: avoid second cleanup on alloc failure
In ast_endpoint_snapshot_create(), a failure to init the
string fields results in two attempts to ao2_cleanup the
same pointer.  Removed RAII_VAR to eliminate problem.

ASTERISK-25375 #close
Reported by: Scott Griepentrog

Change-Id: If4d9dfb1bbe3836b623642ec690b6d49b25e8979
2015-09-04 09:26:46 -05:00
George Joseph 82716410a4 Revert "endpoint/stasis: Eliminate duplicate events on endpoint status change"
This reverts commit 6fca75bb62.

Change-Id: Ifee026cc63e22c5ac5717c37867a9f036373ae5a
2015-05-29 14:52:41 -05:00
George Joseph 6fca75bb62 endpoint/stasis: Eliminate duplicate events on endpoint status change
When an endpoint was created, it's messages were being forwarded to
both the tech endpoint topic and the all endpoints topic.  Since
the tech topic was also forwarded to all, this was resulting in
duplicate messages whenever an endpoint published.  This patch
causes the endpoint to only forward to the tech topic and lets
the tech topic forward to all.

To accomplish this, the existing stasis_cp_single_create function
(which both creates and forwards) was cloned and split into 2
functions, one that creates the topic and one that sets up the
forwarding.  This allows endpoint_internal_create to create
the topic from the endpoint_all cache without forwarding it there,
then allows it to do the forward to the tech's topic.

ASTERISK-25137 #close
Reported-by: Vitezslav Novy
ASTERISK-25116 #close
Reported-by: George Joseph <george.joseph@fairview5.com>
Tested-by: George Joseph <george.joseph@fairview5.com>

Change-Id: I26d7d4926a0861748fd3bdffe316b75b549a801c
2015-05-27 17:16:33 -05:00
George Joseph 51886c68dc pjsip_options: Add qualify_timeout processing and eventing
This is the second follow-on to https://reviewboard.asterisk.org/r/4572/ and the
discussion at
http://lists.digium.com/pipermail/asterisk-dev/2015-March/073921.html

The basic issues are that changes in contact status don't cause events to be
emitted for the associated endpoint.  Only dynamic contact add/delete actions
update the endpoint.  Also, the qualify timeout is fixed by pjsip at 32 seconds
which is a long time.

This patch makes use of the new transaction timeout feature in r4585 and
provides the following capabilities...

1.  A new aor/contact variable 'qualify_timeout' has been added that allows the
user to specify the maximum time in milliseconds to wait for a response to an
OPTIONS message.  The default is 3000ms.  When the timer expires, the contact is
marked unavailable.

2.  Contact status changes are now propagated up to the endpoint as follows...
When any contact is 'Available', the endpoint is marked as 'Reachable'.  When
all contacts are 'Unavailable', the endpoint is marked as 'Unreachable'.  The
existing endpoint events are generated appropriately.

ASTERISK-24863 #close

Change-Id: Id0ce0528e58014da1324856ea537e7765466044a
Tested-by: Dmitriy Serov
Tested-by: George Joseph <george.joseph@fairview5.com>
2015-04-16 09:34:56 -05:00
Matt Jordan 4a58261694 git migration: Refactor the ASTERISK_FILE_VERSION macro
Git does not support the ability to replace a token with a version
string during check-in. While it does have support for replacing a
token on clone, this is somewhat sub-optimal: the token is replaced
with the object hash, which is not particularly easy for human
consumption. What's more, in practice, the source file version was often
not terribly useful. Generally, when triaging bugs, the overall version
of Asterisk is far more useful than an individual SVN version of a file. As a
result, this patch removes Asterisk's support for showing source file
versions.

Specifically, it does the following:

* Rename ASTERISK_FILE_VERSION macro to ASTERISK_REGISTER_FILE, and
  remove passing the version in with the macro. Other facilities
  than 'core show file version' make use of the file names, such as
  setting a debug level only on a specific file. As such, the act of
  registering source files with the Asterisk core still has use. The
  macro rename now reflects the new macro purpose.

* main/asterisk:
  - Refactor the file_version structure to reflect that it no longer
    tracks a version field.
  - Remove the "core show file version" CLI command. Without the file
    version, it is no longer useful.
  - Remove the ast_file_version_find function. The file version is no
    longer tracked.
  - Rename ast_register_file_version/ast_unregister_file_version to
    ast_register_file/ast_unregister_file, respectively.

* main/manager: Remove value from the Version key of the ModuleCheck
  Action. The actual key itself has not been removed, as doing so would
  absolutely constitute a backwards incompatible change. However, since
  the file version is no longer tracked, there is no need to attempt to
  include it in the Version key.

* UPGRADE: Add notes for:
  - Modification to the ModuleCheck AMI Action
  - Removal of the "core show file version" CLI command

Change-Id: I6cf0ff280e1668bf4957dc21f32a5ff43444a40e
2015-04-13 03:48:57 -04:00
Scott Griepentrog 8c65c9167e Various: bugfixes found via chaos
Using DEBUG_CHAOS several instances of a null
pointer crash, and one uninitialized variable
were uncovered and fixed.  Also added details
on why Asterisk failed to initialize.

Review: https://reviewboard.asterisk.org/r/4468/
........

Merged revisions 433064 from http://svn.asterisk.org/svn/asterisk/branches/13


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@433065 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2015-03-17 22:15:42 +00:00
Joshua Colp 2889f074a0 core: Don't create snapshots with locks.
Snapshots are immutable and are never changed. Allocating them
with a lock is wasteful.

Review: https://reviewboard.asterisk.org/r/4469/
........

Merged revisions 432742 from http://svn.asterisk.org/svn/asterisk/branches/13


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@432743 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2015-03-10 23:09:49 +00:00
Matthew Jordan 1106e8fd0f main/stasis: Allow subscriptions to use a threadpool for message delivery
Prior to this patch, all Stasis subscriptions would receive a dedicated
thread for servicing published messages. In contrast, prior to r400178
(see review https://reviewboard.asterisk.org/r/2881/), the subscriptions
shared a thread pool. It was discovered during some initial work on Stasis
that, for a low subscription count with high message throughput, the
threadpool was not as performant as simply having a dedicated thread per
subscriber.

For situations where a subscriber receives a substantial number of messages
and is always present, the model of having a dedicated thread per subscriber
makes sense. While we still have plenty of subscriptions that would follow
this model, e.g., AMI, CDRs, CEL, etc., there are plenty that also fall into
the following two categories:
* Large number of subscriptions, specifically those tied to endpoints/peers.
* Low number of messages. Some subscriptions exist specifically to coordinate
  a single message - the subscription is created, a message is published, the
  delivery is synchronized, and the subscription is destroyed.
In both of the latter two cases, creating a dedicated thread is wasteful (and
in the case of a large number of peers/endpoints, harmful). In those cases,
having shared delivery threads is far more performant.

This patch adds the ability of a subscriber to Stasis to choose whether or not
their messages are dispatched on a dedicated thread or on a threadpool. The
threadpool is configurable through stasis.conf.

Review: https://reviewboard.asterisk.org/r/4193

ASTERISK-24533 #close
Reported by: xrobau
Tested by: xrobau
........

Merged revisions 428681 from http://svn.asterisk.org/svn/asterisk/branches/12
........

Merged revisions 428687 from http://svn.asterisk.org/svn/asterisk/branches/13


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@428688 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-12-01 17:59:21 +00:00
Kinsey Moore f1036f40dc Stasis: Allow message types to be blocked
This introduces stasis.conf and a mechanism to prevent certain message
types from being published. Internally, this works by preventing the
chosen message types from being created which ensures that those
message types can never be published. This patch also adjusts message
publishers such that message payloads are not created if the related
message type is not available.

ASTERISK-23943 #close
Review: https://reviewboard.asterisk.org/r/3823/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@420124 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-08-06 12:55:28 +00:00
Matthew Jordan ccc6e8bd17 endpoints: Fix failing unit tests from r419196
This patch does two things:
(1) It updates the unit tests to expect additional stasis messages. More
    messages are now sent to the endpoint topic, due to forwarding all
    channel messages and the forwarding relationship set up between
    endpoints themselves.
(2) Remove the technology forwarding subscription during
    ast_endpoint_shutdown. This prevents an improper double shutdown of
    an endpoint from occurring.
........

Merged revisions 419318 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@419319 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-07-23 16:46:13 +00:00
Matthew Jordan bb87796f67 ARI: Fix endpoint/channel subscription issues; allow for subscriptions to tech
This patch serves two purposes:
(1) It fixes some bugs with endpoint subscriptions not reporting all of the
    channel events
(2) It serves as the preliminary work needed for ASTERISK-23692, which allows
    for sending/receiving arbitrary out of call text messages through ARI in a
    technology agnostic fashion.

The messaging functionality described on ASTERISK-23692 requires two things:
(1) The ability to send/receive messages associated with an endpoint. This is
    relatively straight forwards with the endpoint core in Asterisk now.
(2) The ability to send/receive messages associated with a technology and an
    arbitrary technology defined URI. This is less straight forward, as
    endpoints are formed from a tech + resource pair. We don't have a
    mechanism to note that a technology that *may* have endpoints exists.

This patch provides such a mechanism, and fixes a few bugs along the way.

The first major bug this patch fixes is the forwarding of channel messages
to their respective endpoints. Prior to this patch, there were two problems:
(1) Channel caching messages weren't forwarded. Thus, the endpoints missed
    most of the interesting bits (such as channel creation, destruction, state
    changes, etc.)
(2) Channels weren't associated with their endpoint until after creation.
    This resulted in endpoints missing the channel creation message, which
    limited the usefulness of the subscription in the first place (a major use
    case being 'tell me when this endpoint has a channel'). Unfortunately,
    this meant another parameter to ast_channel_alloc. Since not all channel
    technologies support an ast_endpoint, this patch makes such a call
    optional and opts for a new function, ast_channel_alloc_with_endpoint.

When endpoints are created, they will implicitly create a technology endpoint
for their technology (if one does not already exist). A technology endpoint is
special in that it has no state, cannot have channels created for it, cannot
be created explicitly, and cannot be destroyed except on shutdown. It does,
however, have all messages from other endpoints in its technology forwarded to
it.

Combined with the bug fixes, we now have Stasis messages being properly
forwarded. Consider the following scenario: two PJSIP endpoints (foo and bar),
where bar has a single channel associated with it and foo has two channels
associated with it. The messages would be forwarded as follows:

channel PJSIP/foo-1 --
                      \
                       --> endpoint PJSIP/foo --
                      /                         \
channel PJSIP/foo-2 --                           \
                                                  ---- > endpoint PJSIP
                                                /
channel PJSIP/bar-1 -----> endpoint PJSIP/bar --

ARI, through the applications resource, can:
 - subscribe to endpoint:PJSIP/foo and get notifications for channels
   PJSIP/foo-1,PJSIP/foo-2 and endpoint PJSIP/foo
 - subscribe to endpoint:PJSIP/bar and get notifications for channels
   PJSIP/bar-1 and endpoint PJSIP/bar
 - subscribe to endpoint:PJSIP and get notifications for channels
   PJSIP/foo-1,PJSIP/foo-2,PJSIP/bar-1 and endpoints PJSIP/foo,PJSIP/bar

Note that since endpoint PJSIP never changes, it never has events itself. It
merely provides an aggregation point for all other endpoints in its technology
(which in turn aggregate all channel messages associated with that endpoint).

This patch also adds endpoints to res_xmpp and chan_motif, because the actual
messaging work will need it (messaging without XMPP is just sad).

Review: https://reviewboard.asterisk.org/r/3760/

ASTERISK-23692
........

Merged revisions 419196 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@419203 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-07-22 16:20:58 +00:00
Matthew Jordan 9cc1a8e893 stasis: Reduce creation of channel snapshots to improve performance
During some performance testing of Asterisk with AGI, ARI, and lots of Local
channels, we noticed that there's quite a hit in performance during channel
creation and releasing to the dialplan (ARI continue). After investigating
the performance spike that occurs during channel creation, we discovered
that we create a lot of channel snapshots that are technically unnecessary.
This includes creating snapshots during:
 * AGI execution
 * Returning objects for ARI commands
 * During some Local channel operations
 * During some dialling operations
 * During variable setting
 * During some bridging operations
And more.

This patch does the following:
 - It removes a number of fields from channel snapshots. These fields were
   rarely used, were expensive to have on the snapshot, and hurt performance.
   This included formats, translation paths, Log Call ID, callgroup, pickup
   group, and all channel variables. As a result, AMI Status,
   "core show channel", "core show channelvar", and "pjsip show channel" were
   modified to either hit the live channel or not show certain pieces of data.
   While this is unfortunate, the performance gain from this patch is worth
   the loss in behaviour.
 - It adds a mechanism to publish a cached snapshot + blob. A large number of
   publications were changed to use this, including:
   - During Dial begin
   - During Variable assignment (if no AMI variables are emitted - if AMI
     variables are set, we have to make snapshots when a variable is changed)
   - During channel pickup
   - When a channel is put on hold/unhold
   - When a DTMF digit is begun/ended
   - When creating a bridge snapshot
   - When an AOC event is raised
   - During Local channel optimization/Local bridging
   - When endpoint snapshots are generated
   - All AGI events
   - All ARI responses that return a channel
   - Events in the AgentPool, MeetMe, and some in Queue
 - Additionally, some extraneous channel snapshots were being made that were
   unnecessary. These were removed.
 - The result of ast_hashtab_hash_string is now cached in stasis_cache. This
   reduces a large number of calls to ast_hashtab_hash_string, which reduced
   the amount of time spent in this function in gprof by around 50%.

#ASTERISK-23811 #close
Reported by: Matt Jordan

Review: https://reviewboard.asterisk.org/r/3568/
........

Merged revisions 416211 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@416216 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2014-06-13 18:24:49 +00:00
Kevin Harwell 28c0cb28d0 channel locking: Add locking for channel snapshot creation
Original commit message by mmichelson (asterisk 12 r403311):

"This adds channel locks around calls to create channel snapshots as well
as other functions which operate on a channel and then end up
creating a channel snapshot. Functions that expect the channel to be
locked prior to being called have had their documentation updated to
indicate such."

The above was initially committed and then reverted at r403398.  The problem
was found to be in core_local.c in the publish_local_bridge_message function.
The ast_unreal_lock_all function locks and adds a reference to the returned
channels and while they were being unlocked they were not being unreffed when
no longer needed.  Fixed by unreffing the channels.

Also in bridge.c a lock was obtained on "other->chan", but then an attempt was
made to unlock "other" and not the previously locked channel.  Fixed by
unlocking "other->chan"

(closes issue ASTERISK-22709)
Reported by: John Bigelow
........

Merged revisions 404237 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@404260 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-12-18 20:33:37 +00:00
Joshua Colp dcb642e2da endpoints: Keep a reference to channel ids when creating snapshot.
The snapshot process for endpoints uses the channel ids present
on the endpoint itself. Without keeping a reference it was possible
for the strings to be freed underneath any consumer of an endpoint
snapshot.

A reference is now held by the snapshot to the channel ids and
released when the snapshot is destroyed.

(issue ASTERISK-22801)
Reported by: Matt Jordan
........

Merged revisions 403542 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@403543 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-12-09 18:32:02 +00:00
David M. Lee 1212906351 Reverting r403311. It's causing ARI tests to hang.
........

Merged revisions 403398 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@403404 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-12-05 22:10:20 +00:00
Mark Michelson 8e8b329e14 Add channel locking for channel snapshot creation.
This adds channel locks around calls to create channel snapshots as well
as other functions which operate on a channel and then end up
creating a channel snapshot. Functions that expect the channel to be
locked prior to being called have had their documentation updated to
indicate such.
........

Merged revisions 403311 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@403314 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-12-03 17:07:29 +00:00
Matthew Jordan 8d7873b836 ARI: Add subscription support
This patch adds an /applications API to ARI, allowing explicit management of
Stasis applications.

 * GET /applications - list current applications
 * GET /applications/{applicationName} - get details of a specific application
 * POST /applications/{applicationName}/subscription - explicitly subscribe to
   a channel, bridge or endpoint
 * DELETE /applications/{applicationName}/subscription - explicitly unsubscribe
   from a channel, bridge or endpoint

Subscriptions work by a reference counting mechanism: if you subscript to an
event source X number of times, you must unsubscribe X number of times to stop
receiveing events for that event source.

Review: https://reviewboard.asterisk.org/r/2862

(issue ASTERISK-22451)
Reported by: Matt Jordan
........

Merged revisions 400522 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@400523 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-10-04 16:01:48 +00:00
David M. Lee 2de42c2a25 Multiple revisions 399887,400138,400178,400180-400181
........
  r399887 | dlee | 2013-09-26 10:41:47 -0500 (Thu, 26 Sep 2013) | 1 line
  
  Minor performance bump by not allocate manager variable struct if we don't need it
........
  r400138 | dlee | 2013-09-30 10:24:00 -0500 (Mon, 30 Sep 2013) | 23 lines
  
  Stasis performance improvements
  
  This patch addresses several performance problems that were found in
  the initial performance testing of Asterisk 12.
  
  The Stasis dispatch object was allocated as an AO2 object, even though
  it has a very confined lifecycle. This was replaced with a straight
  ast_malloc().
  
  The Stasis message router was spending an inordinate amount of time
  searching hash tables. In this case, most of our routers had 6 or
  fewer routes in them to begin with. This was replaced with an array
  that's searched linearly for the route.
  
  We more heavily rely on AO2 objects in Asterisk 12, and the memset()
  in ao2_ref() actually became noticeable on the profile. This was
  #ifdef'ed to only run when AO2_DEBUG was enabled.
  
  After being misled by an erroneous comment in taskprocessor.c during
  profiling, the wrong comment was removed.
  
  Review: https://reviewboard.asterisk.org/r/2873/
........
  r400178 | dlee | 2013-09-30 13:26:27 -0500 (Mon, 30 Sep 2013) | 24 lines
  
  Taskprocessor optimization; switch Stasis to use taskprocessors
  
  This patch optimizes taskprocessor to use a semaphore for signaling,
  which the OS can do a better job at managing contention and waiting
  that we can with a mutex and condition.
  
  The taskprocessor execution was also slightly optimized to reduce the
  number of locks taken.
  
  The only observable difference in the taskprocessor implementation is
  that when the final reference to the taskprocessor goes away, it will
  execute all tasks to completion instead of discarding the unexecuted
  tasks.
  
  For systems where unnamed semaphores are not supported, a really
  simple semaphore implementation is provided. (Which gives identical
  performance as the original taskprocessor implementation).
  
  The way we ended up implementing Stasis caused the threadpool to be a
  burden instead of a boost to performance. This was switched to just
  use taskprocessors directly for subscriptions.
  
  Review: https://reviewboard.asterisk.org/r/2881/
........
  r400180 | dlee | 2013-09-30 13:39:34 -0500 (Mon, 30 Sep 2013) | 28 lines
  
  Optimize how Stasis forwards are dispatched
  
  This patch optimizes how forwards are dispatched in Stasis.
  
  Originally, forwards were dispatched as subscriptions that are invoked
  on the publishing thread. This did not account for the vast number of
  forwards we would end up having in the system, and the amount of work it
  would take to walk though the forward subscriptions.
  
  This patch modifies Stasis so that rather than walking the tree of
  forwards on every dispatch, when forwards and subscriptions are changed,
  the subscriber list for every topic in the tree is changed.
  
  This has a couple of benefits. First, this reduces the workload of
  dispatching messages. It also reduces contention when dispatching to
  different topics that happen to forward to the same aggregation topic
  (as happens with all of the channel, bridge and endpoint topics).
  
  Since forwards are no longer subscriptions, the bulk of this patch is
  simply changing stasis_subscription objects to stasis_forward objects
  (which, admittedly, I should have done in the first place.)
  
  Since this required me to yet again put in a growing array, I finally
  abstracted that out into a set of ast_vector macros in
  asterisk/vector.h.
  
  Review: https://reviewboard.asterisk.org/r/2883/
........
  r400181 | dlee | 2013-09-30 13:48:57 -0500 (Mon, 30 Sep 2013) | 28 lines
  
  Remove dispatch object allocation from Stasis publishing
  
  While looking for areas for performance improvement, I realized that an
  unused feature in Stasis was negatively impacting performance.
  
  When a message is sent to a subscriber, a dispatch object is allocated
  for the dispatch, containing the topic the message was published to, the
  subscriber the message is being sent to, and the message itself.
  
  The topic is actually unused by any subscriber in Asterisk today. And
  the subscriber is associated with the taskprocessor the message is being
  dispatched to.
  
  First, this patch removes the unused topic parameter from Stasis
  subscription callbacks.
  
  Second, this patch introduces the concept of taskprocessor local data,
  data that may be set on a taskprocessor and provided along with the data
  pointer when a task is pushed using the ast_taskprocessor_push_local()
  call. This allows the task to have both data specific to that
  taskprocessor, in addition to data specific to that invocation.
  
  With those two changes, the dispatch object can be removed completely,
  and the message is simply refcounted and sent directly to the
  taskprocessor.
  
  Review: https://reviewboard.asterisk.org/r/2884/
........

Merged revisions 399887,400138,400178,400180-400181 from http://svn.asterisk.org/svn/asterisk/branches/12


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@400186 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-09-30 18:55:27 +00:00
David M. Lee e1b959ccbb Split caching out from the stasis_caching_topic.
In working with res_stasis, I discovered a significant limitation to
the current structure of stasis_caching_topics: you cannot subscribe
to cache updates for a single channel/bridge/endpoint/etc.

To address this, this patch splits the cache away from the
stasis_caching_topic, making it a first class object. The stasis_cache
object is shared amongst individual stasis_caching_topics that are
created per channel/endpoint/etc. These are still forwarded to global
whatever_all_cached topics, so their use from most of the code does
not change.

In making these changes, I noticed that we frequently used a similar
pattern for bridges, endpoints and channels:

     single_topic  ---------------->  all_topic
           ^
           |
     single_topic_cached  ----+---->  all_topic_cached
                              |
                              +---->  cache

This pattern was extracted as the 'Stasis Caching Pattern', defined in
stasis_caching_pattern.h. This avoids a lot of duplicate code between
the different domain objects.

Since the cache is now disassociated from its upstream caching topics,
this also necessitated a change to how the 'guaranteed' flag worked
for retrieving from a cache. The code for handling the caching
guarantee was extracted into a 'stasis_topic_wait' function, which
works for any stasis_topic.

(closes issue ASTERISK-22002)
Review: https://reviewboard.asterisk.org/r/2672/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@395954 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-08-01 13:49:34 +00:00
Joshua Colp 65c492e851 Add support for requiring that all queued messages on a caching topic have been handled before
retrieving from the cache and also change adding channels to an endpoint to be an immediate
operation.

Review: https://reviewboard.asterisk.org/r/2599/


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@391596 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-06-13 11:02:16 +00:00
Matthew Jordan 272dd008d0 Fix memory leak while loading modules, adding formats, and destroying endpoints
This patch fixes three memory leaks
 * When we load a module with the LOAD_PRIORITY flag, we remove its entry from
   the load order list. Unfortunately, we don't free the memory associated with
   entry in the list. This patch corrects that and properly frees the memory
   for the module in the list.

 * When adding a custom format (such as SILK or CELT), the routine for adding
   the format was leaking a reference. RAII_VAR cleans this up properly.

 * We now de-ref the channel_snapshot appropriately when an endpoint is
   disposed of
........

Merged revisions 391489 from http://svn.asterisk.org/svn/asterisk/branches/1.8
........

Merged revisions 391507 from http://svn.asterisk.org/svn/asterisk/branches/11


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@391521 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-06-12 02:29:08 +00:00
Kinsey Moore 759a7e4a30 Rework stasis cache clear events
Stasis cache clear message payloads now consist of a stasis_message
representative of the message to be cleared from the cache. This allows
multiple parallel caches to coexist and be cleared properly by the same
cache clear message even when keyed on different fields.

This change fixes a bug where multiple cache clears could be posted for
channels. The cache clear is now produced in the destructor instead of
ast_hangup.

Additionally, dummy channels are no longer capable of producing channel
snapshots.

Review: https://reviewboard.asterisk.org/r/2596


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@390830 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-06-07 12:56:56 +00:00
David M. Lee b97c71bb11 Fix shutdown assertions in stasis-core
In r388005, macros were introduced to consistently define message
types. This added an assert if a message type was used either before
it was initialized or after it had been cleaned up. It turns out that
this assertion fires during shutdown.

This actually exposed a hidden shutdown ordering problem. Since
unsubscribing is asynchronous, it's possible that the message types
used by the subscription could be freed before the final message of
the subscription was processed.

This patch adds stasis_subscription_join(), which blocks until the
last message has been processed by the subscription. Since joining was
most commonly done right after an unsubscribe, a
stasis_unsubscribe_and_join() convenience function was also added.

Similar functions were also added to the stasis_caching_topic and
stasis_message_router, since they wrap subscriptions and have similar
problems.

Other code in trunk was refactored to join() where appropriate, or at
least verify that the subscription was complete before being
destroyed.

Review: https://reviewboard.asterisk.org/r/2540


git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@389011 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-05-17 21:10:32 +00:00
David M. Lee e06e519a90 Initial support for endpoints.
An endpoint is an external device/system that may offer/accept
channels to/from Asterisk. While this is a very useful concept for end
users, it is surprisingly not a core concept within Asterisk itself.

This patch defines ast_endpoint as a separate object, which channel
drivers may use to expose their concept of an endpoint. As the channel
driver creates channels, it can use ast_endpoint_add_channel() to
associate channels to the endpoint. This updated the endpoint
appropriately, and forwards all of the channel's events to the
endpoint's topic.

In order to avoid excessive locking on the endpoint object itself, the
mutable state is not accessible via getters. Instead, you can create a
snapshot using ast_endpoint_snapshot_create() to get a consistent
snapshot of the internal state.

This patch also includes a set of topics and messages associated with
endpoints, and implementations of the endpoint-related RESTful
API. chan_sip was updated to create endpoints with SIP peers, but the
state of the endpoints is not updated with the state of the peer.

Along for the ride in this patch is a Stasis test API. This is a
stasis_message_sink object, which can be subscribed to a Stasis
topic. It has functions for blocking while waiting for conditions in
the message sink to be fulfilled.

(closes issue ASTERISK-21421)
Review: https://reviewboard.asterisk.org/r/2492/



git-svn-id: https://origsvn.digium.com/svn/asterisk/trunk@387932 65c4cc65-6c06-0410-ace0-fbb531ad65f3
2013-05-08 13:39:08 +00:00