When both Asterisk and a UA send re-invites at the same time, both
send 491 "Transaction in progress" responses to each other and back
off a specified amount of time before retrying. When Asterisk
prepares to send its re-invite, it sets up the session's pending
media state with the new topology it wants, then sends the
re-invite. Unfortunately, when it received the re-invite from the
UA, it partially processed the media in the re-invite and reset
the pending media state before sending the 491 losing the state it
set in its own re-invite.
Asterisk also was not tracking re-invites received while an existing
re-invite was queued resulting in sending stale SDP with missing
or duplicated streams, or no re-invite at all because we erroneously
determined that a re-invite wasn't needed.
There was also an issue in bridge_softmix where we were using a stream
from the wrong topology to determine if a stream was added. This also
caused us to erroneously determine that a re-invite wasn't needed.
Regardless of how the delayed re-invite was triggered, we need to
reconcile the topology that was active at the time the delayed
request was queued, the pending topology of the queued request,
and the topology currently active on the session. To do this we
need a topology resolver AND we need to make stream named unique
so we can accurately tell what a stream has been added or removed
and if we can re-use a slot in the topology.
Summary of changes:
* bridge_softmix:
* We no longer reset the stream name to "removed" in
remove_all_original_streams(). That was causing multiple streams
to have the same name and wrecked the checks for duplicate streams.
* softmix_bridge_stream_sources_update() was checking the old_stream
to see if it had the softmix prefix and not considering the stream
as "new" if it did. If the stream in that slot has something in it
because another re-invite happened, then that slot in old might
have a softmix stream but the same stream in new might actually
be a new one. Now we check the new_stream's name instead of
the old_stream's.
* stream:
* Instead of using plain media type name ("audio", "video", etc) as
the default stream name, we now append the stream position to it
to make it unique. We need to do this so we can distinguish multiple
streams of the same type from each other.
* When we set a stream's state to REMOVED, we no longer reset its
name to "removed" or destroy its metadata. Again, we need to
do this so we can distinguish multiple streams of the same
type from each other.
* res_pjsip_session:
* Added resolve_refresh_media_states() that takes in 3 media states
and creates an up-to-date pending media state that includes the changes
that might have happened while a delayed session refresh was in the
delayed queue.
* Added is_media_state_valid() that checks the consistency of
a media state and returns a true/false value. A valid state has:
* The same number of stream entries as media session entries.
Some media session entries can be NULL however.
* No duplicate streams.
* A valid stream for each non-NULL media session.
* A stream that matches each media session's stream_num
and media type.
* Updated handle_incoming_sdp() to set the stream name to include the
stream position number in the name to make it unique.
* Updated the ast_sip_session_delayed_request structure to include both
the pending and active media states and updated the associated delay
functions to process them.
* Updated sip_session_refresh() to accept both the pending and active
media states that were in effect when the request was originally queued
and to pass them on should the request need to be delayed again.
* Updated sip_session_refresh() to call resolve_refresh_media_states()
and substitute its results for the pending state passed in.
* Updated sip_session_refresh() with additional debugging.
* Updated session_reinvite_on_rx_request() to simply return PJ_FALSE
to pjproject if a transaction is in progress. This stops us from
creating a partial pending media state that would be invalid later on.
* Updated reschedule_reinvite() to clone both the current pending and
active media states and pass them to delay_request() so the resolver
can tell what the original intention of the re-invite was.
* Added a large unit test for the resolver.
ASTERISK-29014
Change-Id: Id3440972943c611a15f652c6c569fa0e4536bfcb
The SCOPE_ENTER and SCOPE_EXIT* macros now print debug messages
at the same level as the scope level. This allows the same
messages to be printed to the debug log when AST_DEVMODE
isn't enabled.
Also added a few variants of the SCOPE_EXIT macros that will
also call ast_log instead of ast_debug to make it easier to
use scope tracing and still print error messages.
Change-Id: I7fe55f7ec28069919a0fc0b11a82235ce904cc21
* Added ast_stream_to_stra and ast_stream_topology_to_stra() macros
which are shortcuts for
ast_str_tmp(256, ast_stream_to_str(stream, &STR_TMP))
* Added the stream position to the string representation of the
stream.
* Fixed some formatting in ast_stream_to_str().
Change-Id: Idaf4cb0affa46d4dce58a73a111f35435331cc4b
Allow passing a topology from the called channel back to the
calling channel.
* Added a new function ast_queue_answer() that accepts a stream
topology and queues an ANSWER CONTROL frame with it as the
data. This allows the called channel to indicate its resolved
topology.
* Added a new virtual function to the channel tech structure
answer_with_stream_topology() that allows the calling channel
to receive the called channel's topology. Added
ast_raw_answer_with_stream_topology() that invokes that virtual
function.
* Modified app_dial.c and features.c to grab the topology from the
ANSWER frame queued by the answering channel and send it to
the calling channel with ast_raw_answer_with_stream_topology().
* Modified frame.c to automatically cleanup the reference
to the topology on ANSWER frames.
Added a few debugging messages to stream.c.
Change-Id: I0115d2ed68d6bae0f87e85abcf16c771bdaf992c
The Streams API becomes the home for the core ACN capabilities.
These include...
* Parsing and formatting of codec negotation preferences.
* Resolving pending streams and topologies with those configured
using configured preferences.
* Utility functions for creating string representations of
streams, topologies, and negotiation preferences.
For codec negotiation preferences:
* Added ast_stream_codec_prefs_parse() which takes a string
representation of codec negotiation preferences, which
may come from a pjsip endpoint for example, and populates
a ast_stream_codec_negotiation_prefs structure.
* Added ast_stream_codec_prefs_to_str() which does the reverse.
* Added many functions to parse individual parameter name
and value strings to their respectrive enum values, and the
reverse.
For streams:
* Added ast_stream_create_resolved() which takes a "live" stream
and resolves it with a configured stream and the negotiation
preferences to create a new stream.
* Added ast_stream_to_str() which create a string representation
of a stream suitable for debug or display purposes.
For topology:
* Added ast_stream_topology_create_resolved() which takes a "live"
topology and resolves it, stream by stream, with a configured
topology stream and the negotiation preferences to create a new
topology.
* Added ast_stream_topology_to_str() which create a string
representation of a topology suitable for debug or display
purposes.
* Renamed ast_format_caps_from_topology() to
ast_stream_topology_get_formats() to be more consistent with
the existing ast_stream_get_formats().
Additional changes:
* A new function ast_format_cap_append_names() appends the results
to the ast_str buffer instead of replacing buffer contents.
Change-Id: I2df77dedd0c72c52deb6e329effe057a8e06cd56
ast_stream_topology_create_from_format_cap() was setting the
stream->formats directly but not freeing the default formats. This
causes a memory leak.
* ast_stream_topology_create_from_format_cap() now calls
ast_stream_set_formats() which properly cleans up the existing
stream formats.
When cloning a stream, the source stream's format caps _pointer_ is
copied to the new stream and it's reference count bumped. If
either stream is set to "removed", this will cause _both_ streams
to have their format caps cleared.
* ast_stream_clone() now creates a new format caps object and copies
the formats from the source stream instead of just copying the
pointer.
ASTERISK-28870
Change-Id: If697d81c3658eb7baeea6dab413b13423938fb53
Some places in Asterisk did not treat the formats on a stream
as immutable when they are.
The ast_stream_get_formats function is now const to enforce this
and parts of Asterisk have been updated to take this into account.
Some violations of this were also fixed along the way.
An additional minor tweak is that streams are now allocated with
an empty format capabilities structure removing the need in various
places to check that one is present on the stream.
ASTERISK-28846
Change-Id: I32f29715330db4ff48edd6f1f359090458a9bfbe
Based on this new endpoint setting, a joint list of preferred codecs
between those received from the Asterisk core (remote), and those
specified in the endpoint's "allow" parameter (local) is created and
is used to create the outgoing SDP offer.
* Add outgoing_call_offer_pref to pjsip_configuration (endpoint)
* Add "call_direction" to res_pjsip_session.
* Update pjsip_session_caps.c to make the functions more generic
so they could be used for both incoming and outgoing.
* Update ast_sip_session_create_outgoing to create the
pending_media_state->topology with the results of
ast_sip_session_create_joint_call_stream().
* The endpoint "preferred_codec_only" option now automatically sets
AST_SIP_CALL_CODEC_PREF_FIRST in incoming_call_offer_pref.
* A helper function ast_stream_get_format_count() was added to
streams to return the current count of formats.
ASTERISK-28777
Change-Id: Id4ec0b4a906c2ae5885bf947f101c59059935437
This change adds support to bridge_softmix to allow the addition
and removal of additional video source streams. When such a change
occurs each participant is renegotiated as needed to reflect the
update. If another video source is added then each participant
gets another source. If a video source is removed then it is
removed from each participant. This functionality allows you to
have both your webcam and screenshare providing video if you
desire, or even more streams. Mapping has been changed to use
the topology index on the source channel as a unique identifier
for outgoing participant streams, this will never change and
provides an easy way to establish the mapping.
The bridge_simple and bridge_native_rtp modules have also been
updated to renegotiate when the stream topology of a party changes
allowing the same behavior to occur as added to bridge_softmix.
If a screen share is added then the opposite party is renegotiated.
If that screen share is removed then the opposite party is
renegotiated again.
Some additional fixes are also included in here. Stream state is
now conveyed in SDP so sendonly/recvonly/inactive streams can
be requested. Removed streams now also remove previous state
from themselves so consumers don't get confused.
ASTERISK-28733
Change-Id: I93f41fb41b85646bef71408111c17ccea30cb0c5
The stream topology has no lock of its own resulting in
another lock protecting it in some way (for example the
channel lock). If multiple channels are being juggled at
the same time this can be problematic. This change makes
the topology a reference counted object instead which
guarantees it will remain valid even without the channel
lock being held.
Change-Id: I4f4d3dd856a033ed55fe218c3a4fab364afedb03
Replaces the never used opaque data array.
Updated stream tests to include get/set metadata and
stream clone with metadata.
Added stream metadata dump to "core show channel"
Change-Id: Id7473aa4b374d7ab53046c20e321037ba9a56863
ast_stream_topology_set_stream had suppressed error codes from
AST_VECTOR_APPEND. The result of AST_VECTOR_APPEND needs to be returned
to the caller so they can take appropriate action on the stream.
Change-Id: I6c0d12755743eadba1357f6153526cc055592856
In WebRTC streams (or media tracks in their world) can be grouped
together using the mslabel. This informs the browser that each
should be synchronized with each other.
This change extends the stream API so this information can
be stored with streams. The PJSIP support has been extended
to use the mslabel to determine grouped streams and store
this association on the streams. Finally when creating the
SDP the group information is used to cause each media stream
to use the same mslabel.
ASTERISK-27379
Change-Id: Id6299aa031efe46254edbdc7973c534d54d641ad
The stream topology (list of streams and order) is now stored with the
configured PJSIP endpoints and used during the negotiation process.
Media negotiation state information has been changed to be stored
in a separate object. Two of these objects exist at any one time
on a session. The active media state information is what was previously
negotiated and the pending media state information is what the
media state will become if negotiation succeeds. Streams and other
state information is stored in this object using the index (or
position) of each individual stream for easy lookup.
The ability for a media type handler to specify a callback for
writing has been added as well as the ability to add file
descriptors with a callback which is invoked when data is available
to be read on them. This allows media logic to live outside of
the chan_pjsip module.
Direct media has been changed so that only the first audio and
video stream are directly connected. In the future once the RTP
engine glue API has been updated to know about streams each individual
stream can be directly connected as appropriate.
Media negotiation itself will currently answer all the provided streams
on an offer within configured limits and on an offer will use the
topology created as a result of the disallow/allow codec lines.
If a stream has been removed or declined we will now mark it as such
within the resulting SDP.
Applications can now also request that the stream topology change.
If we are told to do so we will limit any provided formats to the ones
configured on the endpoint and send a re-invite with the new topology.
Two new configuration options have also been added to PJSIP endpoints:
max_audio_streams: determines the maximum number of audio streams to
offer/accept from an endpoint. Defaults to 1.
max_video_streams: determines the maximum number of video streams to
offer/accept from an endpoint. Defaults to 1.
ASTERISK-27076
Change-Id: I8afd8dd2eb538806a39b887af0abd046266e14c7
The SDP offer/answer model requires an answer to an offer before a new SDP
can be processed. This allows our local SDP creation to be deferred until
we know that we need to create an offer or an answer SDP. Once the local
SDP is created it won't change until the SDP negotiation is restarted.
An offer SDP in an initial SIP INVITE can receive more than one answer
SDP. In this case, we need to merge each answer SDP with our original
offer capabilities to get the currently negotiated capabilities. To
satisfy this requirement means that we cannot update our proposed
capabilities until the negotiations are restarted.
Local topology updates from ast_sdp_state_update_local_topology() are
merged together until the next offer SDP is created. These accumulated
updates are then merged with the current negotiated capabilities to create
the new proposed capabilities that the offer SDP is built.
Local topology updates are merged in several passes to attempt to be smart
about how streams from the system are matched with the previously
negotiated stream slots. To allow for T.38 support when merging, type
matching considers audio and image types to be equivalent. First streams
are matched by stream name and type. Then streams are matched by stream
type only. Any remaining unmatched existing streams are declined. Any
new active streams are either backfilled into pre-merge declined slots or
appended onto the end of the merged topology. Any excess new streams
above the maximum supported number of streams are simply discarded.
Remote topology negotiation merges depend if the topology is an offer or
answer. An offer remote topology negotiation dictates the stream slot
ordering and new streams can be added. A remote offer can do anything to
the previously negotiated streams except reduce the number of stream
slots. An answer remote topology negotiation is limited to what our offer
requested. The answer can only decline streams, pick codecs from the
offered list, or indicate the remote's stream hold state.
I had originally kept the RTP instance if the remote offer SDP changed a
stream type between audio and video since they both use RTP. However, I
later removed this support in favor of simply creating a new RTP instance
since the stream's purpose has to be changing anyway. Any RTP packets
from the old stream type might cause mischief for the bridged peer.
* Added ast_sdp_state_restart_negotiations() to restart the SDP
offer/answer negotiations. We will thus know to create a new local SDP
when it is time to create an offer or answer.
* Removed ast_sdp_state_reset(). Save the current topology before
starting T.38. To recover from T.38 simply update the local topology to
the saved topology and restart the SDP negotiations to get the offer SDP
renegotiating the previous configuration.
* Allow initial topology for ast_sdp_state_alloc() to be NULL so an
initial remote offer SDP can dictate the streams we start with. We can
always update the local topology later if it turns out we need to offer
SDP first because the remote chose to defer sending us a SDP.
* Made the ast_sdp_state_alloc() initial topology limit to max_streams,
limit to configured codecs, handle declined streams, and discard
unsupported types.
* Convert struct ast_sdp to ao2 object. Needed to easily save off a
remote SDP to refer to later for various reasons such as generating
declined m= lines in the local SDP.
* Improve converting remote SDP streams to a topology including stream
state. A stream state of AST_STREAM_STATE_REMOVED indicates the stream is
declined/dead.
* Improve merging streams to take into account the stream state.
* Added query for remote hold state.
* Added maximum streams allowed SDP config option.
* Added ability to create new streams as needed. New streams are created
with configured default audio, video, or image codecs depending on stream
type.
* Added global locally_held state along with a per stream local hold
state. Historically, Asterisk only has a global locally held state
because when the we put the remote on hold we do it for all active
streams.
* Added queries for a rejected offer and current SDP negotiation role.
The rejected query allows the using module to know how to respond to a
failed remote SDP set. Should the using module respond with a 488 Not
Acceptable Here or 500 Internal Error to the offer SDP?
* Moved sdp_state_capabilities.connection_address to ast_sdp_state. There
seems no reason to keep it in the sdp_state_capabilities struct since it
was only used by the ast_sdp_state.proposed_capabilities instance.
* Callbacks are now available to allow the using module some customization
of negotiated streams and to complete setting up streams for use. See the
typedef doxygen for each callback for what is allowable and when they are
called.
* Added topology answerer modify callback.
* Added topology pre and post apply callbacks.
* Added topology offerer modify callback.
* Added topology offerer configure callback.
* Had to rework the unit tests because I changed how SDP topologies are
merged. Replaced several unit tests with new negotiation tests.
Change-Id: If07fe6d79fbdce33968a9401d41d908385043a06
* Made ast_format_cap_from_stream_topology() not include any formats from
declined streams.
* Made ast_stream_topology_get_first_stream_by_type() ignore declined
streams to return the first active stream of the type.
* Updated unit tests to check these changes have the expected effect.
Change-Id: Iabbc6a3e8edf263a25fd3056c3c614407c7897df
This sets up the "plumbing" in bridge_softmix to
be able to accommodate Asterisk asking as an SFU
(selective forwarding unit) for conferences.
The way this works is that whenever a channel enters or leaves a
conference, all participants in the bridge get sent a stream topology
change request. The topologies consist of the channels' original
topology, along with video destination streams corresponding to each
participants' source video streams. So for instance, if Alice, Bob, and
Carol are in the conference, and each supplies one video stream, then
the topologies for each would look like so:
Alice:
Audio,
Source video(Alice),
Destination Video(Bob),
Destination video (Carol)
Bob:
Audio,
Source video(Bob)
Destination Video(Alice),
Destination video (Carol)
Carol:
Audio,
Source video(Carol)
Destination Video(Alice),
Destination video (Bob)
This way, video that arrives from a source video stream can then be
copied out to the destination video streams on the other participants'
channels.
Once the bridge gets told that a topology on a channel has changed, the
bridge constructs a map in order to get the video frames routed to the
proper destination streams. This is done using the bridge channel's
stream_map.
This change is bare-bones with regards to SFU support. Some key features
are missing at this point:
* Stream limits. This commit makes no effort to limit the number of
streams on a specific channel. This means that if there were 50 video
callers in a conference, bridge_softmix will happily send out topology
change requests to every channel in the bridge, requesting 50+
streams.
* Configuration. The plumbing has been added to bridge_softmix, but
there has been nothing added as of yet to app_confbridge to enable SFU
video mode.
* Testing. Some functions included here have unit tests.
However, the functionality as a whole has only been verified by
hand-tracing the code.
* Selectivenss. For a "selective" forwarding unit, this does not
currently have any means of being selective.
* Features. Presumably, someone might wish to only receive video from
specific sources. There are no external-facing functions at the moment
that allow for users to select who they receive video from.
* Efficiency. The current scheme treats all video streams as being
unidirectional. We could be re-using a source video stream as a
desetnation, too. But to simplify things on this first round, I did it
this way.
Change-Id: I7c44a829cc63acf8b596a337b2dc3c13898a6c4d
ast_stream_clone() cannot copy the opaque user data stored on a stream.
We don't know how to clone the data so it isn't copied into the clone.
Change-Id: Ia51321bf38ecbfdcc53787ca77ea5fd2cabdf367
This patch is the first cut at adding stream support to the bridging framework.
Changes were made to the framework that allows mapping of stream topologies to
a bridge's supported media types.
The first channel to enter a bridge initially defines the media types for a
bridge (i.e. a one to one mapping is created between the bridge and the first
channel). Subsequently added channels merge their media types into the bridge's
adding to it when necessary. This allows channels with different sized
topologies to map correctly to each other according to media type. The bridge
drops any frame that does not have a matching index into a given write stream.
For now though, bridge_simple will align its two channels according to size or
first to join. Once both channels join the bridge the one with the most streams
will indicate to the other channel to update its streams to be the same as that
of the other. If both channels have the same number of streams then the first
channel to join is chosen as the stream base.
A topology change source was also added to a channel when a stream toplogy
change request is made. This allows subsystems to know whether or not they
initiated a change request. Thus avoiding potential recursive situations.
ASTERISK-26966 #close
Change-Id: I1eb5987921dd80c3cdcf52accc136393ca2d4163
This change extends the ast_request functionality by adding another
function and callback to create an outgoing channel with a requested
stream topology. Fallback is provided by either converting the
requested stream topology into a format capabilities structure if
the channel driver does not support streams or by converting the
requested format capabilities into a stream topology if the channel
driver does support streams.
The Dial application has also been updated to request an outgoing
channel with the stream topology of the calling channel.
ASTERISK-26959
Change-Id: Ifa9037a672ac21d42dd7125aa09816dc879a70e6
* Added additional fields to ast_sdp_options.
* Re-organized ast_sdp.
* Updated field names to correspond to RFC4566 terminology.
* Created allocs/frees for SDP children.
* Created getters/setters for SDP children where appropriate.
* Added ast_sdp_create_from_state.
* Refactored res_sdp_translator_pjmedia for changes.
Change-Id: Iefbd877af7f5a4d3c74deead1bff8802661b0d48
The "core show channel" CLI command will now output the streams
present on the channel with their details.
ASTERISK-26811
Change-Id: I9c95b57aa09415005f0677a1949a0feb07e4987a
This change adds unit tests for the various API calls relating
to stream topologies. This includes creation, destruction,
inspection, and manipulation.
Through this a few bugs were uncovered in the implementation:
1. Creating a topology using a format capabilities would fail as
the code considered a return value of 0 from the append stream
function to indicate an error which is incorrect.
2. Not all functions which placed a stream into a topology
set the position on the stream itself.
3. Appending a stream would cause a frack if the position
provided was the last one. This occurred because the existing
stream was queried but the index was outside of what the
vector was currently at for size.
ASTERISK-26786
Change-Id: Id5590e87c8a605deea1a89e53169a9c011d66fa0
This change adds the media stream topology definition and API for
accessing and using it.
Some refactoring of the stream was also done.
ASTERISK-26786
Change-Id: Ic930232d24d5ad66dcabc14e9b359e0ff8e7f568
This change adds the media stream definition and API for
accessing and using it. Unit tests have also been written
which exercise aspects of the API.
ASTERISK-26773
Change-Id: I3dbe54065b55aaa51f467e1a3bafd67fb48cac87
George Joseph
Joshua C. Colp
Joshua Colp
Corey Farrell
Mark Michelson
Richard Mudgett
Kevin Harwell