conference-protocol.md (4248B)
1 # Conference protocol 2 3 This document aims to describe evolutions we will do for managing conferences (audio/video). The goal is to improve the current implementation which simply merge SIP calls together and provide a grid view, to a view where participants are listed, can be muted independantly, or the video layout changed (to show only one participant) 4 5 ## Definitions 6 7 + Host: Is the user which mix the audio/video streams for the others 8 + Participant: Every user in the conference, even the host 9 10 ## Disclaimer 11 12 This document only describes the first steps for now. This means the identification of participants and position in the video mixer sent to all participants. 13 14 ## Improve on layouts 15 16 Actually, Jami only provides the possibility to show a grid view of the users. We want to be able to only show one member of the conference (the one which shares its screen for example) 17 18 ### Possible layouts 19 20 + GRID: Every member is shown with the same height/width 21 + ONE_BIG_WITH_SMALL: One member is zoomed and the other preview are shown 22 + ONE_BIG: One member take the full screen rendered 23 24 ### New API 25 26 Two new methods are available to manage the conference Layout in CallManager: 27 28 ```cpp 29 /** 30 * Change the conference layout 31 * @param confId 32 * @param layout 0 = matrix, 1 = one big, others in small, 2 = one in big 33 */ 34 void setConferenceLayout(const std::string& confId, int layout); 35 36 /** 37 * Change the active participant (used in layout != matrix) 38 * @param confId 39 * @param participantId If participantId not found, the local video will be shown 40 */ 41 void setActiveParticipant(const std::string& confId, const std::string& participantId); 42 ``` 43 44 ### Implementation 45 46 The implementation is pretty straight forward. Everything is managed by `conference.cpp` (to link participant to sources) and `video_mixer.cpp` (to render the wanted layout). 47 48 49 ## Syncing Conferences Informations 50 51 Note: Actually, the word participant is used for callId mixed in a conference. This can lead at first to some problems for the API and must be fixed in the furture 52 53 The goal is to notify all participants the metadatas of the rendered video. This means what participant is in the conference and where the video is located. 54 55 ### Layout Info 56 57 The Layout is stored as a VectorMapStringString for clients and internally with a vector<ParticipantInfo> with the following format: 58 59 ``` 60 Layout = { 61 { 62 "uri": "participant", "x":"0", "y":"0", "w": "0", "h": "0", "isModerator": "true" 63 }, 64 { 65 "uri": "participant1", "x":"0", "y":"0", "w": "0", "h": "0", "isModerator": "false" 66 } 67 (...) 68 } 69 ``` 70 71 ### New API 72 73 A new method (in CallManager) and a new signal to respectively get current conference infos and updates are available: 74 75 ```cpp 76 VectorMapStringString getConferenceInfos(const std::string& confId); 77 78 void onConferenceInfosUpdated(const std::string& confId, const VectorMapStringString& infos); 79 ``` 80 81 ### Implementation 82 83 The `Conference` Object (which only exists if we mix calls, this means that we are the master) manages the informations for the whole conference, based on the LayoutInfos of each `Call` objects. The getConferenceInfos will retrieve infos directly from this object. 84 85 So, every `Call` object now have a LayoutInfo and if updated, ask the `Conference` object to updates its infos. 86 87 The master of a conference sends its infos via the SIP channel as a message with the following MIME type: 88 `application/confInfo+json` 89 90 So, if a call receives some confInfo, we know that this call is a member of a conference. 91 92 To summarize, `Call` manages received layouts, `Conference` managed sent layouts. 93 94 ## Moderators 95 96 A conference can be controlled (for now, only the layout an be controlled). 97 A moderator is added if on the same device of the host for now. 98 99 ### Implementation 100 101 To change a layout, the moderator can send a payload with "application/confOrder+json" as type: 102 103 ``` 104 { 105 "layout": int 106 } 107 ``` 108 109 To set a participant as active, the moderator can send a payload with "application/confOrder+json" as type: 110 111 ``` 112 { 113 "activeParticipant": "uri" 114 } 115 ``` 116 117 ## Future 118 119 + Control moderators on a conference 120 + Moderators can hangup 121 + Moderators can mute/unmute 122 + Multiple master management 123 + Separate streams to allow more controls?