commit 62ae198757f1928fee6022dd127c11b27ba8ba83 Author: Nick Econopouly <wry@mm.st> Date: Fri, 19 Feb 2021 14:38:38 -0500 Initial commit Diffstat:
| A | building-jami/android.md | | | 144 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | building-jami/build-method.md | | | 6 | ++++++ |
| A | building-jami/build.md | | | 192 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | building-jami/build_method_flowchart.png | | | 0 | |
| A | building-jami/build_method_flowchart.svg | | | 357 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | building-jami/building-individually.md | | | 14 | ++++++++++++++ |
| A | building-jami/index.rst | | | 37 | +++++++++++++++++++++++++++++++++++++ |
| A | building-jami/ios.md | | | 24 | ++++++++++++++++++++++++ |
| A | building-jami/linux.md | | | 60 | ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | building-jami/macos.md | | | 4 | ++++ |
| A | building-jami/master-repository.md | | | 23 | +++++++++++++++++++++++ |
| A | building-jami/windows.md | | | 4 | ++++ |
| A | conf.py | | | 67 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | extra/Group-chat-feature-(design-draft).md | | | 2 | ++ |
| A | extra/Jami-distributed-network.md | | | 118 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | extra/gsoc.md | | | 152 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | extra/index.rst | | | 14 | ++++++++++++++ |
| A | general/all-features-by-client.rst | | | 126 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | general/distributed-network-topo.png | | | 0 | |
| A | general/faq.rst | | | 617 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | general/feature-requests.md | | | 49 | +++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | general/index.rst | | | 16 | ++++++++++++++++ |
| A | general/introduction.md | | | 39 | +++++++++++++++++++++++++++++++++++++++ |
| A | general/old-understanding.md | | | 196 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | general/technical-overview.txt | | | 248 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | general/troubleshooting.rst | | | 38 | ++++++++++++++++++++++++++++++++++++++ |
| A | guides/how-to-contribute-to-this-documentation.txt | | | 137 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | guides/how-to-make-a-bug-report.md | | | 229 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | guides/how-to-set-up-a-dhtproxy-or-bootstrap-server.md | | | 4 | ++++ |
| A | guides/how-to-set-up-a-turn-server.txt | | | 75 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | guides/how-to-submit-a-patch.md | | | 63 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | guides/index.rst | | | 15 | +++++++++++++++ |
| A | index.rst | | | 33 | +++++++++++++++++++++++++++++++++ |
| A | technical/APIs.md | | | 281 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/Choosing-CRF-value-for-encoder.md | | | 104 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/HDMI-CEC-(fr).md | | | 26 | ++++++++++++++++++++++++++ |
| A | technical/LRC-documentation.md | | | 79 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/advanced-features/create-a-plugin.md | | | 484 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/advanced-features/index.rst | | | 10 | ++++++++++ |
| A | technical/advanced-features/plugins.md | | | 297 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/advanced-features/tensorflow-plugin.md | | | 104 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/basic-features/file-transfer.md | | | 200 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/basic-features/index.rst | | | 12 | ++++++++++++ |
| A | technical/basic-features/let's-do-a-call.md | | | 77 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/basic-features/manage-accounts.md | | | 138 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/basic-features/manage-contacts.md | | | 215 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/basic-features/swarm.md | | | 492 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/certificates.rst | | | 63 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/chatview-i18n-(design-draft).md | | | 116 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/conference-protocol.md | | | 124 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/connection-manager.md | | | 67 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/guidelines/Banned-Contact.md | | | 49 | +++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/guidelines/Coding-rules.md | | | 9 | +++++++++ |
| A | technical/guidelines/Kde-guidelines.md | | | 100 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/guidelines/Libringclient-Coding-Rules.md | | | 103 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/guidelines/Qt-and-QML-coding-guidelines.md | | | 112 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/guidelines/Qt-and-QML-testing-tools.md | | | 59 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/guidelines/UI-and-UX-development.md | | | 24 | ++++++++++++++++++++++++ |
| A | technical/identifiers.md | | | 61 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/important-RFCs.md | | | 43 | +++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/improving-jami's-quality.md | | | 85 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/index.rst | | | 32 | ++++++++++++++++++++++++++++++++ |
| A | technical/name-server-protocol.md | | | 210 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/protocol.md | | | 501 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/release-process.md | | | 185 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/sync-protocol.md | | | 94 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
| A | technical/working-with-gerrit.md | | | 96 | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ |
67 files changed, 7755 insertions(+), 0 deletions(-)
diff --git a/building-jami/android.md b/building-jami/android.md @@ -0,0 +1,144 @@ +# Building Jami on Android + ++ Please make sure Android SDK and NDK are installed, and their paths +are properly set. If you need more information, please visit +<https://github.com/savoirfairelinux/ring-client-android> ++ These instructions are for Ubuntu, check equivalent if you use a different distribution + +## Prepare environment + +- Install Java JDK 7 or 8 (Oracle or OpenJDK) + +<http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html> + +```bash +sudo apt install openjdk +``` + +<!-- --> + +- Install Android Studio: + <https://developer.android.com/studio/index.html> + + You can install all dependecies like ndk (side by side), lldb with android-studio in settings > appearance & behavior > system settings > android sdk > sdk tools + +<!-- --> + +- Install the Android NDK: + <https://developer.android.com/ndk/downloads/index.html> + +<!-- --> + +- Install required tools + +<!-- --> + +```bash +sudo apt install autoconf automake libtool autopoint swig python +``` + +Add these variables to your bash profile: + +```bash +export JAVA_HOME=<path_to_java_JDK> +export ANDROID_HOME=<path_to_root_folder_sdk> +export ANDROID_SDK=$ANDROID_HOME +export ANDROID_NDK=<path_to_root_folder_ndk> +export ANDROID_NDK_ROOT=$ANDROID_NDK +export PATH=$PATH:$ANDROID_HOME/tools:$ANDROID_HOME/platform-tools:$ANDROID_NDK:$JAVA_HOME/bin +``` + +ie for Ubuntu : + +```bash +export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre/ +export ANDROID_HOME=/home/{username}/Android/Sdk +export ANDROID_SDK=/home/{username}/Android/Sdk +export ANDROID_NDK_ROOT=/home/{username}/Android/Sdk/ndk-bundle/ +export ANDROID_NDK=/home/{username}/Android/Sdk/ndk-bundle/ +export PATH=$PATH:$ANDROID_HOME/tools:$ANDROID_HOME/platform-tools:$ANDROID_NDK:$JAVA_HOME/bin +``` + +## Build and install locally + +- Clone whole project repository + +<!-- --> + +```bash +git clone --recurse-submodules https://review.jami.net/ring-project +``` + +- Initialize project + +<!-- --> +```bash +cd ring-project +./make-ring.py --init --distribution=Android +``` + + +- Compile + +<!-- --> + +```bash +ANDROID_ABI="armeabi-v7a arm64-v8a" ./make-ring.py --install --distribution=Android +``` + +**Output**: You can find the *.apk* file in the +*./client-android/ring-android/app/build/outputs* folder. + +**Errors** +- ```configure: error: source directory already configured; run "make distclean" there first``` + + It means you build for another distro like x86_64-pc-linux-gnu and builds are conflicting. The simpliest solution is to make another ring-project only for Android to avoid this conflict. + +## Troubleshooting + +- Check case in your sdk path. Since api 26, + /home/user/Android/**s**dk have become /home/user/Android/**S**dk + +<!-- --> + +- You can check each path with *echo*. e.g : echo $JAVA\_HOME + +<!-- --> + +- `sudo` will use root's bash profile. And you don't need it. + +<!-- --> + +- If build fails, you can try to clean contribs with: + +```bash +cd ring-project/ && git clean -xdf +cd ../daemon && git clean -xdf +``` + +- Makeinfo issue + makeinfo: command not found + WARNING: 'makeinfo' is missing on your system. + **Solution**: Install texinfo package containing makeinfo dep. + +<!-- --> + +- Unable to locate tools.jar + **Solution**: Your java installation is not pointing to a JDK. + Install one, or make JAVA_HOME point to it. + +<!-- --> + +- When building the apk error in build-tools + error while loading shared libraries: libstdc++.so.6 + **Solution**: Install lib32stdc++6 lib32z1-dev + +<!-- --> + +- When compiling on Fedora + error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory + **Solution***: sudo dnf install ncurses-compat-libs + +<!-- --> + +- When building, you may get a Gradle error. You should install Gradle 5.4.1 by running apt-get install or installing directly on android-studio. diff --git a/building-jami/build-method.md b/building-jami/build-method.md @@ -0,0 +1,6 @@ +# Which Build Method to Use + +The following flow chart should help you decide whether to build from +the [master repository](https://git.jami.net/savoirfairelinux/ring-project) or not: + + diff --git a/building-jami/build.md b/building-jami/build.md @@ -0,0 +1,192 @@ +2\. **LibRingClient** (or LRC) is written in +QtCore and facilitates clients' portability between operating systems. +It does not interact with the Android client. + +**Building gnome client without installing LibRing and LibRingClient** + +This allows you to build every component separately and to run them from +their location without having to install them. To do this, you must +first build LibRing and then tell LibRingClient where it is located. +Then, you build the LibRingClient and tell client-gnome where +LibRingClient it is located. Finally, you build client-gnome. + +1. Build the daemon as explained above. +2. Configure the build of LibRingClient by specifying the location of + the (non-installed) daemon with e.g.: + cd build && cmake .. -DRING_BUILD_DIR=$HOME/ring-project/daemon/src -DCMAKE_BUILD_TYPE=Debug + +3. Build LibRingClient by running "make". +4. Configure the build of client-gnome by specifying the location of + LibRingClient using the specific variable designed for it: + cd build && cmake .. -DLibRingClient_PROJECT_DIR=$HOME/ring-project/lrc -DCMAKE_BUILD_TYPE=Debug + +5. Build client-gnome by running "make". + +To run an install-less Jami, you must manually start the daemon and then +the client, e.g.: + + term1: $HOME/ring-project/daemon/bin/dring -cdp + term2: $HOME/ring-project/client-gnome/jami-gnome --debug + +Client Qt for Jami +-------------------------------------------------- + +## On GNU/Linux + ++ LibRing and LibRingClient must be installed first. If you have not +already done so, go to the [How to Build LibRing (or +Daemon)](#how-to-build-libring-or-daemon) and [How to Build +LibRingClient (or LRC)](#how-to-build-libringclient-or-lrc) sections. + ++ **Qt 5.14**: In order to use the client Qt it is necessary to have the +Qt version 5.14 or higher. If your system does not have it you can install it +[from sources or download the binary installer](https://www.qt.io/download). +To compile the LibRingClient with a locally installed Qt you need to specify +its path, for example: +```sh +cmake -DQT5_VER=5.15.0 -DQT5_PATH=/home/<username>/Qt/5.15.0/gcc_64 -DRING_BUILD_DIR=<daemon-source-path> -DCMAKE_INSTALL_PREFIX=<lrc-installation-path> -DRING_XML_INTERFACES_DIR=<daemon-source-path>/bin/dbus .. +``` + +#### Installing Build Dependencies + +- For Debian based: +```bash +qtmultimedia5-dev libqt5svg5-dev qtwebengine5-dev qtdeclarative5-dev qtquickcontrols2-5-dev qml-module-qtquick2 qml-module-qtquick-controls qml-module-qtquick-controls2 qml-module-qtquick-dialogs qml-module-qtquick-layouts qml-module-qtquick-privatewidgets qml-module-qtquick-shapes qml-module-qtquick-window2 qml-module-qtquick-templates2 qml-module-qt-labs-platform qml-module-qtwebengine qml-module-qtwebchannel libqrencode-dev libnm-dev +``` +- For Fedora: +```bash +sudo dnf install qt5-qtsvg-devel qt5-qtwebengine-devel qt5-qtmultimedia-devel qt5-qtdeclarative-devel qt5-qtquickcontrols2-devel qt5-qtquickcontrols qrencode-devel NetworkManager-libnm-devel +``` + +#### Build Instructions +Once LibRingClient is built you can compile the client: + +```sh +cd client-qt +mkdir build +cd build +cmake .. -DQT5_VER=5.15.0 -DQT5_PATH=/home/<username>/Qt/5.15.0/gcc_64 -DLRC=<path_to_lrc> -DCMAKE_INSTALL_PREFIX=<installation_path> +make +``` +Variables `QT5_VER` and `QT5_PATH` are used to specify version and path for a custom installation of Qt. + +If lrc library is installed in a custom directory you can set its path with the variable LRC. Additionally you can specify built library location with `LRCLIB` (otherwise it will seach inside LRC with the suffixes `/lib`, `/build` and `/build-local`). + +Then, you are finally ready to launch jami-qt in your build directory. + +If you want to install it to the path provided by `CMAKE_INSTALL_PREFIX` you can run: + +```sh +make install +``` + +#### Debugging + +Compile the client and LibRingClient with `-DCMAKE_BUILD_TYPE=Debug` + +#### Known linker issues + + With Ubuntu 20.04, even if the build process finish with success, the linker might give you the following message: + +```sh +/usr/bin/ld: /home/<username>/Qt/5.15.0/gcc_64/lib/libQt5WebEngineCore.so: .dynsym local symbol at index 3 (>= sh_info of 3) +(...) +``` +This has been solved by switching to [gold linker](https://forum.qt.io/topic/109387/gammaray-build-error-on-ubuntu): + +```sh +sudo ln -sf /usr/bin/x86_64-linux-gnu-ld.gold /usr/bin/ld +``` + +## On native Windows + +- Make sure that daemon, lrc are built first + +```bash + cd client-windows + python make-client.py -d + python make-client.py -b + powershell -ExecutionPolicy Unrestricted -File copy-runtime-files.ps1 +``` +- ```--toolset```, ```--sdk``` options are available, as well. +- To control the version of qmake.exe, ```--qtver``` option can be used + +#### Packaging on native Windows + +- To be able to generate a msi package, first download and install [Wixtoolset](https://wixtoolset.org/releases/). +- In Visual Studio, download WiX Toolset Visual Studio Extension. +- Build client-windows project first, then the JamiInstaller project, msi package should be stored in ring-project\client-windows\JamiInstaller\bin\Release + +Mac OS X Client for Jami +--------------------------------------------------- + ++ LibRing and LibRingClient must be installed first. If you have not +already done so, go to the [How to Build LibRing (or +Daemon)](#how-to-build-libring-or-daemon) and [How to Build +LibRingClient (or LRC)](#how-to-build-libringclient-or-lrc) sections. + + +#### Other Requirements + +- Qt5 (we link against Qt5Core, Qt5Widgets, Qt5Gui) +- Cocoa framework and Xcode toolchains + +#### Getting the Source Code + +```bash +git clone https://review.jami.net/ring-client-macosx +``` + +#### Build Instructions + +```bash +mkdir build && cd build +export CMAKE_PREFIX_PATH=<dir_to_qt5> +``` + +##### Setting up Compilation with XCode + +Generate an Xcode project with CMake: + +```bash +cmake ../ -DCMAKE_INSTALL_PREFIX=<libringclient_install_path> -G Xcode +open Ring.xcodeproj/ +``` + + +Build and run it from Xcode. + +##### Setting up Compilation by Command Line + +```bash +cmake ../ -DCMAKE_INSTALL_PREFIX=<libringclient_install_path> +make +open Ring.app/ +``` + + ++ The app built using +'make' contains only links to required libraries. To fully build and +package as a standalone Bundle, see the **Packaging** section. + +##### Debugging + +For now, the build type of the client is "Debug" by default. However it +is useful to also have the debug symbols of LibRingClient. To do this, +specify `-DCMAKE_BUILD_TYPE=Debug` when compiling LibRingClient in the +cmake options. + +##### Packaging + +To make a standalone Bundle we use a cmake module: +[BundleUtilities](https://cmake.org/cmake/help/v3.0/module/BundleUtilities.html). +All dependencies are copied inside the Bundle and links are fixed. + +We can then generate a "DragNDrop" dmg file with +[CPack](https://cmake.org/Wiki/CMake:Packaging_With_CPack). + +```bash +cmake ../ -DCMAKE_INSTALL_PREFIX=<libringclient_install_path> +make install -j +cpack -G DragNDrop Ring +``` diff --git a/building-jami/build_method_flowchart.png b/building-jami/build_method_flowchart.png Binary files differ. diff --git a/building-jami/build_method_flowchart.svg b/building-jami/build_method_flowchart.svg @@ -0,0 +1,357 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<svg + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="210mm" + height="297mm" + viewBox="0 0 210 297" + version="1.1" + id="svg8" + inkscape:version="1.0.1 (3bc2e813f5, 2020-09-07, custom)" + sodipodi:docname="build_method_flowchart.svg"> + <defs + id="defs2"> + <linearGradient + id="d" + x1="98" + x2="98" + y1="98" + y2="195.5" + gradientUnits="userSpaceOnUse"> + <stop + stop-color="#fff" + offset="0" + id="stop1106" /> + <stop + stop-color="#E9E5E5" + offset="1" + id="stop1108" /> + </linearGradient> + <linearGradient + id="c" + x1="104.61061" + x2="104.61061" + y2="37.039019" + gradientTransform="scale(0.29208988,3.4236038)" + y1="15.961631" + gradientUnits="userSpaceOnUse"> + <stop + stop-color="#0339F1" + offset="0" + id="stop1111" /> + <stop + stop-color="#01ADE5" + offset="1" + id="stop1113" /> + </linearGradient> + <linearGradient + id="b" + x1="54.804602" + x2="101.43272" + y1="58.307517" + y2="109.79727" + gradientTransform="scale(0.93053152,1.0746546)" + gradientUnits="userSpaceOnUse"> + <stop + stop-color="#0756EE" + offset="0" + id="stop1116" /> + <stop + stop-color="#00A3F5" + offset=".37448" + id="stop1118" /> + <stop + stop-color="#18B5B6" + offset=".71708" + id="stop1120" /> + <stop + stop-color="#80DF43" + offset="1" + id="stop1122" /> + </linearGradient> + <linearGradient + id="a" + x1="149.64282" + x2="201.0253" + y1="47.501322" + y2="108.70309" + gradientTransform="scale(0.82995643,1.2048825)" + gradientUnits="userSpaceOnUse"> + <stop + stop-color="#20BBA6" + offset="0" + id="stop1125" /> + <stop + stop-color="#9BEC23" + offset=".4988" + id="stop1127" /> + <stop + stop-color="#D9EF35" + offset="1" + id="stop1129" /> + </linearGradient> + <style + id="style1261">.a{fill:#3ddc84;}</style> + </defs> + <sodipodi:namedview + id="base" + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1.0" + inkscape:pageopacity="0.0" + inkscape:pageshadow="2" + inkscape:zoom="4.7813953" + inkscape:cx="671.14859" + inkscape:cy="218.51491" + inkscape:document-units="mm" + inkscape:current-layer="layer1" + inkscape:document-rotation="0" + showgrid="false" + inkscape:window-width="1920" + inkscape:window-height="948" + inkscape:window-x="0" + inkscape:window-y="0" + inkscape:window-maximized="1" /> + <metadata + id="metadata5"> + <rdf:RDF> + <cc:Work + rdf:about=""> + <dc:format>image/svg+xml</dc:format> + <dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> + <dc:title></dc:title> + </cc:Work> + </rdf:RDF> + </metadata> + <g + inkscape:label="Layer 1" + inkscape:groupmode="layer" + id="layer1"> + <text + xml:space="preserve" + style="font-size:8.46667px;font-family:'Times New Roman';-inkscape-font-specification:'Times New Roman, ';opacity:0.9985;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.2;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:0.887308;stop-color:#000000" + x="91.655647" + y="74.477058" + id="text837-3-9"><tspan + sodipodi:role="line" + id="tspan835-5-1" + x="91.655647" + y="74.477058" + style="font-size:8.46667px;fill:#000000;fill-opacity:1;stroke-width:0.2;stroke-miterlimit:4;stroke-dasharray:none">iOS</tspan></text> + <path + d="m 88.811308,74.747456 c -0.35773,0.347169 -0.75242,0.293048 -1.12863,0.129361 -0.39997,-0.166985 -0.76562,-0.177543 -1.18803,0 -0.52603,0.227049 -0.80522,0.161047 -1.12203,-0.129361 -1.78864,-1.841447 -1.52464,-4.64652 0.50822,-4.752123 0.49303,0.0264 0.83822,0.272588 1.12863,0.293047 0.43165,-0.08778 0.84482,-0.339249 1.30683,-0.306246 0.55508,0.04488 0.97023,0.264005 1.24743,0.658034 -1.14183,0.686422 -0.87122,2.191261 0.17755,2.613668 -0.20989,0.551114 -0.47917,1.09563 -0.93063,1.498242 z m -1.84144,-4.771923 c -0.0535,-0.818422 0.60985,-1.491642 1.37283,-1.55764 0.10494,0.943822 -0.85802,1.650044 -1.37283,1.55764 z" + id="path1015-2" + style="stroke-width:0.00660015" /> + <rect + style="opacity:0.9985;fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.8;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:0.887308;stop-color:#000000" + id="rect833" + width="103.12037" + height="24.286444" + x="44.704578" + y="169.61264" + ry="5.8318963" /> + <text + xml:space="preserve" + style="font-size:8.46667px;font-family:'Times New Roman';-inkscape-font-specification:'Times New Roman, ';opacity:0.9985;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.2;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:0.887308;stop-color:#000000" + x="63.574253" + y="183.79192" + id="text837"><tspan + sodipodi:role="line" + id="tspan835" + x="63.574253" + y="183.79192" + style="font-size:8.46667px;fill:#000000;fill-opacity:1;stroke-width:0.2;stroke-miterlimit:4;stroke-dasharray:none">use the master repo</tspan></text> + <rect + style="opacity:0.9985;fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.799999;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:0.887308;stop-color:#000000" + id="rect833-3" + width="137.79369" + height="24.286438" + x="171.94162" + y="168.47379" + ry="5.8318963" /> + <text + xml:space="preserve" + style="font-size:8.46667px;font-family:'Times New Roman';-inkscape-font-specification:'Times New Roman, ';opacity:0.9985;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.2;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:0.887308;stop-color:#000000" + x="181.95418" + y="182.64066" + id="text837-6"><tspan + sodipodi:role="line" + id="tspan835-7" + x="181.95418" + y="182.64066" + style="font-size:8.46667px;fill:#000000;fill-opacity:1;stroke-width:0.2;stroke-miterlimit:4;stroke-dasharray:none">build each component individually</tspan></text> + <rect + style="opacity:0.9985;fill:#4fff65;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.251655;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:0.887308;stop-color:#000000" + id="rect860" + width="58.90266" + height="58.90266" + x="50.184956" + y="-99.792046" + ry="7.3381104" + transform="rotate(45)" /> + <text + xml:space="preserve" + style="font-size:8.46667px;font-family:'Times New Roman';-inkscape-font-specification:'Times New Roman, ';opacity:0.9985;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.2;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:0.887308;stop-color:#000000" + x="75.378746" + y="8.4622536" + id="text864"><tspan + sodipodi:role="line" + id="tspan862" + x="75.378746" + y="8.4622536" + style="fill:#000000;fill-opacity:1;stroke-width:0.2">operating system?</tspan></text> + <rect + style="opacity:0.9985;fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.799999;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:0.887308;stop-color:#000000" + id="rect833-5" + width="48.645531" + height="24.286442" + x="-6.172801" + y="55.844456" + ry="5.8318963" /> + <g + fill-rule="evenodd" + id="g1142" + transform="matrix(0.1378673,0,0,0.1378673,7.3438664,-19.682064)"> + <rect + x="1" + y="1" + width="194" + height="194" + rx="45" + fill="url(#d)" + stroke="#d9d9d9" + id="rect1134" + style="fill:url(#d)" /> + <path + d="M 32.882,137.54 H 28.3187 V 76.561 H 32.882 Z M 30.5354,61.613 c -1.9244,0 -3.5158,-1.5265 -3.5158,-3.4509 0,-1.9812 1.5833,-3.5158 3.5158,-3.5158 1.9812,0 3.5564,1.5265 3.5564,3.5158 0,1.9244 -1.5752,3.4509 -3.5564,3.4509 z" + fill="url(#c)" + id="path1136" + style="fill:url(#c)" /> + <path + d="m 77.205,138.73 c -22.313,0 -36.409,-16.248 -36.409,-42.076 0,-25.699 14.161,-42.011 36.409,-42.011 22.248,0 36.401,16.313 36.401,42.011 0,25.829 -14.096,42.076 -36.401,42.076 z m 0,-79.89 c -19.422,0 -31.821,14.664 -31.821,37.813 0,23.166 12.456,37.887 31.821,37.887 19.422,0 31.821,-14.721 31.821,-37.887 0,-23.157 -12.399,-37.813 -31.821,-37.813 z" + fill="url(#b)" + id="path1138" + style="fill:url(#b)" /> + <path + d="m 146.1,138.73 c -16.751,0 -28.76,-9.4595 -29.442,-22.987 h 4.474 c 0.68206,11.092 11.1,18.854 25.309,18.854 13.868,0 23.547,-7.8761 23.547,-18.513 0,-8.5582 -5.7731,-13.479 -19.471,-16.93 l -9.6787,-2.3791 c -15.111,-3.8569 -21.972,-9.971 -21.972,-20.21 0,-12.74 11.895,-21.915 26.787,-21.915 15.395,0 26.892,9.0616 27.404,21.063 h -4.474 c -0.62522,-9.7924 -10.19,-16.93 -23.044,-16.93 -12.293,0 -22.086,7.3646 -22.086,17.668 0,8.1603 6.0005,12.854 19.13,16.134 l 9.1184,2.3222 c 15.793,3.9056 22.873,9.971 22.873,20.835 0,13.527 -11.376,22.987 -28.476,22.987 z" + fill="url(#a)" + id="path1140" + style="fill:url(#a)" /> + </g> + <g + id="g1173" + transform="translate(2.8161789,1.0302956)"> + <text + xml:space="preserve" + style="font-size:8.46667px;font-family:'Times New Roman';-inkscape-font-specification:'Times New Roman, ';opacity:0.9985;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.2;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:0.887308;stop-color:#000000" + x="12.696874" + y="69.718971" + id="text837-3"><tspan + sodipodi:role="line" + id="tspan835-5" + x="12.696874" + y="69.718971" + style="font-size:8.46667px;fill:#000000;fill-opacity:1;stroke-width:0.2;stroke-miterlimit:4;stroke-dasharray:none">iOS</tspan></text> + <path + d="m 9.8525369,69.98937 c -0.35773,0.347169 -0.75242,0.293048 -1.12863,0.129361 -0.39997,-0.166985 -0.76562,-0.177543 -1.18803,0 -0.52603,0.227049 -0.80522,0.161047 -1.12203,-0.129361 -1.78864,-1.841447 -1.52464,-4.64652 0.50822,-4.752123 0.49303,0.0264 0.83822,0.272588 1.12863,0.293047 0.43165,-0.08778 0.84482,-0.339249 1.30683,-0.306246 0.55508,0.04488 0.9702301,0.264005 1.2474301,0.658034 -1.1418301,0.686422 -0.8712201,2.191261 0.17755,2.613668 -0.20989,0.551114 -0.47917,1.09563 -0.9306301,1.498242 z m -1.84144,-4.771923 c -0.0535,-0.818422 0.60985,-1.491642 1.37283,-1.55764 0.10494,0.943822 -0.85802,1.650044 -1.37283,1.55764 z" + id="path1015" + style="stroke-width:0.00660015" /> + </g> + <rect + style="opacity:0.9985;fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.799999;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:0.887308;stop-color:#000000" + id="rect833-5-6" + width="54.545307" + height="24.286442" + x="69.969788" + y="59.572247" + ry="5.8318963" /> + <g + id="g1287" + transform="translate(3.3476053,1.4689065)"> + <text + xml:space="preserve" + style="font-size:8.46667px;font-family:'Times New Roman';-inkscape-font-specification:'Times New Roman, ';opacity:0.9985;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.2;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:0.887308;stop-color:#000000" + x="86.361671" + y="73.323875" + id="text1207"><tspan + sodipodi:role="line" + id="tspan1205" + x="86.361671" + y="73.323875" + style="stroke-width:0.2">Android</tspan></text> + <path + class="a" + d="m 81.502531,71.764169 a 0.47431654,0.47431654 0 1 1 0.474318,-0.474316 0.47431654,0.47431654 0 0 1 -0.474318,0.474316 m -5.222422,0 a 0.47431654,0.47431654 0 1 1 0.474317,-0.474316 0.47431654,0.47431654 0 0 1 -0.474317,0.474316 m 5.390409,-2.845897 0.948633,-1.635404 a 0.1958038,0.1958038 0 1 0 -0.340914,-0.192692 l -0.958516,1.655166 a 5.7895274,5.7895274 0 0 0 -2.425929,-0.523723 5.8870589,5.8870589 0 0 0 -2.430873,0.518785 l -0.958515,-1.655169 a 0.1958038,0.1958038 0 0 0 -0.340915,0.192692 l 0.943693,1.635404 a 5.5883876,5.5883876 0 0 0 -2.895307,4.46648 h 11.34901 a 5.5598792,5.5598792 0 0 0 -2.890367,-4.461539" + id="path1269" + style="fill:#000000;fill-opacity:1;stroke-width:0.049408" /> + </g> + <text + xml:space="preserve" + style="font-size:8.46667px;font-family:'Times New Roman';-inkscape-font-specification:'Times New Roman, ';opacity:0.9985;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.2;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:0.887308;stop-color:#000000" + x="169.70874" + y="68.81044" + id="text837-3-9-7"><tspan + sodipodi:role="line" + id="tspan835-5-1-0" + x="169.70874" + y="68.81044" + style="font-size:8.46667px;fill:#000000;fill-opacity:1;stroke-width:0.2;stroke-miterlimit:4;stroke-dasharray:none">iOS</tspan></text> + <path + d="m 166.8644,69.080841 c -0.35773,0.347169 -0.75242,0.293048 -1.12863,0.129361 -0.39997,-0.166985 -0.76562,-0.177543 -1.18803,0 -0.52603,0.227049 -0.80522,0.161047 -1.12203,-0.129361 -1.78864,-1.841447 -1.52464,-4.64652 0.50822,-4.752123 0.49303,0.0264 0.83822,0.272588 1.12863,0.293047 0.43165,-0.08778 0.84482,-0.339249 1.30683,-0.306246 0.55508,0.04488 0.97023,0.264005 1.24743,0.658034 -1.14183,0.686422 -0.87122,2.191261 0.17755,2.613668 -0.20989,0.551114 -0.47917,1.09563 -0.93063,1.498242 z m -1.84144,-4.771923 c -0.0535,-0.818422 0.60985,-1.491642 1.37283,-1.55764 0.10494,0.943822 -0.85802,1.650044 -1.37283,1.55764 z" + id="path1015-2-9" + style="stroke-width:0.00660014" /> + <rect + style="opacity:0.9985;fill:#ffffff;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.799999;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:0.887308;stop-color:#000000" + id="rect833-5-6-3" + width="54.545307" + height="24.286442" + x="148.02287" + y="53.905632" + ry="5.8318963" /> + <g + id="g1430" + transform="translate(-3.3875782,1.9324472)"> + <text + xml:space="preserve" + style="font-size:8.46667px;font-family:'Times New Roman';-inkscape-font-specification:'Times New Roman, ';opacity:0.9985;fill:#000000;fill-opacity:1;fill-rule:evenodd;stroke:#000000;stroke-width:0.2;stroke-linejoin:round;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:0.887308;stop-color:#000000" + x="166.99396" + y="66.989616" + id="text1327"><tspan + sodipodi:role="line" + id="tspan1325" + x="166.99396" + y="66.989616" + style="stroke-width:0.2">Windows</tspan></text> + <g + id="g1421" + transform="matrix(0.19671048,0,0,0.19671048,144.84878,52.74351)"> + <path + fill="#f25022" + d="M 69.216352,40.749778 H 85.43531 V 56.968736 H 69.216352 Z" + id="path1389" + style="fill:#000000;fill-opacity:1;stroke-width:0.264583" /> + <path + fill="#7fba00" + d="M 87.128643,40.749778 H 103.3476 V 56.968736 H 87.128643 Z" + id="path1391" + style="fill:#000000;fill-opacity:1;stroke-width:0.264583" /> + <path + fill="#00a4ef" + d="M 69.216352,58.662069 H 85.43531 V 74.881028 H 69.216352 Z" + id="path1393" + style="fill:#000000;fill-opacity:1;stroke-width:0.264583" /> + <path + fill="#ffb900" + d="M 87.128643,58.662069 H 103.3476 V 74.881028 H 87.128643 Z" + id="path1395" + style="fill:#000000;fill-opacity:1;stroke-width:0.264583" /> + </g> + </g> + </g> +</svg> diff --git a/building-jami/building-individually.md b/building-jami/building-individually.md @@ -0,0 +1,14 @@ +# Building Components Individually + +If you are building each component individually, simply ``git clone`` +each repository and follow the instructions in its README. Do it in +the following order: + + * [jami-daemon](https://git.jami.net/savoirfairelinux/ring-daemon) + * [libjamiclient](https://git.jami.net/savoirfairelinux/ring-lrc) (not used on Android) + * A client from <https://git.jami.net/savoirfairelinux>, such as the + [QT](https://git.jami.net/savoirfairelinux/jami-client-qt), + [GNOME](https://git.jami.net/savoirfairelinux/ring-client-gnome), + or + [macOS](https://git.jami.net/savoirfairelinux/ring-client-macosx) + clients. diff --git a/building-jami/index.rst b/building-jami/index.rst @@ -0,0 +1,37 @@ +########## +Building Jami +########## + +A working Jami setup consists of three ingredients: + + * `jami-daemon <https://git.jami.net/savoirfairelinux/ring-daemon>`_, the core program, + * optionally, `libjamiclient + <https://git.jami.net/savoirfairelinux/ring-lrc>`_, a library used + by some of the clients + * A client, such as `the QT client + <https://git.jami.net/savoirfairelinux/jami-client-qt>`_ which + provides the user interface + + +There are two ways to build Jami: + + 1. Build each part individually. They must be built in the order given above. + 2. Build Jami using the `master repository + <https://git.jami.net/savoirfairelinux/ring-project>`_, which + provides a script to build all of the components at once. This + is easier for most people. + +Whichever method you choose, you should follow the build instructions +in the README or INSTALL file of your chosen repository. + +.. toctree:: + :maxdepth: 1 + + build-method + master-repository + building-individually + windows + linux + macos + android + ios diff --git a/building-jami/ios.md b/building-jami/ios.md @@ -0,0 +1,24 @@ +# Building Jami for iOS + +After installing [Brew](https://brew.sh) and Python3 (brew install +python3): + +```bash +git clone https://review.jami.net/ring-project +cd ring-project +./make-ring.py --init +./make-ring.py --dependencies --distribution iOS +./make-ring.py --install --distribution iOS +cd client-ios/Ring && ./fetch-dependencies.sh && cd .. +xcodebuild build -project Ring/Ring.xcodeproj/ -configuration "Release" -arch "x86_64" -destination "platform=iOS Simulator,name=iPhone $DATE,OS=11" -sdk iphonesimulator11.0 VALID_ARCHS="x86_64" + + +``` +it could be useful to do this if an error occurs: +```bash +ln -s /usr/local/opt/gettext/bin/autopoint /usr/local/bin +``` +if you get build errors linked to swift, you should upgrade swiftgen: +```bash +brew upgrade swiftgen +``` diff --git a/building-jami/linux.md b/building-jami/linux.md @@ -0,0 +1,60 @@ +# Building Jami on Linux + +Simply follow the instructions in the README of the [master +repository](https://git.jami.net/savoirfairelinux/ring-project). + +Alternatively, follow the build instructions for [each individual +component](building-individually.html) if you choose to build them +separately. + + +## Dependencies + +Jami-daemon provides some dependencies in its `contrib` directory, +along with instructions for compiling them. However, some dependencies +should be installed with your package manager. + +### Fedora + +```bash +sudo yum groupinstall group "Development Tools" "Development Libraries" +sudo yum install gcc-c++ yasm intltool libyaml-devel alsa-lib-devel pulseaudio-libs-devel libsamplerate-devel dbus-c++-devel pcre-devel gsm-devel speex-devel speexdsp-devel expat-devel qttools5-dev libsndfile-devel gnutls-devel gettext-devel cmake libtool systemd-devel uuid-devel libXfixes-devel jsoncpp-devel autoconf-archive qt5-qtbase-devel qt5-qttools-devel +``` + +For video support, you'll also need ffmpeg, which is only available in +the RPMfusion repository as described here +<http://rpmfusion.org/Configuration> + +Then install ffmpeg: + +```bash +sudo yum install ffmpeg-devel +``` + +To build and run the tests with make check, you'll also need + +```bash +sudo yum install cppunit-devel cppcheck sipp dbus +``` + +### Debian-based + +Building dependencies/instructions for Debian, Ubuntu, Crunchbang Linux, +etc. + +```bash +sudo apt-get install autoconf autoconf-archive automake autopoint cmake libpulse-dev libsamplerate0-dev libgsm1-dev libspeex-dev libtool libdbus-1-dev libasound2-dev libopus-dev libspeexdsp-dev libexpat1-dev libpcre3-dev libyaml-cpp-dev libboost-dev libdbus-c++-dev qttools5-dev libsndfile1-dev libsrtp-dev libjack-dev libupnp-dev libavcodec-dev libavformat-dev libswscale-dev libavdevice-dev libudev-dev yasm uuid-dev libgnutls28-dev libjsoncpp-dev libvdpau-dev libva-dev qtbase5-dev qttools5-dev and qttools5-dev-tools +``` + +If you want to run tests (e.g. when setting up a new VM on Jenkins), you +must install cppunit and sipp + +```bash +sudo apt-get install libcppunit-dev sip-tester dbus +``` + +For H.264 support, you'll need + +```bash +sudo apt-get install libavcodec-extra-* +``` diff --git a/building-jami/macos.md b/building-jami/macos.md @@ -0,0 +1,4 @@ +# Building Jami on macOS + +Simply follow the instructions in the README of the [master +repository](https://git.jami.net/savoirfairelinux/ring-project#on-osx). diff --git a/building-jami/master-repository.md b/building-jami/master-repository.md @@ -0,0 +1,23 @@ +# Using the Master Repository + +You must first install [Git](https://git-scm.com/). + +Run the following command to download the project: + +```bash +$ git clone https://review.jami.net/ring-project +``` + +Then follow the instructions in the +[README](https://git.jami.net/savoirfairelinux/ring-project). + +## Updating your copy + + +Due to high activity on the other projects, the Git submodules may need to +be manually updated next time you rebuild everything: + +``` +$ cd ring-project +$ git submodule update --init daemon lrc client-gnome +``` diff --git a/building-jami/windows.md b/building-jami/windows.md @@ -0,0 +1,4 @@ +# Building Jami on Windows + +Follow the instructions in the QT client's +[repository](https://git.jami.net/savoirfairelinux/jami-client-qt#building-on-native-windows). diff --git a/conf.py b/conf.py @@ -0,0 +1,67 @@ +# Configuration file for the Sphinx documentation builder. +# +# This file only contains a selection of the most common options. For a full +# list see the documentation: +# https://www.sphinx-doc.org/en/master/usage/configuration.html + +# -- Path setup -------------------------------------------------------------- + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# +# import os +# import sys +# sys.path.insert(0, os.path.abspath('.')) + + +# -- Project information ----------------------------------------------------- + +project = 'Jami' +copyright = '2021, SFL Employees and Contributors' +author = 'SFL Employees and Contributors' + + +# -- General configuration --------------------------------------------------- + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = ['recommonmark'] + +source_suffix = { + '.rst': 'restructuredtext', + '.txt': 'restructuredtext', + '.md': 'markdown', +} + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +# This pattern also affects html_static_path and html_extra_path. +exclude_patterns = [] + + +# -- Options for HTML output ------------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +# +html_theme = 'sphinx_rtd_theme' + +html_theme_options = { + 'navigation_depth': 3, + 'sticky_navigation': False, + 'collapse_navigation': False, +} + +html_sidebars = { '**': ['globaltoc.html', 'relations.html', 'sourcelink.html', 'searchbox.html'] } + +# html_sidebars = { '**': ['globaltoc.html', 'relations.html', 'sourcelink.html', 'searchbox.html'] } + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = ['_static'] diff --git a/extra/Group-chat-feature-(design-draft).md b/extra/Group-chat-feature-(design-draft).md @@ -0,0 +1 @@ +Moved to https://git.jami.net/savoirfairelinux/ring-project/wikis/technical/2.3.-Swarm+ \ No newline at end of file diff --git a/extra/Jami-distributed-network.md b/extra/Jami-distributed-network.md @@ -0,0 +1,118 @@ +# The Jami Network + +### Connectivity + +Jami relies on a [distributed +network](tutorials/Jami-distributed-network), that brings multiple +advantages when compared to federated networks: + +- No point of failure, +- More resilient to censorship, +- Do not depend on anything other than its users, +- Trust amongst nodes is not necessary. + + + +This network forms a Distributed Hash Table (DHT) + +The core problem of distributed communication systems is peer +connectivity, Jami achieves it through two elements: + +- Encrypted announcements on DHT, +- Use of standard protocols for NAT hole punching. + +Jami is built on two distinct distributed networks: + +- the OpenDHT kademlia network to provide distributed connectivity + establishment and message distribution, +- the JamiNS blockchain to provide distributed name registration. + +The OpenDHT network +------------------- + +See +[<https://github.com/savoirfairelinux/opendht>](https://github.com/savoirfairelinux/opendht) +for more information about OpenDHT, which provides a distributed +key-value datastore for connectivity establishment (with ICE) and +message distribution in Jami. + +An OpenDHT network can be joined by knowing about any node already +connected to the network. This node will then share its knowledge about +other nodes on the network. + +Jami clients use a persistent node cache to reconnect to the network +after a first session. A configurable, known, stable "bootstrap" node is +used for the first connection or if cached nodes don't answer. + +Jami clients currently use bootstrap.jami.net:4222 as the default +(configurable) bootstrap node and network ID 0 (the default, public +OpenDHT network). + +### Contribute to the OpenDHT network + +Every Jami account runs an OpenDHT node, contributing to the network and +allowing Jami to scale. + +Jami users can have full independence by running their own stable +OpenDHT node and configure it as a bootstrap node in Jami, while helping +to improve stability, robustness and resilience for every user of the +public OpenDHT network. + +A standalone node can be run using the [dhtnode +utility](https://github.com/savoirfairelinux/opendht/wiki/Running-a-node-with-dhtnode) +included with OpenDHT. dhtnode doesn't persist any data and has a +default in-memory storage limit of 8 MiB. + +Stable community-run DHT nodes will be added to the default bootstrap +list at the request of their owner, as more bootstrap nodes means a more +resilient, independent network. + +The JamiNS blockchain +--------------------- + +The JamiNS blockchain is experimental and its architecture is expected +to evolve. + +Jami clients don't run blockchain nodes themselves but rather +communicate with a JamiNS server using HTTP for name registration and +query, with a REST API. This is because the resources needed to run a +blockchain node are too high for most end-users. + +The nameserver can be configured by-account in Jami, allowing to connect +Jami clients to various more or less centralized user directories. + +### Contribute to the JamiNS blockchain + +The default Jami name service is ns.jami.net, provided by Savoir-faire +Linux, connected to an Ethereum blockchain node; the goal being to give +everyone the possibility (if they which so) to run their own blockchain +node and HTTP service, mine some Ether, and use it to pay the +transaction fee needed to register their username in Jami. + +Code of the Ethereum contract, the blockchain genesis file, and the +NodeJS module (HTTP server) can be found here : +[1](https://github.com/savoirfairelinux/ring-nameservice) + +### Running a Jami Node + +#### Pre-requisites: +1. Geth 1.8.23+ (download from [HERE](https://geth.ethereum.org/downloads/)) +2. Jami genesis file (download from [HERE](https://github.com/savoirfairelinux/ring-nameservice/blob/master/instructions/genesis.json)) + + +#### Joining the Jami Network + +The process of joining the Jami network is similar to the process of joining a regular ethereum network with the difference that the genesis file is used to initialize the data directory. + +1. Download the Jami genesis file from the Jami github repo +2. Create a directory on your computer which you will use to store the Jami blockchain data + * Example for MacOS (/Users/username/jamichain) + * Example for Linux (/home/username/jamichain) + * Example for Windows (C:\Users\username\jamichain) +3. Use geth to initialize the directory you created in (2) by running ```./geth --datadir /home/username/jamichain init genes is.json ``` +4. You can now start geth with the command-line options you need and specifying one of Jami's bootnodes as follows: + +``` +geth --datadir=/home/username/jamichain --syncmode=full --networkid 1551 --bootnodes "enode://11ba6d3bfdc29a8afb24dcfcf9a08c8008005ead62756eadb363523c2ca8b819efbb264053db3d73949f1375bb3f03090f44cacfb88bade38bb6fc2cb3d890a5@173.231.120.228:30301" console +``` +This will start a geth daemon with an attached console which is now syncing with Jami's network. diff --git a/extra/gsoc.md b/extra/gsoc.md @@ -0,0 +1,151 @@ +The Ring project has previously participated in the Google Summer of +Code program on 2016 and 2017 under the umbrella of the [Debian +project](https://www.debian.org/) and the [GNU +project](https://www.gnu.org/), mentoring 7 students during these two +editions. + +This year we are willing to participate as an individual organization. +Thus, we are welcoming every contribution to [Ring's source +code](https://gerrit-ring.savoirfairelinux.com/#/admin/projects/) that +could help to further develop Ring as well as our community. You can +also contribute to Ring's documentation. + +### Project ideas: + +#### A Node.js client for Ring + +#### Ring account provisioning server + +#### Auto video quality: Optimizing the auto video quality algorithm (RTCP/RTSP feedback) + +Likely mentor(s): Olivier, Philippe + + For the moment ring uses a slider to set video stream quality disregarding the connection quality. + The student will explore the RTCP and RTSP protocols implementation in FFmpeg to later implement an automation of the video stream quality setting, though the informations furnished by the control packets. + - Renegotiate SDP depending on network conditions (change codec parameters or codecs, limit data flow, etc) + - Improve algorithm so it can maximize quality without going over a maximum bit rate, or maintain a certain quality regardless of network traffic + - Use other RTCP facilities to improve video quality (Ring only uses packet loss) + +#### Ring IoT + +Likely mentor(s): Anthony + + The ring-client-iot project, started last year by a previous GSoC student, can be used as a base for a headless Ring client. Using it to start a new client, more focused + on little devices without direct human interaction, would enable new applications in the embedded field. From this point, multiple directions can be followed as a GSoC + project : + - Build Ring daemon and client-iot on ARM platforms (possibly including it in the Buildroot ecosystem). + - Add commands to read sensor values, trigger actions on system and eventual peripherals, etc. + - Improve manageability of accounts authorized to send commands to the device. + - Integrate the recent file transfer feature with IoT applications (display any picture received by Ring on a screen, etc.) + +#### Add video surveillance features to Ring + +Likely mentor(s): Maxim, Nicolas + + The goal of this project is to enable remote video surveillance capabilities in Ring which would allow someone to keep an eye on their home, a child, etc. When a sound or movement event is triggered, a notification should be sent and the video should be recorded and stored on both the local and remote device to provide forensic evidence. + + Tasks: + + - Implement/integrate movement/sound detection algorithms in the Ring daemon. + - Leverage the features already present in Ring to implement the message notifications as well as the video streaming and recording. + - Design and implement the UI elements of at least one client that will define how this functionality is enabled and used. + +#### Refactoring User Interfaces + +#### Continue working on Telepathy client + +Likely mentor(s): Olivier + + A previous work was initiated in 2016 for the integration of Ring in Telepathy [https://github.com/alok4nand/telepathy-bell], WIP: + 1. Improving the Account Management, text messaging, and Contact Management: Making the connection manager usable as a Ring client for text messaging for a non tech user + + 2. Exploring video calls: Currently the daemon handles video one frame at a time (not a video stream). If using dbus the actual frames are shared via shm (shared memory), would the video via frames model work with Telepathy/Empathy? + + 3. Implement trust requests (ie: friend requests) + +#### Ring as a WebRTC service + +Likely mentor(s): Guillaume + + For all communications LibRing uses simple system sockets (mainly UDP, and TCP for data transfer). + The idea is to add a WebRTC connectivity to extend Ring and facilitate portage of Ring. + This project needs to add new settings to indicate how and when to use WebRTC sockets and re-design most of libring low-level implementations. + + Example of tasks to be done: + - Identify impacted code. + - Make a WebRTC small client as validation test. + - Implement WebRTC connectivity at low-level. + - Change libring API and settings if needed. + - Modify/Test a currently supported client to validate the full chain. + +#### Peer to peer file transfer + +Likely mentor(s): Guillaume + + Current code base uses a TURN only connection (TURN/TCP/TLS) to manage a peer connectivity for our reliable data transfer feature. + TURN is a relay protocol to solve connectivity issues between peers behind NAT or firewall. The side effects are latency and non-scalability. + + To reduce these effects we want to introduce a non-relay connection, true P2P, usable for hosts on the same network as example. + To make connection transparent to the application (so to the user), the true P2P-way is tried as first stage and the TURN-way is used as fallback. + This change doesn't require to change the libring API. It's only an implementation change. The data-transfer protocol (i.e. data exchanged over DHT at connection request) could be changed. + + Example of tasks to be done: + - Identify impacted code. + - Create a validation test. + - Implement the true P2P in data-transfer code. + - Support the TURN fallback. + - Verify the data-transfer using validation tests created earlier. + +#### Conference server + +#### Matrix integration + +Likely mentor(s): Pierre, Anthony + +In the same spirit as <https://github.com/matrix-org/matrix-appservice-irc> or <https://github.com/jfrederickson/matrix-xmpp-bridge>, the goal of this project is to integrate Ring to Matrix as well as possible. +That way, communication between users of Ring and users of Matrix would be possible. + +#### Redesign of the media system + +Likely mentor(s): Philippe, Andreas + + Ring's media system is massive and needs an overhaul. + The goals of this project are: + - Implement zero-copy video frame buffer manipulations (ex: OpenGL/OpenCL, VDPAU). + - Update the usage of Ring's media APIs and codecs to conform to current standards, + - Provide a cleaner and better design for platform specific implementations and reduce conditionally compiled code clutter. + - Extend the audio API to allow for client implementations on platforms where media device access is restricted to client specific APIs. + +#### Create a clean wrapper for AndroidTV's API. + +Likely mentor(s): Pierre, Adrien + + For now, Ring is the only video chat client available on AndroidTV. This client is more basic than the Android one mainly because it's hard to get good quality and maintainable code with the current state of the AndroidTV framework. + Right now, the Ring AndroidTV client contains a wrapper for AndroidTV's API, but there's still boilerplate code. + The goals of this project are: + - To identify improvements that can be made on this wrapper + - To create an API for Leanback easily usable in the context of Ring (will leverage any knowledge on design patterns and API design) + - To implement the new API in Ring + - To create an external library, independent of Ring and usable in the context of any AndroidTV application. + +------------------------------------------------------------------------ + +### Contributions GSoC 2016: + +1. Improving distributed and secure communication using free software + [(link)](https://summerofcode.withgoogle.com/archive/2016/projects/4886025126019072/) +2. Indexation over a distributed network + [(link)](https://summerofcode.withgoogle.com/archive/2016/projects/6573755878866944/) +3. Ring project + [(link)](https://summerofcode.withgoogle.com/archive/2016/projects/6477112403820544/) +4. Telepathy Connection Manager for Ring protocol + [(link)](https://summerofcode.withgoogle.com/archive/2016/projects/5047255782391808/) + +### Contributions GSoC 2017 + +1. Setting up unit tests for SIP calls in Ring + [(link)](https://summerofcode.withgoogle.com/archive/2017/projects/5836252280520704/) +2. Ring - Create a C++ plugin for Ring + [(link)](https://summerofcode.withgoogle.com/archive/2017/projects/5704614754123776/) +3. Ring: NodeJS Plugin for Seamless Cross-platform Client Development + [(link)](https://summerofcode.withgoogle.com/archive/2017/projects/6532521776906240/)+ \ No newline at end of file diff --git a/extra/index.rst b/extra/index.rst @@ -0,0 +1,14 @@ +########## +Extra +########## + +These are documents that either need to be finished or haven't been +categorized yet: + +.. toctree:: + :maxdepth: 1 + + introduction + faq + All-features-by-client + feature-requests diff --git a/general/all-features-by-client.rst b/general/all-features-by-client.rst @@ -0,0 +1,126 @@ +All Features by Client +====================== + +Legend: + +- ✓ : available +- X : not available yet +- N/A : not applicable for this client + +Text messaging +-------------- + ++------------------------------------+-------+---------+---------+------------+-----+-------+ +| **Client** | Linux | Windows | Android | Android TV | iOS | macOS | ++====================================+=======+=========+=========+============+=====+=======+ +| Text Messaging | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ++------------------------------------+-------+---------+---------+------------+-----+-------+ +| Group chat | X | X | X | X | X | X | ++------------------------------------+-------+---------+---------+------------+-----+-------+ +| Clear History | ✓ | ✓ | ✓ | X | ✓ | ✓ | ++------------------------------------+-------+---------+---------+------------+-----+-------+ +| Chat history shared across devices | X | X | X | X | X | X | ++------------------------------------+-------+---------+---------+------------+-----+-------+ +| Send Files | ✓ | ✓ | ✓ | X | ✓ | ✓ | ++------------------------------------+-------+---------+---------+------------+-----+-------+ +| Auto accept images | ✓ | ✓ | ✓ | X | ✓ | ✓ | ++------------------------------------+-------+---------+---------+------------+-----+-------+ +| Custom download location | ✓ | ✓ | X | X | X | ✓ | ++------------------------------------+-------+---------+---------+------------+-----+-------+ +| Typing indicator | ✓ | ✓ | ✓ | X | ✓ | X | ++------------------------------------+-------+---------+---------+------------+-----+-------+ +| Read status | X | X | ✓ | X | ✓ | X | ++------------------------------------+-------+---------+---------+------------+-----+-------+ + +Calling +------- + +====================== ===== ======= ======= ========== === ===== +**Client** Linux Windows Android Android TV iOS macOS +====================== ===== ======= ======= ========== === ===== +Audio Calling ✓ ✓ ✓ X ✓ ✓ +Video Calling ✓ ✓ ✓ ✓ ✓ ✓ +Host call conference ✓ ✓ ✓ ✓ ✓ ✓ +Auto bitrate ✓ ✓ X X ✓ ✓ +Change video quality ✓ ✓ ✓ X X X +Custom ringtones ✓ ✓ ✓ X X ✓ +Select camera ✓ ✓ ✓ X ✓ ✓ +Video call recording ✓ ✓ X X X ✓ +Leave audio message ✓ ✓ ✓ X ✓ ✓ +Leave video message ✓ ✓ ✓ X ✓ ✓ +Screen sharing ✓(1) ✓ ✓ X X ✓ +media sharing ✓ ✓ X X X ✓ +Hardware encoding \* ✓ ✓ ✓ ✓ ✓ ✓ +Hardware decoding \* ✓ ✓ ✓ ✓ ✓ ✓ +Mute sound ✓ ✓ ✓ X ✓ ✓ +Texting while on call ✓ ✓ ✓ X ✓ ✓ +Remote recording notif ✓ X X X X X +Rendez-vous mode ✓ ✓ X ✓ X ✓ +Conference Layout ✓ ✓ ✓ X ✓ ✓ +====================== ===== ======= ======= ========== === ===== + +\* This enables 4k video calling support + +(1) only on X (not Wayland) + +Account settings +---------------- + +=========================== ===== ======= ======= ========== === ===== +**Client** Linux Windows Android Android TV iOS macOS +=========================== ===== ======= ======= ========== === ===== +Profile avatar ✓ ✓ ✓ ✓ ✓ ✓ +Disable account ✓ ✓ ✓ X ✓ ✓ +Delete account ✓ ✓ ✓ X ✓ ✓ +Contact availability ✓ ✓ ✓ ✓ ✓ ✓ +Register username ✓ ✓ ✓ ✓ ✓ ✓ +Delete Contacts ✓ ✓ ✓ ✓ ✓ ✓ +Block Contacts ✓ ✓ ✓ X ✓ ✓ +Link new device via DHT ✓ ✓ ✓ ✓ ✓ ✓ +Link new device via archive ✓ ✓ X X X ✓ +Auto answer ✓ ✓ ✓ ✓ X ✓ +Custom ringtones ✓ ✓ ✓ X X ✓ +=========================== ===== ======= ======= ========== === ===== + +Other features +-------------- + ++--------------------------------+-------+---------+---------+------------+-----+-------+ +| **Client** | Linux | Windows | Android | Android TV | iOS | macOS | ++================================+=======+=========+=========+============+=====+=======+ +| Scan QR Code | X | X | ✓ | X | ✓ | X | ++--------------------------------+-------+---------+---------+------------+-----+-------+ +| Display QR Code | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ++--------------------------------+-------+---------+---------+------------+-----+-------+ +| Ban contact | ✓ | ✓ | ✓ | X | ✓ | ✓ | ++--------------------------------+-------+---------+---------+------------+-----+-------+ +| System notifications | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ++--------------------------------+-------+---------+---------+------------+-----+-------+ +| Supported languages | 85 | 85 | 85 | 85 | 85 | 85 | ++--------------------------------+-------+---------+---------+------------+-----+-------+ +| Contacts shared across devices | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ++--------------------------------+-------+---------+---------+------------+-----+-------+ +| Multi account | ✓ | ✓ | ✓ | X | ✓ | ✓ | ++--------------------------------+-------+---------+---------+------------+-----+-------+ +| SIP account | ✓ | ✓ | ✓ | X | ✓ | ✓ | ++--------------------------------+-------+---------+---------+------------+-----+-------+ +| SIP transferring | ✓ | X | X | X | X | X | ++--------------------------------+-------+---------+---------+------------+-----+-------+ +| Dark theme support | ✓ | ✓ | ✓ | ✓ | X | ✓ | ++--------------------------------+-------+---------+---------+------------+-----+-------+ +| JAMS support | ✓ | ✓ | ✓ | X | ✓ | ✓ | ++--------------------------------+-------+---------+---------+------------+-----+-------+ +| Plugin support | ✓ | ✓ | ✓ | ✓ | X | X | ++--------------------------------+-------+---------+---------+------------+-----+-------+ + +Advanced settings +----------------- + +================== ===== ======= ======= ========== === ===== +**Client** Linux Windows Android Android TV iOS macOS +================== ===== ======= ======= ========== === ===== +DHT Proxy support ✓ ✓ ✓ X ✓ ✓ +Push notification N/A N/A ✓ N/A ✓ N/A +UPnP ✓ ✓ ✓ ✓ X ✓ +TURN configuration ✓ ✓ ✓ ✓ X ✓ +================== ===== ======= ======= ========== === ===== diff --git a/general/distributed-network-topo.png b/general/distributed-network-topo.png Binary files differ. diff --git a/general/faq.rst b/general/faq.rst @@ -0,0 +1,617 @@ +FAQ +=== + +This is an exhaustive list of frequently asked questions, including +some technical questions. + +.. contents:: + :local: + :depth: 3 + +Basics +------ + +What is Jami? +~~~~~~~~~~~~~ + +Read the :doc:`Introduction <introduction>`. + +What makes Jami different from other communication platforms? +~~~~~~~~~~~~~~~~~~~~~ + +Jami doesn't work like most communication platforms because it is +*distributed*: + +.. image:: distributed-network-topo.png + +Some of the consequences may seem surprising. For instance, since +accounts are stored on your device, passwords are optional. However, +the most significant practical differences are that you have more +*freedom* and *privacy*. + +TODO: expand on this + +What do the red/green status circles next to avatars mean? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +On your own account, a red circle means that you aren't connected to +the DHT. You may need to check your connection or restart the app. + +On other contacts, a red circle means that they are not online, and a +green circle means they are online and you should be able to message +them. + +Note that a green circle only means that the contact has announced +their presence on the DHT. It does not indicate a direct connection to +their device. In some cases, a contact may be able to send and receive +DHT messages but cannot make calls or file transfers because of their +firewall. + + +Why is a feature missing on my client? +~~~~~~~~~~~~~~~~~~~~ + +Not every client implements all features; check the list :doc:`here +<all-features-by-client>` to see if your client is missing the +feature. + +You can make feature requests at +https://git.jami.net/. + +Does Jami support read reciepts? Can I turn them on or off? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can enable or disable read receipts on Android. Other platforms +only have partial or no support. Please see :doc:`All Features by +Client <all-features-by-client>` for the current status. + +Does Jami support typing notifications? Can I turn them on or off? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Most of the client support sending and receiving typing +notifications. You can enable/disable them in the general settings. + +Can I share my screen? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Yes, on all platforms except for iOS. Search for a dedicated "Share +screen" button while you are in a video call. + + +Can I make group conference calls? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Yes. You can add Jami contacts to existing calls (audio or video) by +clicking the "Add participant" button. + +Does Jami have group chats? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Not yet. Group chats are `coming soon +<technical-overview.html#swarms>`_. + + +Why is my contact not seeing my avatar? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Due to technical limitation, avatars are only transfered to your +contacts during a video or audio call. This limitation will disappear +when `group chats <technical-overview.html#swarms>`_ are released. + +Why aren't my sent messages showing up on all linked devices? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +All of your devices receive the same messages from your contacts, but +*sent* messages will not show up on all of your devices. + +The `swarm <technical-overview.html#swarms>`_ update will introduce +full conversation sync between linked devices for all conversations +(including one-on-one conversations). + + + +How can I make a bug report? +~~~~~~~~~~~~~~ + +Please see :doc:`How to Make a Bug Report <../guides/how-to-make-a-bug-report>`. + +Where are the configuration files located? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Jami saves its configuration (account, certificates, history) at +different locations depending the platform. + +- **GNU/Linux**: global configuration is under + **~/.config/jami/dring.yml** and account files are under + **~/.local/share/jami/**. Finally, a cache can be stored in + **~/.cache/jami** + + +- **OSX**: The full configuration is under **~/Library/Application Support/Jami** if installed via https://jami.net. + The app store version uses + **~/Library/Containers/com.savoirfairelinux.ring.macos/Data/Library/Application Support/jami** + +- **Android**: The full configuration is under **/data/data/cx.ring** + (may require root privileges) + +- **Windows**: global configuration is under + **%AppData%/Local/jami/dring.yml** and Account files are under + **%AppData%/Local/jami/**. Finally, a cache is stored in + **%USERPROFILE%/.cache/jami** + +Note: audio and video messages are recorded in the local-data in the +folder: ``sent_data`` + +TODO: check this ^^^ and add note about file downloads (like images) + +How much bandwidth do I need for calls? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For audio calls, Jami uses about 100 Kbps. For a video call, you need +about 2 Mbit/s for medium quality. If your connection is slower, the +bitrate will be automatically reduced. + +If you are hosting a video conference, you will need approximately 2 +Mbps more per participant. For a conference with 10 participants, each +participants will need 2Mbps up & down and the host will need 20Mbps +up and down. + +Auto-adaptation is done between 200Kbit/s / max:6Mbit/s + +TODO: ^^^^^^^^^^^^^ What does this last line mean? + +Account management +------------------ + +What is a Jami account? +~~~~~~~~~~~~~~~~~~~~~~~ + +A Jami account is an `asymmetric encryption key +<https://en.wikipedia.org/wiki/Public-key_cryptography>`_. Your +account is identified by a Jami ID, which is a `fingerprint +<https://en.wikipedia.org/wiki/Public_key_fingerprint>`_ of your +public key. + +What information do I need to provide to create a Jami account? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +When you create a new Jami account, you don’t have to provide private +information like an email, address, or phone number. + +This is the information you can provide if you choose (it's all +optional): + +1. An avatar +2. A display name, which is the name that clients will display for + your contact. It can contain special characters. +3. An optional username, which is a unique identifier that is directly + associated with your JamiID. This username->Jami ID mapping is + stored on a server (ns.jami.net by default, but you can host your + own) +4. A password. This password is used to protect the account archive in + your device. + +For more information about Jami accounts, see `the Technical Overview +<technical-overview.html#what-is-a-jami-account>`_. + +Where is my Jami ID? +~~~~~~~~~~~~~~~~ + +Your Jami ID should be displayed prominently in whichever app you're +using. It looks like a long string of numbers and letters: +``f2c815f5554bcc22689ce84d45aefdda1bce9146`` + +Why don't I have to use a password? +~~~~~~~~~~~~ + +You are not forced to have a password on your account. On a +centralized system you would use your password to authenticate with a +public server where your account is stored. Someone who knows your +password could steal your identity. + +With Jami, your account is stored in a `folder +<#where-are-the-configuration-files-located>`_ on your device. **The +password is only used to encrypt your account to protect you from +someone who has physical access to your device.** + +If your device is encrypted, you may not want or need to use a +password. + +Why don't I have to register a username? +~~~~~~~~~~~~ + +The most permanent, secure identifier is your `Jami Id +<#where-is-my-jami-id>`_, but since these are difficult to use for +some people, you also have the option of registering a +username. Username registration requires a name server, such as Jami's +default one at ns.jami.net. + +If you don't register a username, you can still choose to register one +later at any time. + +Can I change my username? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +With the default nameserver you cannot change your username. + +What is the difference between a username and a display name? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can use your username as an identifier. The username points to +your `Jami Id <#where-is-my-jami-id>`_, which is your permanent, +secure identifier. Two people cannot have the same username. + +A display name allows you to choose another name that identifies you +to your contacts. Display names can be edited or changed at any time +and only your contacts can see them. + + +How can I back up my account? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +There are two ways to back-up your account: + +1. Link another device to your account so your account will be on two + devices. You can find this option in the account settings page. +2. Back up the `account archive + <technical-overview.html#account-storage-and-backup>`_ . This file + can be found in the account files `folder + <#where-are-the-configuration-files-located>`_. In some clients, + you can export this archive from the account settings. + +Can I retrieve my username without my keys? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you used the default name server at ``ns.jami.net``, **you +can’t**. There is no way to prove it’s your username without your key. + +If you use a different name server, there may be a way to move a +username to a new Jami Id at the discretion of the administrator of +that name server. + +For more information about name servers, see `the Technical Overview +<technical-overview.html#name-servers>`_. + +Can I recover my account if I forget my password? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +No. There can't be a traditional account recovery process; you are the +only person with access to your data. If you are worried about +forgetting your password, please use a password manager. + +What happens when I delete my account? +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Your account is only stored on your own devices. If you delete your +account from each device, the account is gone and you cannot get it +back. Nobody else can use your account after that. + +Your contacts will still have the messages you sent them, but all +public record of your account on the DHT will disappear. + +**Note for accounts with a username:** + +The default nameserver at ``ns.jami.net`` will not delete your +username, but nobody will be able to message you at that username or +register a new account with that username. + +Other name servers may allow username deletion (not recommended) at +the administrator's discretion. + +If you do not want to lose your account, please `back it up +<#how-can-i-back-up-my-account>`_! + +What happens when I link a new device? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +When you link a device to your account, your `account archive +<technical-overview.html#account-storage-and-backup>`_ is put on the +Jami network for a few minutes. It is protected by a password Jami +gives you. + +The new device receives your full account certificate with the master +RSA keys, but it generates a new device key for signing/encrypting +messages. + +Advanced +-------- + +What protocol does Jami use for the end-to-end encryption? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +We use TLS 1.3 with a perfect forward secrecy requirement for the +negotiated ciphers for calls and file transfers. Messages are +encrypted with an RSA key. + + +What data passes through my machine when I participate in the Jami network? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +**All these data are encrypted**. There is: + +- ICE descriptors of other Jami users. ICE is a protocol that help + establishing communication between two computers +- certain text messages +- as indicated above, accounts currently being linked to a new device + +Audio/video streams and some text messages pass through the VOIP +protocol. Text messages can be sent either via VOIP or DHT (the +distributed network) depending on whether a VOIP communication channel +is already open or not. + +Why am I able to communicate with myself? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Many users use Jami to transfer data from one machine to another. + +Should I enable push notifications? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Push notifications allow Jami to operate in a way more adapted to the +context of mobility (energy consumption, data…). However, for the +moment, notifications go through Google’s servers, via the Firebase +service. Only one identifier is transferred and it is unusable for +anyone who does not have access to your account. + +What is a bootstrap server? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +TODO + +What is a TURN server? What is STUN? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +TODO + +What is DHT proxy? +~~~~~~~~~~~~~~~~~~ + +The DHT proxy is a server that registers on the DHT for you and relays +your information to you. Thus, it is the server that will be active on +the DHT and will participate in the network, and no longer the target +device. Multiple devices can register on the same DHT proxy. + +How the username registration service work? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +For default parameters the usernames are registered on an Ethereum +blockchain. By default, it’s ns.jami.net that is used, but if you are a +developper, you can create your own system. Hence, nothing forces you to +implement it with a blockchain. You can check results at +http://ns.jami.net/name/test, where “test” is a username for which we +are looking for a matching `Infohashs <guidelines/Identifiers>`__. Once +registered, this server doesn’t provide a way to remove the mapping. +More informations there: +https://git.jami.net/savoirfairelinux/ring-project/wikis/technical/Name-Server-Protocol + +How can I change the timeout for a call? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In the ``dring.yml`` file, you can change your ringingTimeout (in +seconds) + +How to back up and reimport conversations and accounts +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Note: This is only for client based on LRC (desktop ones) + +First you will need to export all your accounts (For GNU/Linux: Settings +=> Account => Export account). Then you will need to copy the database +(in ``~/.local/share/jami`` for example). + +Then on the new device, when you will open Jami for the first time, you +have to re-import your accounts via the archive previously saved. This +will re-import your settings and contacts (with empty conversations). +Then close the client and replace the database with the one previously +saved. That’s all! + +How secure are you? +~~~~~~~~~~~~~~~~~~~ + +\*\* We use TLS/SRTP to secure connection and communications over the +network.*\* + +We implement SRTP over SIP using recommendations written in following +RFCs: + +- ```http://tools.ietf.org/html/rfc3711`` <http://tools.ietf.org/html/rfc3711>`__ +- ```http://tools.ietf.org/html/rfc4568`` <http://tools.ietf.org/html/rfc4568>`__ + +Typically 2 kind of sockets are negotiated. One for the control socket, +the other for the media sockets + +Typical control session will use the following cipher suite: +(TLS1.3)-(ECDHE-SECP384R1)-(RSA-PSS-RSAE-SHA384)-(AES-256-GCM) +(TLS_ECDHE_RSA_AES_256_GCM_SHA384) + +DTLS (fallback) supported: +“SECURE192:-KX-ALL:+ANON-ECDH:+ANON-DH:+SECURE192:-VERS-TLS-ALL:+VERS-DTLS-ALL:-RSA:%SERVER_PRECEDENCE:%SAFE_RENEGOTIATION” +TLS: +“SECURE192:-KX-ALL:+ANON-ECDH:+ANON-DH:+SECURE192:-RSA:-GROUP-FFDHE4096:-GROUP-FFDHE6144:-GROUP-FFDHE8192:+GROUP-X25519:%SERVER_PRECEDENCE:%SAFE_RENEGOTIATION” + +Supported crypto suite for the media session are: + +- AES_CM_128_HMAC_SHA1_80 / SRTP_AES128_CM_HMAC_SHA1_80 +- AES_CM_128_HMAC_SHA1_32 / SRTP_AES128_CM_HMAC_SHA1_32 + +When do public IPs get exposed? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +We can describe 3 main connectivity scenarios. A classic configuration +(1.), behind a VPN (2.), via Tor (3.). As Jami is a p2p app, I think you +understand that (2.) or (3.) is a bit mandatory to avoid IP leaking. + +Moreover, even if it’s my answer, you can choose to not trust my answer +and check the code, or use wireshark or other tools. Generally, I (and +the other devs I think) are using the first scenario (sometimes the +second one), and we surely can’t test all the network we want, so if you +discover a bug, please open a issue. + +Anyway, in these 3 scenarios, there is 3 main actions: + +- Send a message (this will use the DHT) +- Send a file (TCP ICE connection as described here: + https://git.jami.net/savoirfairelinux/ring-project/wikis/technical/2.5.%20File%20transfer) +- Do a call (TCP + UDP ICE connection as described here: + https://git.jami.net/savoirfairelinux/ring-project/wikis/technical/2.4.%20Let’s%20do%20a%20call) + +Classic config +^^^^^^^^^^^^^^ + +- Send a message + +The Jami application is running a DHT (https://opendht.net) node on your +device. So every operations on the DHT will use your ips. This is why +Jami has the option to use a dhtproxy (eg dhtproxy.jami.net), this will +avoid to use your node, but will use another node on the network (which +will see your ip). Note that your message is not sent directly to the +other device. In fact your message is sent on some nodes of the DHT and +your contact will retrieve the message on this node. So, your contact +don’t see your IP at this step, but the node who get the message will +(or they will see the IP of the proxy). + +- Send a file + +As described in the docs, you will send a message with all the IP you +know that your peer can contact in an encrypted packet. So, if your peer +send you a file or you send a file, your addresses will appear in the +ICE message. + +- Calls + +Same as above, the IP is present in the ICE. + +Behind a VPN +^^^^^^^^^^^^ + +- Send a message + +The IP of your VPN will be used by the DHT node. If you want a proof, +you can compile dhtnode and run the ‘la’ command to get your public +detected address. This is what I got: + +:: + + ./tools/dhtnode -b bootstrap.jami.net + Bootstrap: bootstrap.jami.net:4222 + OpenDHT node be58fdc9f782269bfc0bbfc21a60bca5f02cb881 running on port 54299 + (type 'h' or 'help' for a list of possible commands) + + >> la + Reported public addresses: + IPs OF MY VPN + +So, if you don’t use a proxy, your VPN addresses will be used for using +the DHT. If you use a dhtproxy, the dhtproxy will see your VPN addresses + +- Send a file + +Same as above, the ICE will contains: + addresses from your LAN + public +address of your VPN + TURN address if TURN is enabled + +- Do a call + +Same as above, your public address is replaced by your VPN address. You +can see it in the logs from daemon. See +https://git.jami.net/savoirfairelinux/ring-project/wikis/tutorials/Bug-report-guide#logs + +Tor +^^^ + +- Send a message + +Tor basically doesn’t supports UDP. This means that you can’t use your +DHT node locally, you MUST use a DHTProxy. That proxy will see the Exit +node. + +- Send a file + +I prefer a proof that any description. So, I did a file transfer with +Jami + TOR. This is what I see in the logs for the remote: + +:: + + [1574218330.556|10688|p2p.cpp :241 ] [Account:93a03f519f394143] add remote ICE candidate: Hc0a8c801 1 TCP 2130706431 192.168.200.1 33293 typ host tcptype passive + [1574218330.556|10688|p2p.cpp :241 ] [Account:93a03f519f394143] add remote ICE candidate: Hc0a8c801 1 TCP 2130706431 192.168.200.1 9 typ host tcptype active + [1574218330.556|10688|p2p.cpp :241 ] [Account:93a03f519f394143] add remote ICE candidate: Hc0a80103 1 TCP 2130706431 192.168.1.3 33293 typ host tcptype passive + [1574218330.556|10688|p2p.cpp :241 ] [Account:93a03f519f394143] add remote ICE candidate: Hc0a80103 1 TCP 2130706431 192.168.1.3 9 typ host tcptype active + [1574218330.556|10688|p2p.cpp :241 ] [Account:93a03f519f394143] add remote ICE candidate: R33fe279d 1 TCP 16777215 51.254.39.157 27427 typ relay tcptype passive + [1574218330.556|10688|p2p.cpp :241 ] [Account:93a03f519f394143] add remote ICE candidate: Sc0a8c801 1 TCP 1694498815 185.220.101.24 33293 typ srflx tcptype passive + +The first ones are some 192.168.x.x so we don’t care. 51.254.39.157 is +the TURN address in France (my device is in the Canada). 185.220.101.24 +is the Tor exit node: + +:: + + inetnum: 185.220.101.0 - 185.220.101.127 + netname: MK-TOR-EXIT + +- Do a call + +This will not work (actually, you can create the SIP control connection +because it’s a TCP connection), but medias are negotiated in UDP, so +this will fail. + +What ports does Jami use? +~~~~~~~~~~~~~~~~~~~~~~~~~ + +Jami works as a server and gets new ports for each connections (randomly +binded). These are the ranges that can be used for each component: + +- dht: UDP [4000, 8888] +- audio: UDP [16384-32766] +- video: UDP [49152-65534] +- SIP Control: UDP/TCP randomly binded + +So for ufw, we recommend to run: ``sudo ufw default allow outgoing`` + +For now, you can’t specify a specific range to configure ports used by +Jami. The inbound traffic can be controlled without issue, Jami should +work and can use a TURN server if needed. + +If you run your own proxy or nameserver: + +- dhtproxy, nameserver: TCP [80-100], 443 + +If you run your own TURN server: + +- TURN/STUN: TCP+UDP 3478, 5349 + +How can I configure the codecs even more? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Codecs can be configured via a file. In the configurations files, you +can create a file called ``encoder.json`` like this: + +:: + + { + "libx264": { + "profile": 100, + "level": 42, + "crf": 20, + "preset": "ultrafast" + }, + "h264_vaapi": { + "low_power": 1 + }, + "libopus": { + "application": "voip" + } + } + +or: + +:: + + { + "libopus": { + "bit_rate": 128000 + } + } + +This file is located in the same directory of +```dring.yml`` <#basics-5>`__ + +The best way to check which options are supported is through the command +“ffmpeg -h encoder=[encoder_name]” where encoder_name can be whichever +of libx264, libvpx, mpeg4, h263, libopus, libspeex, g722, pcm_alaw, +pcm_mulaw (FFmpeg names for all of Jami’s supported encoders). diff --git a/general/feature-requests.md b/general/feature-requests.md @@ -0,0 +1,49 @@ +# Feature Requests + +This page exists to classify features request coming from users +feedback (to avoid to let the ticket open for years). (under +construction, will sort the tickets) + +## Planned/In progress + ++ Group chat. Yeah we know it's missing but it's currently in progress (https://git.jami.net/groups/savoirfairelinux/-/epics/2) ++ Syncing history (linked to group chat support) ++ Read notifications (already available on some platforms) + +## Wanted but not planned + ++ A web interface. Because installing an app can be boring. Joining a conference via a link can be cool. For now, nobody is working on it. ++ Push to talk, voice detection ++ Ability to start a call without the camera (https://git.jami.net/savoirfairelinux/ring-project/issues/428) + +## Can be implemented, contributions welcome (or will take months/years to come) + ++ Support for Panic buttons (https://git.jami.net/savoirfairelinux/ring-project/issues/623) ++ Matrix bridge ++ Full tor support or other alternatives such as lokinet (https://git.jami.net/savoirfairelinux/ring-project/issues/922, https://git.jami.net/savoirfairelinux/ring-project/issues/622, https://git.jami.net/savoirfairelinux/ring-project/issues/495), i2p (https://git.jami.net/savoirfairelinux/ring-project/issues/630) ++ Bluetooth support (https://git.jami.net/savoirfairelinux/ring-project/issues/774) ++ Secret based turn server (https://git.jami.net/savoirfairelinux/ring-project/issues/886) ++ Option to compress files before sending it (https://git.jami.net/savoirfairelinux/ring-client-android/issues/720) ++ Trim recorded clips before sending ++ Spell checking support https://git.jami.net/savoirfairelinux/ring-client-gnome/issues/1169 ++ Echo bot to test audio https://git.jami.net/savoirfairelinux/ring-project/issues/392 ++ Search into system contacts (https://git.jami.net/savoirfairelinux/ring-client-gnome/issues/1191, https://git.jami.net/savoirfairelinux/ring-client-gnome/issues/829, etc) ++ Support for markdown https://git.jami.net/savoirfairelinux/ring-lrc/issues/416 ++ Handle click on jami:uri system wide https://git.jami.net/savoirfairelinux/ring-project/issues/653 + +## Depends on mass changes + ++ Emoticon Message Reactions (https://git.jami.net/savoirfairelinux/ring-project/issues/1034) (need to wait for group chat) + +## Others + ++ A thunderbird plugin (https://git.jami.net/savoirfairelinux/ring-project/issues/516) ++ OpenAlias (https://git.jami.net/savoirfairelinux/ring-project/issues/928) ++ CMIS integration (https://git.jami.net/savoirfairelinux/ring-project/issues/455) ++ Sound safety (https://git.jami.net/savoirfairelinux/ring-project/issues/441) ++ Ability to see multiple chats at the same time (https://git.jami.net/savoirfairelinux/ring-client-gnome/issues/909) ++ Vocoder option (https://git.jami.net/savoirfairelinux/ring-client-gnome/issues/957) ++ Socks5 support https://git.jami.net/savoirfairelinux/ring-project/issues/430 ++ Cardbook integration https://git.jami.net/savoirfairelinux/ring-project/issues/383 ++ Multiple instances running: https://git.jami.net/savoirfairelinux/ring-project/issues/629 ++ Whiteboard https://git.jami.net/savoirfairelinux/ring-daemon/issues/181 diff --git a/general/index.rst b/general/index.rst @@ -0,0 +1,16 @@ +########## +General +########## + +This chapter introduces Jami and tries to answer the most common +questions. + +.. toctree:: + :maxdepth: 1 + + introduction + faq + technical-overview + all-features-by-client + feature-requests + troubleshooting diff --git a/general/introduction.md b/general/introduction.md @@ -0,0 +1,39 @@ + + +# Introduction + +Jami is a free and private communication platform. + +It's free and open source, end-to-end encrypted, and requires no +central authority. + +Features include text chats, voice and video calls, screen sharing, +file sharing, conference calls, and group chats ([coming +soon](technical-overview.html#swarms)). Jami can also function as a +regular SIP client. + +Jami works on Windows, macOS, Linux, iOS, and Android. Multiple +devices can be linked to one account. No personal information is +required to create an account. + +## How does it work? + +Jami uses a [distributed hash +table](https://en.wikipedia.org/wiki/Distributed_hash_table) to +connect peers. Jami accounts are asymmetric x.509 certificates +generated by the GnuTLS library. Calls are made over the SIP protocol +after negotiating a secure connection using TLS. + +For more in-depth information, see [the Technical +Overview](technical-overview.html). + + +## Who makes Jami? + +This project is led by [Savoir-faire +Linux](https://www.savoirfairelinux.com/en/) -- a Canadian/Quebecois +GNU/Linux consulting company -- and is supported by a global +community. + +Jami is Free software: its sources are licensed under the +[GPLv3+](https://www.gnu.org/licenses/gpl-3.0.html). diff --git a/general/old-understanding.md b/general/old-understanding.md @@ -0,0 +1,196 @@ +# Understanding Jami + +The goal of this page is to explain the Jami network clearly and +concisely. + +## Prerequisites + +The page does not go into the details. (For more in-depth information +about implementation, libraries, and APIs see the [Technical +Reference](../technical/index.html).) + +Nevertheless, distributed systems are complex. You may have to +research some of these terms and draw your own diagrams. At the +least, you should have a basic understanding of [public-key +cryptography](https://en.wikipedia.org/wiki/Public-key_cryptography) +and computing concepts like [software +libraries](https://en.wikipedia.org/wiki/Library_(computing)). + +If you are still struggling to understand part of this document, +please reach out to us and we can try to help you understand it or +make improvements, if needed. + +## What is a Jami account? + +A Jami account is an x.509 certificate as defined by [RFC +5280](https://tools.ietf.org/html/rfc5280) with a 4096-bit RSA key +pair. + +Jami uses the [gnutls](https://www.gnutls.org/) library to generate +and manage RSA keys and certificates. + +### Account certificate + +An account certificate is generated at account creation and represents +the identity of a Jami user. The certificate contains: + +- the public key +- its 160 bit fingerprint (the Jami Id) +- the fingerprint of the certificate issuer, which can be a + certificate authority, but most likely is just same as the Jami ID, + indicating a self-signature +- the signature: a signed hash of the certificate + + +### Device certificate + +Each device has its own certificate. Device certificates look like +account certificates except that instead of being self-signed they are +signed by the RSA key associated with the account certificate. + +For more information about Jami certificates, see +[Certificates](certificates.html). + +The fingerprint of the device's public key is called the Device +Id. + +### Account storage and backup + +Jami serializes account data using JSON, including contacts, daemon +settings, certificates, revoked devices, display name, and almost +anything else besides message history. The file is compressed using +Gzip and stored in the [account files +folder](faq.html#where-are-the-configuration-files-located) as +`archive.gz`. If you use an account password, this file will also be +encrypted. + +Linking a device entails simply copying this file to another device +over the DHT. + +## The DHT + +Jami uses a [distributed hash +table](https://en.wikipedia.org/wiki/Distributed_hash_table) library, +[OpenDHT](https://github.com/savoirfairelinux/opendht) (also led by +Savoir-faire Linux), to find peers, connect new devices, and even +temporarily store messages. Understanding how a DHT works is out of +the scope of this overview, but if you want to learn more, [this blog +post](https://enconn.fr/blog/p2p-internals-dht/) is a good +introduction. + +For now, think of it as a *distributed* way to store key-value pairs +on the Jami network. + +You can think of the key `k` as a "location" on the DHT. `put(k, v)` +stores the value `v` on the DHT at location `k`, while `get(k)` gets +the value(s) stored at `k`. + +The key, in this case, is always a Jami Id or Device Id. Jami checks +that values are signed by a device key belonging to the correct Jami +ID. + +### DHT example: announcing and checking presence + +To announce your presence to the Jami network, simply `put(<Jami ID>, +<Device ID>)` on the DHT. To check for the presence of your contacts, +check the DHT for devices announced at the contact's Jami ID: +`get(<Jami ID>)`. + +### Doing more with the DHT + +It should be clear by now that the DHT is a useful data store that +serves as an intermediary for peers. Clients exchange IP addresses, +device and account certificates, text messages, device revocations, +and more over the DHT. + +Once accounts have exchanged their certificates, they can encrypt and +sign messages before storing them on the DHT. So your IP address and +message content is always encrypted to a 4096-bit RSA key while on the +public DHT. + +The OpenDHT library takes care of encrypting and signing values with +your certificate. + +OpenDHT has other useful features, like the ability to "listen" on a +DHT location (rather than polling every X seconds), but we will not +cover all of them. + +## Significance of the Jami Id + +Jami Ids are a fundamental way to "bootstrap" secure communication +between two peers. As long as you have the correct Jami Id for your +friend, nobody can impersonate them or read messages you've sent +them. The Jami Id is used to verify certificates during the initial +certificate exchange. OpenDHT handles certificate exchange with an +"identity layer" for publishing certificates. See [OpenDHT's +documentation](https://github.com/savoirfairelinux/opendht/wiki) for +more information. + + + + +## Calls and file transfers +The DHT's bandwidth and latency are not good enough for real-time +communications (RTC) and file tranfers. For calls and file transfers, +we must make direct (peer-to-peer) connections between devices. + +Once our devices are connected, Jami uses the IETF [SIP +standard](https://tools.ietf.org/html/rfc3261) to share files and +stream audio and video with the [PJSIP](https://www.pjsip.org/) +library. The hard part is first establishing a secure connection. Jami +does this with ICE. + +### Connecting devices with ICE + +We've seen that securely exchanging IP addresses over the DHT is easy, +but how do we actually connect two devices? We need to navigate +firewalls and other network conditions. + +[RFC 5245](https://tools.ietf.org/html/rfc5245) defines ICE +(Interactive Connectivity Establishment), a protocol for NAT +traversal. Jami uses ICE to establish a peer-to-peer communication +between two devices. + +As a quick summary, each client gathers a list of potential IP:port +addresses called **ICE candidates**. The peer initiating the call or +tranfer stores these candidates on the DHT (encrypted to the +recipient, of course). Once the recipient accepts the connection by +storing their candidate list, the devices each build a list of +**candidate pairs** and the devices negotiate a connection using rules +specified by ICE. + + +### Encrypting the connection with GnuTLS + +Once a peer-to-peer communication channel has been established, the +recipient device listens on it for incoming DTLS connections (acting +as a DTLS server) while the caller initiates an outgoing DTLS +connection (acting as a DTLS client). + +The DTLS communication must be +[RFC6347](https://tools.ietf.org/html/rfc6347) compliant. + + +Peers must only support PFS (perfect forward secrecy) cypher +suites. The set of supported cypher suites is implementation defined +but should include at least ECDHE-AES-GCM (TODO: specify the exact +suites recommended to support). + +During the DTLS handshake, both peers must provide their respective +device certificate chain and must authenticate the other peer. Jami +checks that its public key is the same key used during the DHT ICE +exchange. + +This connection is similar to the connection your browser makes with a +web server (web browsers actually use DTLS for WebRTC). Jami uses +GnuTLS instead of implementing its own crypto. This keeps connections +up-to-date with the latest algorithms and security fixes, and helps us +avoid the pitfalls of DIY crypto. + +TODO: Someone please review the wording and correctness of this section^^^^ + +## + + + +## Swarm: a new generation of group conversations diff --git a/general/technical-overview.txt b/general/technical-overview.txt @@ -0,0 +1,248 @@ +Technical Overview +================== + +The goal of this page is to explain the Jami network clearly and +concisely without going into details like implementation, libraries, +and APIs. For more in-depth information, see the `Technical Reference +<../technical/index.html>`__. + +.. contents:: + :local: + :depth: 3 + +Before reading +------------- + +Distributed systems have many moving parts. You may have to research +some of these terms and draw your own diagrams. You should have a +basic understanding of `public-key cryptography +<https://en.wikipedia.org/wiki/Public-key_cryptography>`__ and +computing concepts like `software libraries +<https://en.wikipedia.org/wiki/Library_(computing)>`__. + +If you are still struggling to understand part of this document, please +reach out to us and we can try to help you understand it or make +improvements, if needed. + +What is a Jami account? +----------------------- + +A Jami account is an x.509 certificate as defined by `RFC +5280 <https://tools.ietf.org/html/rfc5280>`__ with a 4096-bit RSA key +pair. + +Jami uses the `GnuTLS <https://www.gnutls.org/>`__ library to generate +and manage RSA keys and certificates. + +Account certificate +~~~~~~~~~~~~~~~~~~~ + +An account certificate is generated at account creation and represents +the identity of a Jami user. The certificate contains: + +- the public key +- its 160 bit fingerprint (the Jami Id, usually displayed as 40 hex digits) +- the fingerprint of the certificate issuer, which can be a certificate + authority, but most likely is just same as the Jami ID, indicating a + self-signature +- the signature: a signed hash of the certificate + +Device certificate +~~~~~~~~~~~~~~~~~~ + +Each device has its own certificate. Device certificates look like +account certificates except that instead of being self-signed they are +signed by the RSA key associated with the account certificate. + +For more information about Jami certificates, see +`Certificates <certificates.html>`__. + +The fingerprint of the device’s public key (analogous to a Jami Id) is +called the Device Id. + +Account storage and backup +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Jami serializes account data using JSON, including contacts, daemon +settings, certificates, revoked devices, display name, and almost +anything else besides message history. The file is compressed using Gzip +and stored in the `account files +folder <faq.html#where-are-the-configuration-files-located>`__ as +``archive.gz``. If you use an account password, this file will also be +encrypted. + +Linking a device entails simply copying this file to another device over +the DHT. + +The DHT +------- + +Jami uses a `distributed hash +table <https://en.wikipedia.org/wiki/Distributed_hash_table>`__ library, +`OpenDHT <https://github.com/savoirfairelinux/opendht>`__ (also led by +Savoir-faire Linux), to find peers, connect new devices, and even +temporarily store messages. Understanding how a DHT works is out of the +scope of this overview, but if you want to learn more, `this blog +post <https://enconn.fr/blog/p2p-internals-dht/>`__ is a good +introduction. + +For now, think of it as a *distributed* way to store key-value pairs on +the Jami network. + +You can think of the key ``k`` as a “location” on the DHT. ``put(k, v)`` +stores the value ``v`` on the DHT at location ``k``, while ``get(k)`` +gets the value(s) stored at ``k``. + +The key, in this case, is always a Jami Id or Device Id. Jami checks +that values are signed by a device key belonging to the correct Jami ID. + +DHT example: announcing and checking presence +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To announce your presence to the Jami network, simply +``put(<Jami ID>, <Device ID>)`` on the DHT. To check for the presence of +your contacts, check the DHT for devices announced at the contact’s Jami +ID: ``get(<Jami ID>)``. + +Doing more with the DHT +~~~~~~~~~~~~~~~~~~~~~~~ + +It should be clear by now that the DHT is a useful data store that +serves as an intermediary for peers. Clients exchange IP addresses, +device and account certificates, text messages, device revocations, and +more over the DHT. + +Once accounts have exchanged their certificates, they can encrypt and +sign messages before storing them on the DHT. This means that your IP +address and messages are always encrypted to a 4096-bit RSA key while +on the public DHT. + +The OpenDHT library takes care of encrypting and signing values with +your certificate. OpenDHT also handles certificate exchange with an +“identity layer” for publishing certificates. See `OpenDHT’s +documentation <https://github.com/savoirfairelinux/opendht/wiki>`__ +for more information. + +OpenDHT has other useful features, like the ability to “listen” on a DHT +location (rather than polling every X seconds), but we will not cover +all of them. + +Significance of the Jami Id +--------------------------- + +Jami Ids are a fundamental way to “bootstrap” secure communication +between two peers. As long as you have the correct Jami Id for your +friend, nobody can impersonate them or read messages you’ve sent them. +The Jami Id is used to verify certificates during the initial +certificate exchange. + + +Calls and file transfers +------------------------ + +The DHT’s bandwidth and latency are not good enough for real-time +communications (RTC) and file tranfers. For calls and file transfers, we +must make direct (peer-to-peer) connections between devices. + +Once our devices are connected, Jami uses the IETF `SIP +standard <https://tools.ietf.org/html/rfc3261>`__ to share files and +stream audio and video with the `PJSIP <https://www.pjsip.org/>`__ +library. The hard part is establishing a connection, and then securing it. Jami +does the former with ICE and the latter with GnuTLS. + +Connecting devices with ICE +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +We’ve seen that securely exchanging IP addresses over the DHT is easy, +but how do we actually connect two devices? We need to navigate +firewalls and other network conditions. + +`RFC 5245 <https://tools.ietf.org/html/rfc5245>`__ defines ICE +(Interactive Connectivity Establishment), a protocol for NAT traversal. +Jami uses ICE to establish a peer-to-peer communication between two +devices. + +As a quick summary, each client gathers a list of potential IP:port +addresses called **ICE candidates**. The peer initiating the call or +tranfer stores these candidates on the DHT (encrypted to the recipient, +of course). Once the recipient accepts the connection by storing their +candidate list, the devices each build a list of **candidate pairs** and +the devices negotiate a connection using rules specified by ICE. + +Encrypting the connection with GnuTLS +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Once a peer-to-peer communication channel has been established, the +recipient device listens on it for incoming DTLS connections (acting as +a DTLS server) while the caller initiates an outgoing DTLS connection +(acting as a DTLS client). + +The DTLS communication must be +`RFC6347 <https://tools.ietf.org/html/rfc6347>`__ compliant. + +Peers must only support PFS (perfect forward secrecy) cypher suites. The +set of supported cypher suites is implementation defined but should +include at least ECDHE-AES-GCM + +TODO: specify the exact suites recommended to support. + +During the DTLS handshake, both peers must provide their respective +device certificate chain and must authenticate the other peer. Jami +checks that its public key is the same key used during the DHT ICE +exchange. + +This connection is similar to the connection your browser makes with a +web server (web browsers actually use DTLS for WebRTC). Jami uses GnuTLS +instead of implementing its own crypto. This keeps connections +up-to-date with the latest algorithms and security fixes, and helps us +avoid the pitfalls of DIY crypto. + +TODO: Someone please review the wording and correctness of this +section^^^^ + +Swarms +------ + +Swarms are a new generation of group conversations. + +`Here is the January 2021 announcement +<https://jami.net/swarm-introducing-a-new-generation-of-group-conversations/>`_ about the feature. + +The design is still changing frequently, so we will defer to the +`Technical Reference <../technical/basic-features/swarm.html>`_ for a +full explanation. + +However, as a quick summary: Swarms use `Git <https://git-scm.com/>`_ +to store and sync conversations between participants over a p2p +connection. Each swarm is a single distributed Git repository, and +each message is a commit. + +Participants include all linked devices, so swarms will bring +much-requested full conversation sync between devices. This also fixes +a few other things like `contacts not seeing avatars +<faq.html#why-is-my-contact-not-seeing-my-avatar>`_. + +Group conference calls +---------------------- + +Optional servers: DHT proxy, TURN, STUN +---------------------------- + +DHT proxy and push notifications +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +TURN and STUN +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + + +DHT bootstrap servers +-------------------------- + +Name servers +-------------------------------------- + +ns.jami.net +~~~~~~~~~~~~~~~~~~~~ + +JAMS: Managed Jami networks for organizations +-------------------------------------------- diff --git a/general/troubleshooting.rst b/general/troubleshooting.rst @@ -0,0 +1,38 @@ +Troubleshooting +================= + +Audio/Video +------------------------------ + +How to configure echo cancellation? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Go to settings -> media -> and set change your audio manager. We +recommend using Pulse audio. + +Video is laggy +~~~~~~~~~~~~~~ + +Use a lower definition. Go to settings -> media -> and change +resolution. You can also try auto mode by clicking the HQ button during +a call. + +I can’t see myself +~~~~~~~~~~~~~~~~~~ + +End call and check that you have selected the right video device. Go to +settings -> media -> and change device. + +Check that you can effectively see yourself in the right panel of +settings -> media. You can double check that your device is working by +installing the cheese application. + +Others can’t ear me +~~~~~~~~~~~~~~~~~~~ + +End call and try changing audio driver or audio input device. + +I have no sound +~~~~~~~~~~~~~~~ + +End call and try changing audio driver or audio output device. diff --git a/guides/how-to-contribute-to-this-documentation.txt b/guides/how-to-contribute-to-this-documentation.txt @@ -0,0 +1,137 @@ +############################ +How to Contribute to This Documentation +############################ + +Contributions to these docs are always welcome, from small corrections +to whole new chapters. + +This page will walk through the steps to create a new page or submit a +correction. The patch review process is the same as :doc:`for any +other Jami project <Submit-your-first-patch>`, so we will not explain +every command. + +**TODO: internationalization** + +Dependencies +================================ + +You will need Git installed and configured to use your SSH keypair, +and you will need an account on `Gerrit <review.jami.net/>`_ . If you +need help with this, see :doc:`the beginning of our patch submission +guide <Submit-your-first-patch>` + +If you want to preview your changes in a web browser, you must install +`Sphinx <https://www.sphinx-doc.org>`_. You also need to install a +markdown parser for Sphinx: + +.. code-block:: bash + + $ pip install --upgrade recommonmark + + +Cloning the repository +========================================= +Clone the repository and configure the push settings like this: + +.. code-block:: bash + + $ git clone "ssh://USERNAME@review.jami.net:29420/jami-docs.git" + $ cd jami-docs + $ git config remote.origin.push HEAD:refs/for/master + +It might be a good idea to checkout a new branch before you make changes: + +.. code-block:: bash + + $ git checkout -b my_wiki_change + +Editing a page +========================================= + +Pages are written in either markdown or `reStructuredText +<https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html>`_. You +can click "View page source" at the top of any page to see how the +page was written. + +Go ahead and make your changes to ``.rst`` or ``.md`` files in the +``source`` folder. + +Previewing your work +======================================= + +From the base of the repository, run: + +.. code-block:: bash + + $ make html + +You should now be able to view the documentation in your web +browser. The homepage is at ``build/html/index.html``. + +Saving your work +================================================ + +.. code-block:: bash + + $ git add source/file/you/edited.md + $ git commit + +Your commit message should look something like this: + +.. code-block:: none + + wiki: short summary of your change in present tense + + Longer description of your change in complete sentences, if necessary. + + Gitlab issue numbers (e.g. #445), if relevent. + +Submitting a change +================================================ + +The first time you try to push your changes, Gerrit will complain that +you don't have a Change-Id in your commit, and provide an ``scp`` +command to install the commit hook. After running the command, you +should be able to recommit and push your change: + +.. code-block:: bash + + $ git commit --amend --no-edit + $ git push + + +Modifying your work +================================================ + +A reviewer may ask you to make changes to your patch before accepting +it. This is no problem! Simply make the changes, ``git add`` them, and +run ``git commit --amend`` to modify the patch. + +After your first time running ``git commit``, **do not** run ``git +commit`` again on a patch. Instead, you should always type: + +.. code-block:: bash + + $ git commit --amend + +Adding a page +=============================================== + +If you decide to add a whole new page to the documentation, you must +also add it to the ``toctree`` directive of that chapter. + +For instance, if you added a new guide called +``Hosting-JAMS-on-AWS.md`` in the ``guides`` folder, you should add it +in the ``toctree`` directive of ``guides/index.rst``: + + +.. code-block:: ReST + + .. toctree:: + + Bug-report-guide + ... + Hosting-JAMS-on-AWS + + +(Don't include the filename extension.) diff --git a/guides/how-to-make-a-bug-report.md b/guides/how-to-make-a-bug-report.md @@ -0,0 +1,229 @@ +How to Make a Bug Report +---------------- + +**Note: We are currently a few devs active on the project. We can’t +answer and tags each issues opened, but we read it. A good issue provide +an important feedback and thank you for that, any feedback is +appreciated.** + +Setup your environment +~~~~~~~~~~~~~~~~~~~~~~ + +- Be ready for data loss. Backup your account and link your account to + as many devices as possible. +- Install the latest version (or even a beta version) of Jami. It is + useless to report bugs on older versions. + +How to report a bug +~~~~~~~~~~~~~~~~~~~ + +0. Create an account on `our git +repository <https://git.jami.net/users/sign_in>`__ (if it is not already +done) + +1. Choose the right project to post your issue in: \* `The Android +client <https://git.jami.net/savoirfairelinux/ring-client-android>`__ \* +`The Windows +client <https://git.jami.net/savoirfairelinux/ring-client-windows>`__ \* +`The Linux +client <https://git.jami.net/savoirfairelinux/ring-client-gnome>`__ \* +`The iOS +client <https://git.jami.net/savoirfairelinux/ring-client-ios>`__ \* +`The macOS +client <https://git.jami.net/savoirfairelinux/ring-client-macosx>`__ \* +`The Jami project in general (or if you are not +sure) <https://git.jami.net/savoirfairelinux/ring-project>`__ \* `If you +know what you are doing you may choose one of the other +projects <https://git.jami.net/>`__ + +2. If you have multiple issues, please file separate bug reports. It +will be much easier to track bugs that way. + +3. Title is an explicit summary of the bug (e.g.: header bar is too big +due to icon size) + +4. Figure out the steps to reproduce the bug: + +- If you have precise steps to reproduce it — great! — you’re on your + way to creating a useful bug report. +- If you can reproduce occasionally, but not after following specific + steps, you must provide additional information for the bug report to + be useful. +- If you can’t reproduce the problem, there’s probably no use in + reporting it, unless you provide unique information about its + occurrence. +- If the bug leads to other bugs afterward that you can’t reproduce + some other way, there’s probably no use in reporting it. + +5. Make sure your software is up to date. Ideally, test an +in-development version to see whether your bug has already been fixed + +6. Try to isolate from environment and reproduce (ie: test on multiple +devices) + +7. Describe your environment(s) by specifying the following: + +- OS version +- precise device model (important for mobile devices) +- if you are using a beta version +- what build you are using (from ring.cx, F-droid, Play Store, App + store, you own…). If you have built your own version of Ring, please + specify the exact dring version (LibRing or Daemon) and client + version (You can obtained it with ``dring -v`` and ``jami-gnome -v``. + But note that our packages are updated quite often) and the Git + commit. +- network conditions: Are both devices on the same local network? + Different networks? Is one or both behind NAT? Are you using LTE? + Wifi? +- other elements if needed: SIP provider, hardware… + +Writing a clear summary +~~~~~~~~~~~~~~~~~~~~~~~ + +How would you describe the bug using approximately 10 words? This is the +first part of your bug report a developer will see. + +A good summary should quickly and uniquely identify a bug report. It +should explain the problem, not your suggested solution. + +:: + + Good: "Cancelling a file transfer crashes Ring" + Bad: "Software crashes" + +:: + + Good: "All calls hang up after 32 seconds" + Bad: "Not able to call my friends" + +Writing precise steps to reproduce +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- How can a developer reproduce the bug on his or her own device? + +Steps to reproduce are the most important part of any bug report. If a +developer is able to reproduce the bug, the bug is very likely to be +fixed. If the steps are unclear, it might not even be possible to know +whether the bug has been fixed. We are totally aware that some bugs may +look obvious to you, but they are probably related to your environment. +The more precise you are, the quicker the bug is fixed. + +- What should you include in a bug report? + +Indicate whether you can reproduce the bug at will, occasionally, or not +at all. Describe your method of interacting with Ring in addition to the +intent of each step. After your steps, precisely describe the observed +(actual) result and the expected result. Clearly separate facts +(observations) from speculations. + +Good : +^^^^^^ + +I can always reproduce by following these steps: + +:: + + 1. Start Ring by clicking on the desktop icon + 2. Start a new conversation with anyone + 3. Click file transfer icon + + Expected results: A window opens and ask me to choose a file to send. + Actual results: When I click the file transfer icon, nothing happens. + +Bad : +^^^^^ + +:: + + Try to transfer a file + It doesn't work. + +Obtained Result +~~~~~~~~~~~~~~~ + +Please include: + +- The daemon (or LibRing) and client debug logs. +- The core dump if one was produced. + +Expected Result +~~~~~~~~~~~~~~~ + +It’s a description of expected or desired behaviour. + +Providing additional information +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The following information is requested for most bug reports. You can +save time by providing this information below the Expected results. + +Logs +^^^^ + +On GNU/Linux +'''''''''''' + +Classic logs (by default logs only >= warning are loggued): + +:: + + journalctl --since "24h ago" | grep dring + +Full log: Since the Ring GUI and daemon are separated processes, the +easiest way to get logs from both is to start them one at a time, +manually. + +1. Ensure that no ring client or daemon instances are running with + ``ps aux | grep ring`` + + - Ring may still be running even if no windows are open depending on + your preferences. + - If either client or daemon are running, kill them with + ``kill PID`` + +2. On one terminal, start the daemon with ``dring -d -c`` + + - This executable is normally not in the PATH, and on the Ubuntu + packages, it is located at + ``/usr/lib/x86_64-linux-gnu/dring -d -c`` or + ``/usr/lib/ring/dring -d -c`` + +3. In another terminal, start the client with (here is a Gnome example) + ``jami-gnome -d`` + +For getting a backtrace, you can run the program inside gdb: + +̀ gdb -ex run jami-gnome\ ``or``\ gdb -ex run –args /usr/lib/ring/dring +-cd\` + +When it does crash, you can type ``bt`` then press **Enter**. And copy +the backtrace to the issue. (or ``thread apply all bt`` is event better) + +On Mac OS +''''''''' + +Open the Terminal app and launch Ring with +``/Applications/Ring.app/Contents/MacOS/Ring`` + +- Open Console +- Click on “All messages” +- Search field: “Jami” + +On Android (debug builds only) +'''''''''''''''''''''''''''''' + +- You need to have adb setup on your computer. +- Launch Ring on your smartphone and then execute +- ``adb logcat *:D | grep \``\ adb shell ps \| egrep ‘cx.ring’ \| cut + -c10-15\` > logring.txt\` +- You now have a file containing the log of the client + +For Windows +''''''''''' + +Open a terminal(cmd.exe) and launch Jami.exe with the following options: + +- ``-d`` to open a separate console window to receive logs +- ``-f`` to write logs to ``%localappdata%\jami\jami.log`` +- ``-c`` to print logs directly to the Visual Studio debug output + window diff --git a/guides/how-to-set-up-a-dhtproxy-or-bootstrap-server.md b/guides/how-to-set-up-a-dhtproxy-or-bootstrap-server.md @@ -0,0 +1,4 @@ +# How to Set Up a DHTproxy or DHT Bootstrap Server + + +TODO ([contributions welcome!](contributing-to-this-documentation.html)) diff --git a/guides/how-to-set-up-a-turn-server.txt b/guides/how-to-set-up-a-turn-server.txt @@ -0,0 +1,75 @@ +############################ +How to Set Up a TURN Server +############################ + +Jami can be configured to use TURN or STUN servers (`RFC +5766 <https://tools.ietf.org/html/rfc5766>`__) to establish a connection +between two peers. + +The default TURN server is “turn.jami.net”, with username “ring”, +password “ring”, and realm “ring”. + +In this guide, we will setup a +`coturn <https://github.com/coturn/coturn>`__ server. There are other +TURN/STUN server implementations available under a free license, such as +`TurnServer <http://turnserver.sourceforge.net/>`__ and +`Restund <http://www.creytiv.com/restund.html>`__. + +Installing +========== + +COTURN is available in most Linux distributions. On Debian, install it +with the following command: + +.. code:: bash + + apt-get install coturn + +Configuring +=========== + +Here is a basic ``turnserver.conf`` file: + +:: + + listening-port=10000 + listening-ip=0.0.0.0 + min-port=10000 + max-port=30000 + lt-cred-mech + realm=sfl + +This also will function as a STUN server. The STUN server does not +require a username and password (STUN uses very little bandwidth). + +Creating users on your TURN server +================================== + +To create users on your TURN server, use the ``turnadmin`` binary (this +might require superuser permissions). + +.. code:: bash + + turnadmin -a -u bob -p secretpassword -r sfl + +Launching the TURN server +========================= + +.. code:: bash + + turnserver -c turnserver.conf + +Configuring Jami to authenticate with the TURN server +===================================================== + +You can configure Jami to use your TURN server from the advanced section +of your account settings: + +============== ============================ ============== +Field Value Example +============== ============================ ============== +**server url** host and port of your server 0.0.0.0:10000 +**username** username bob +**password** password secretpassword +**realm** realm sfl +============== ============================ ============== diff --git a/guides/how-to-submit-a-patch.md b/guides/how-to-submit-a-patch.md @@ -0,0 +1,63 @@ +# How to Submit a Patch + +These are the steps + +## Setting up git and ssh + +(**to work smoothly with Gerrit.**) + + +first see: + +* [Creating a gerrit review](https://git.jami.net/savoirfairelinux/ring-project/wikis/tutorials/Working-with-gerrit#to-create-the-review) + +## SSH setup + +https://review.jami.net/Documentation/user-upload.html#ssh + +1. Generate a personal dedicated public / private key set. +```bash +ssh-keygen -t rsa -f ~/.ssh/jami_gerrit_review_rsa` +``` + +Your identification has been saved in `jami_gerrit_review_rsa`. Your public key has been saved in `jami_gerrit_review_rsa.pub.` + + +2. Tell gerrit your public key + 1. Login to [gerrit](https://review.jami.net) via your Gitlab account (Sign-in=>OAuth Gitlab) + 2. Follow the ssh key instructions found from (your) user options [settings](https://review.jami.net/settings/) + +3. Set up your local ssh (via `ssh-add` or in `~/.ssh/config`) +4. Test all of the above (ssh on the service or just try to clone a repo on gerrit via ssh) + + +## Your Repository + +This *assumes* you have moved any existing project clone out of the way. + +1. Clone a (fresh) copy of the project. `git clone ssh://USERNAME@review.jami.net:29420/PROJECT_NAME_GOES_HERE.git` eg `git clone ssh://foo@review.jami.net:29420/ring_project.git`. +2. Configure this clones local .git/config (optional) +3. Generate commit Change-Ids +aka: commit-msg hook script + +A shell script, to be installed to .git/hooks/commit-msg. + + +The script creates a unique Change_Id:hash for each commit made from your repository. + + +- The first time you attempt a _push_ to review `git push origin HEAD:refs/for/master`, + gerrit will notice a lack of Change-Id. + +- Gerrit will also present a secure copy (scp) string. + Use *that* to download a copy of .git/hooks/commit-msg. + + +References +---------- + ++ [Some Gritty Details](https://review.jami.net/Documentation/user-upload.html#_gritty_details) + + + +Original author: *Peter Gossner* diff --git a/guides/index.rst b/guides/index.rst @@ -0,0 +1,15 @@ +########## +Guides: +########## + +These are how-to guides that should `follow this format +<https://documentation.divio.com/how-to-guides/>`_. + +.. toctree:: + :maxdepth: 1 + + how-to-make-a-bug-report + how-to-contribute-to-this-documentation + how-to-submit-a-patch + how-to-set-up-a-TURN-server + how-to-set-up-a-dhtproxy-or-bootstrap-server diff --git a/index.rst b/index.rst @@ -0,0 +1,33 @@ +.. Jami documentation master file, created by + sphinx-quickstart on Sat Feb 13 12:46:38 2021. + You can adapt this file completely to your liking, but it should at least + contain the root `toctree` directive. + +Home +============================================== + +.. image:: https://jami.net/assets/images/logo-jami.svg + :alt: Jami Logo + +This is the documentation for Jami, a free and private communication +platform. + +This documentation is community-driven and :doc:`anyone can +contribute<guides/how-to-contribute-to-this-documentation>`! + +.. toctree:: + :maxdepth: 2 + + general/index + guides/index + building-jami/index + technical/index + extra/index +.. + + Indices and tables + ================== + + * :ref:`genindex` + * :ref:`modindex` + * :ref:`search` diff --git a/technical/APIs.md b/technical/APIs.md @@ -0,0 +1,281 @@ +# APIs + +## OpenDHT + +The documentation related to the API of OpenDHT is [here](https://github.com/savoirfairelinux/opendht/wiki/API-Overview) and will not be detailed in the following part. + +## Daemon + +### The managers + +The API of the daemon is decomposed between 5 Managers + 1 Instance file: ++ The **CallManager** interface is used to manage call and conference related actions. Since Ring-daemon supports multiple incoming/outgoing calls, any actions involving a specific call must address the method by the means of a unique callID. Ring-daemon will generate a unique callID for outgoing and incoming calls. ++ The **ConfigurationManager** used to handle the configuration stuff: accounts settings, user preferences, ... ++ The **PresenceManager** is used to track the presence of contacts ++ The **VideoManager** used to manage video devices and renderers ++ The **Instance** is used to count the number of clients actually registered to the core. When initializing your client, you need to register it against the core by using this interface. + +### DBUS + +All the documentation and code for the dbus API is located in `ring-daemon/bin/dbus`. + +If you use linux, you can use `d-feet` when the daemon is running to manipulate the API (or with any another tool). + +The LRC project uses this API (and use libwrap on windows and mac os). + +### JNI + +All the documentation and code for the JNI API is located in `ring-daemon/bin/jni`. + +#### Generation process + +- `cd ring-project/` +- Check `./daemon/src/dring` +- Create a new file .i in `./daemon/bin/jni/` from an existing file in the same folder +- Update with your new interfaces from documentation +- Check that callback block is at the beginning AND at the end of file +- `cd daemon/bin/jni` +- Edit ./daemon/bin/jni/jni\_interface.i and add yours with the other .i files that are in the same folder +- `export PACKAGEDIR=\*\*\*\*/ring-project/client-android/ring-android/libringclient/src/main/java/cx/ring/daemon.` + ie: `export PACKAGEDIR=/home/pduchemin/Documents/ring-project/client-android/ring-android/libringclient/src/main/java/cx/ring/daemon` +- Run + - `./make-swig.sh` +- or, if you start from scratch + - `./make-ring.py --init --distribution=Android` + - `./make-ring.py --install --distribution=Android` + +### node js + +All the documentation and code for the Node JS API is located in `ring-daemon/bin/nodejs`. This API is not used in any known project and maybe is not up-to-date. + +### REST + +All the documentation and code for the REST API is located in `ring-daemon/bin/restcpp`. This API is not used in any known project and maybe is not up-to-date. + +### Python wrapper + +A Python wrapper is available in `ring-daemon/tools/dringctrl`. This wrapper uses d-bus. + +This is, for example a quick and dirty IRC bot to interact with Ring made with this API: + +`config.json`: +```json +{ + "users": { + }, + "debug": true, + "host": "irc.freenode.net", + "follow_all": { + }, + "port": 6697, + "nick": "NICKNAME", + "ssl": true, + "channel": "#REPLACE" +} +``` + +`irc,py`: +```python +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- +from controler import DRingCtrl +from threading import Thread +import irc3 +from irc3 import utils +import json +import sys + + +@irc3.plugin +class RingBridge: + '''This plug-in react to event from IRC and send messages to IRC''' + + requires = [ + 'irc3.plugins.core', + 'irc3.plugins.userlist', + 'irc3.plugins.command', + 'irc3.plugins.human', + ] + + def __init__(self, bot): + self.bot = bot + self.log = self.bot.log + self.channels = utils.as_list(self.bot.config.get('autojoins', [])) + self.config = None + + @irc3.event(irc3.rfc.PRIVMSG) + def on_privmsg(self, mask=None, target=None, data=None, **kwargs): + '''React to messages from IRC''' + message = data.split(' ') + follow_all = self.bot.config_bridge['follow_all'] + if message[0] == '!tell': + msg_to_send = ('%s: ' % mask.nick) + ''.join( + '%s ' % m for m in message[2:])[:-1] + call_id = None + users = self.bot.config_bridge['users'] + # Get ring_id from nick + for ring_id in users.keys(): + if users[ring_id].lower() == message[1].lower(): + call_id = ring_id + if call_id: + # if this nick want to receive all messages, we don't need + # to send the message here + if call_id not in follow_all.keys() \ + or not follow_all[call_id] \ + or target == self.bot.config_bridge['channel']: + print('Send: %s to %s' % (msg_to_send, call_id)) + self.bot.controler.sendTextMessage(call_id, + msg_to_send) + else: + self.bot.privmsg(self.bot.config_bridge['channel'], + 'I don\'t know how to contact this person') + # Send message to everyone who wants to receive message from #channel + if target.lower() != self.bot.config_bridge['channel'].lower(): + return + msg_to_send = ('%s: %s' % (mask.nick, data)) + for user in follow_all.keys(): + if follow_all[user]: + self.bot.controler.sendTextMessage(user, + msg_to_send) + + @irc3.extend + def send_message_to_irc(self, ring_id, message): + '''send message to #channel''' + self.bot.privmsg(self.bot.config_bridge['channel'], + '%s: %s' % + (self.bot.config_bridge['users'][ring_id], message), + True) + + +def threadClient(controler, config_bridge): + config = dict( + nick=config_bridge['nick'], + autojoins=[config_bridge['channel']], + host=config_bridge['host'], + port=config_bridge['port'], + ssl=config_bridge['ssl'], + debug=config_bridge['debug'], + includes=[__name__] + ) + bot = irc3.IrcBot.from_config(config) + # link irc and ring controlers + bot.controler = controler + bot.config_bridge = config_bridge + controler.irc_client = bot + + bot.run(forever=True) + + +class IRCRingController(DRingCtrl): + def __init__(self, config_bridge): + super().__init__('ircbridge', True) + self.irc_client = None + self.config_bridge = config_bridge + self.accountId = self.configurationmanager.getAccountList()[0] + + def updateConfig(self): + '''Change actual config''' + with open('config.json', 'w') as config_file: + json.dump(self.config_bridge, config_file, indent=4) + if self.irc_client: + self.irc_client.config_bridge = self.config_bridge + + def onIncomingAccountMessage(self, accountId, fromAccount, payloads): + '''React to message from Ring''' + # Avoid to react to message from self. + if fromAccount == self.accountId: + return + # If we have multiple accounts, avoid to react from another account + if accountId != self.accountId: + return + message = '%s: %s' % (fromAccount, payloads['text/plain']) + print('Receive new message from %s' % message) + # React to !commands + if payloads['text/plain'] == '!follow': + self.config_bridge['follow_all'][fromAccount] = True + self.updateConfig() + elif payloads['text/plain'] == '!unfollow': + self.config_bridge['follow_all'][fromAccount] = False + self.updateConfig() + elif payloads['text/plain'].split(' ')[0] == '!add': + irc_nick = payloads['text/plain'].split(' ')[1] + self.config_bridge['users'][fromAccount] = irc_nick + self.updateConfig() + elif payloads['text/plain'] == '!rm': + del self.config_bridge['users'][fromAccount] + del self.config_bridge['follow_all'][fromAccount] + self.updateConfig() + # Send message to IRC + else: + try: + if self.irc_client: + self.irc_client.send_message_to_irc(fromAccount, + payloads['text/plain']) + except: + print('Can\'t read message received: %s' % payloads) + + def sendTextMessage(self, accountId, message): + '''Send a message to a ring id''' + if accountId == self.accountId: + return + self.configurationmanager.sendTextMessage(self.accountId, + accountId, + { + 'text/plain': + str(message) + }) + + +if __name__ == '__main__': + config_bridge = None + with open('config.json') as config_file: + config_bridge = json.loads(config_file.read()) + if not config_bridge: + print('Can\'t find config.json') + sys.exit(-1) + irc_controler = IRCRingController(config_bridge) + thread = Thread(target=threadClient, args=(irc_controler, config_bridge,)) + thread.start() + irc_controler.run() +``` + +## LRC + +### Doxygen doc + +The Doxygen documentation is available [here](https://jenkins.ring.cx/view/libringclient/job/libringclient-doxygen/doxygen/annotated.html) and currently generated by Jenkins each week. + +### Database schema + + + +```sql +CREATE TABLE profiles (id INTEGER PRIMARY KEY, \ + uri TEXT NOT NULL, \ + alias TEXT, \ + photo TEXT, \ + type TEXT, \ + status TEXT); + +CREATE TABLE conversations (id INTEGER,\ + participant_id INTEGER, \ + FOREIGN KEY(participant_id) REFERENCES profiles(id)); + +CREATE TABLE interactions (id INTEGER PRIMARY KEY,\ + account_id INTEGER, \ + author_id INTEGER, \ + conversation_id INTEGER, \ + timestamp INTEGER, \ + body TEXT, \ + type TEXT, \ + status TEXT, \ + daemon_id TEXT, \ + FOREIGN KEY(account_id) REFERENCES profiles(id), \ + FOREIGN KEY(author_id) REFERENCES profiles(id), \ + FOREIGN KEY(conversation_id) REFERENCES conversations(id)); + +CREATE TABLE profiles_accounts (profile_id INTEGER NOT NULL, \ + account_id TEXT NOT NULL, \ + is_account TEXT, \ + FOREIGN KEY(profile_id) REFERENCES profiles(id)); +``` diff --git a/technical/Choosing-CRF-value-for-encoder.md b/technical/Choosing-CRF-value-for-encoder.md @@ -0,0 +1,103 @@ +## Context +Bandwidth usage by the Jami application is not optimal for all types of connections. Indeed, in some cases, the user experience is not good (satellite connection, ...) despite the bandwidth management algorithm. + +## Observation +It is not necessary to aim for an optimal quality (CRF < 20) because beyond that, the visual perception is almost similar while the data flow (bitrate) required is much higher. + +## Objective +The purpose of this document is to verify the impact of a change in video quality with the CRF parameter of the encoder. + +## Test +These tests were performed by comparing : +- The first one encoded with Jami's current parameters +- The second encoded with a lower quality + +Each of these tests were performed for the following resolutions: 1080p, 720p and 436p. + +For each of these resolutions several bitrates have been used: +- 300 Kbit/s (Jami low value) +- 1.5 Mbit/s (Intermediate value) +- 6 Mbit/s (High value) + +The graphs show the evolution of the bitrate with the file being tested (resolution and specific set bitrate). + +A visual comparison (side by side) was made for each test. + +Thanks to this test we can estimate the bitrate that will be emitted in Jami according to the chosen parameters. We also have an overview of the visual quality. + +* * * + +1080p / 300 kbit/s / CRF28 + +1080p / 300 kbit/s / CRF38 + + +Visual comparison (CRF28 a gauche / CRF38 a droite) +https://nextcloud.savoirfairelinux.com/index.php/s/twzPmT3MRPjQ3b7 + +1080p / 1.5 Mbps / CRF22 + +1080p / 1.5 Mbit/s / CRF30 + + +Visual comparison (CRF22 left / CRF30 right) +https://nextcloud.savoirfairelinux.com/index.php/s/XK6Ef3rwBFriRXT + + 1080p / 6 Mbps / CRF17 + + 1080p / 6 Mbit/s / CRF23 + + +Visual comparison (CRF17 on the left / CRF23 on the right) +https://nextcloud.savoirfairelinux.com/index.php/s/RG3TksjQKtMorW5 + +* * * + +720p / 300 kbps / CRF28 + +720p / 300 kbit/s / CRF38 + +Visual comparison (CRF28 left / CRF38 right) +https://nextcloud.savoirfairelinux.com/index.php/s/Z4T6EzGqyyWTJ9f + +720p / 1.5 Mbps / CRF22 + +720p / 1.5 Mbit/s / CRF30 (Test with reduced CRF) + + +Visual comparison (CRF22 left / CRF30 right) +https://nextcloud.savoirfairelinux.com/index.php/s/H2Z5NQT3zJ9wzod + +720p / 6 Mbps / CRF17 + +720p / 6 Mbit/s / CRF23 + + +Visual comparison (CRF17 left / CRF23 right) +https://nextcloud.savoirfairelinux.com/index.php/s/gACSARfHjAKqgRR + +* * * + +436p / 300 kbps / CRF28 + +436p / 300 kbit/s / CRF38 + + +Visual comparison (CRF28 left / CRF38 right) +https://nextcloud.savoirfairelinux.com/index.php/s/gj9CpQox5fiaWMT + +436p / 1.5 Mbps / CRF22 + +436p / 1.5 Mbit/s / CRF30 + + +Visual comparison (CRF22 left / CRF30 right) +https://nextcloud.savoirfairelinux.com/index.php/s/BeCaqs2rMRpNQwS + +436p / 6 Mbps / CRF17 + +436p / 6 Mbit/s / CRF23 + + +Visual comparison (CRF17 left / CRF23 right) +https://nextcloud.savoirfairelinux.com/index.php/s/KLeXq9NeLKTfyqs+ \ No newline at end of file diff --git a/technical/HDMI-CEC-(fr).md b/technical/HDMI-CEC-(fr).md @@ -0,0 +1,25 @@ +Au 1er Mars 2018: + +Gérer le HDMI-CEC +----------------- + +- CEC est une norme encore sous utilisée et chaque constructeur en + fait sa propre version + - max compat : <http://libcec.pulse-eight.com/Vendor/Support> +- Framework android natif très élémentaire (api) et pas encore + implémenté par les constructeurs +- Libcec est un projet de Kodi qui permet le contrôle en cec + - les kernels android ne sont pas compatibles + - nécessite un accès root/adb + - même kodi ne fonctionne pas en cec, libcec a été retiré du build + android + - <http://kodi.wiki/view/NVIDIA_SHIELD_TV#Known_issues_and_limitations> +- Il existe des projets de driver permettant d'utiliser le cec sur + android, mais ils nécessitent tous de flasher l'appareil +- Il est possible d'utiliser un dongle pour envoyer les commandes cec + en usb puis usb -> hdmi + - coûte 45 usd sans garantie de fonctionnement + - <https://stackoverflow.com/questions/45639210/use-libcec-usb-dongle-in-android-app> + - <https://www.pulse-eight.com/p/104/usb-hdmi-cec-adapter> +- La notion de user space driver pourrait nous servir pour écrire + notre propre pilote, mais elle n'existe pas sur AndroidTV+ \ No newline at end of file diff --git a/technical/LRC-documentation.md b/technical/LRC-documentation.md @@ -0,0 +1,78 @@ +*This page is a stub.* + +### Introduction + +- Lrc (Libringclient) is an interface between the clients and + the daemon. It ensures to get the same behaviour for all client + using it. + + +note: red = missing feature. + +### DatabaseManager + +- this class is an interface between lrc and the sqlite database. This + class should not be called directly from the client. + + + +### NewCallModel class and NewCall namespace + +- NewCallModel is an interface used to manage the calls. + +<!-- --> + +- When we need information about calls, members functions from the + model can return NewCall::Info. + +<!-- --> + +- When we need to perform some operation on a call, we pass it callId + to the delegate performing the operation. + +<!-- --> + +- note about the name : we are using New as prefix to avoid conflict + with the current CallModel and Call objects. + + + +### ContactModel class and Contact namespace + +- ContactModel is an interface to manage the contacts. + +<!-- --> + +- When we need information about contact(s), members functions from + the model can return Contact::Info + +<!-- --> + +- When we need to perform some operation on a contact, we pass it uri + to the delegate performing the operation. + + + +### ConversationModel class, Conversation namespace and Message namespace + +- ConversationModel is an interface used to manage conversations + and messages. + +<!-- --> + +- When we need information about some conversation(s), members + functions from the model can return Conversation::Info. + Conversation::Info contains Messages. + +<!-- --> + +- note about Message : so far, we didn't need a MessageModel, but this + could be come soon. + + + +### resources + +[Diagram1.dia](/uploads/f6d0a81d67b075c13c57b7d9771ca065/Diagram1.dia) + +[Diagram2.dia](/uploads/00b3212a7642e58fd35b74c48e712332/Diagram2.dia)+ \ No newline at end of file diff --git a/technical/advanced-features/create-a-plugin.md b/technical/advanced-features/create-a-plugin.md @@ -0,0 +1,484 @@ +# Creating Plugins + +**NOTE: this page introduces the Jami Plugins SDK.** + +As from September of 2020, Jami team has added plugins as a call feature for GNU/Linux, Windows, and Android users. +This meaning that now you can personalize your call and chat experience by using one of our available plugins. +But that is not all, you can also transform your awesome ideas into a brand new plugin! + +Here you will be guided throught the SDK that will help you start your plugin developpment. +The text is organized as: +* A description of our [#SDK](#sdk); +* An example of how to create your own base plugin with our SDK - [#Create my first plugin](#create-my-first-plugin). + +## SDK +We developped a Plugin System for Jami and we have a few plugins available to be used. +However as an open source project, we now desire users to be able to create, use, and distribute their own plugins. +To achieve that goal, we also developped a Jami Plugins SDK. +This kit is fully writen in python, and can be invoked running [pluginMainSDK.py]('') from `<plugins>` folder. +To get started you must: + +``` bash +mkdir jami-plugins && cd jami-plugins +git clone https://review.jami.net/ring-daemon daemon +git clone https://review.jami.net/jami-plugins plugins +cd plugins +pip3 install -r SDK/requirements.txt +python3 SDK/pluginMainSDK.py +``` + +You will notice that this script will generate a Jami Plugins SDK shell that allows users to: + +* [Create full plugin skeleton](#create-full-plugin-skeleton); +* [Create, modify or delete a manifest.json](#create-modify-or-delete-manifest-json); +* [Create a package.json](#create-a-package-json); +* [Create or delete a preference](#create-or-delete-a-preference); +* [Create functionality](#create-functionality); +* [Create main](#create-main); +* [Assemble files](#assemble-files); +* [Build](#build); +* [Merge jpls](#merge-jpls). + +Each one of these functionalities will be detailled next. +We also will explain the importance of the files it generates and any related SDK limitations. + +### Create full plugin skeleton +This option performs a sequence of actions to properly create all base files needed to start a plugin development. The steps are: + +1. gather authorship information; +2. define a plugin name; +3. create manifest; +4. create functionalities and preferences; +5. create main file; +6. define basic package.json; +7. define basic build files (build.sh and CMakeLists.txt). + +If all is completed successfully, the plugin may be build, installed, and loaded. Also the functionallities may be toggled, however, since their data processes are not implemented, they will perform no action. In our `HelloWorld` plugin, we implement a simple process using **OpenCV**. For more complex options, you can refer to our available plugins at [jami-plugins](https://git.jami.net/savoirfairelinux/jami-plugins). Feel free to implement any ideas you may have, but you should respect those constrains: +* use ffmpeg, opencv, and onnx from ring-daemon project; +* if using tensorflow, choose version 2.1.0. Why? + * We have all needed header files of tensorflow 2.1.0 in <jami-plugins>/contrib/libs.tar.gz; + * We provide docker images with libraries for Tensorflow C++ API and Tensorflow + Lite, for both Android and Linux development. +* if you need other libraries, check if we support it's build with ring-daemon project, + otherwise, you will have to build it and ensure correct link to the plugin libraries. + +To fully create a basic plugin with pre-implementation of desired functionalities APIs, preferences, package, manifest, main file, and basic build related files, from inside Jami Plugins SDK shell, the user must call: +``` +(Jami Plugins SDK) plugin +``` + +The SDK will ask other informations needed to correctly accomplish the task. + +### Create, modify or delete manifest.json +Every plugin must have a manifest. This file contains the official name, a description, and carries the plugin build version as in the example bellow. Without it, the plugin system will not be able to find the plugin library it should load. Due to it's importance, every time Jami Plugin SDK is told to create files to a non existing plugin or to a plugin with no manifest, it will first create a new manifest. + +``` bash +{ + "name": "HelloWorld", + "description" : "HelloWorld plugin will guide you throught Jami Plugins SDK use!", + "version" : "1.0.0" +} +``` + +To create/modify (or to delete) a manifest.json, from inside Jami Plugins SDK shell, the user must call: +``` +(Jami Plugins SDK) manifest (-del) +``` + +The SDK will ask other informations needed to correctly accomplish the task. + +### Create a package.json +Jami currently supports plugins for GNU/Linux, Android and Windows. For the latter, the build system used is the same as for Jami, it is, we call the plugin build using a python script. This script will look for a `package.json` file to aquire build informations and commands. Without this file our build pipeline for Windows will not work. + +An example package.json file is shown bellow. +* `name` and `version` in the package.json and manifest.json files should match. +* `extractLibs` indicates to the build system if the files under `<jami-plugins>/contrib/libs.tar.gz` will be used. This archive contains header files for **Tensorflow**. Thus, you only need to set `extractLibs` to true if you plan to use this library. +* To play with audio or video, the plugin is dependent of **ffmpeg**. By adding it to `deps`, the build system will automatically compile this library from `<ring-daemon>/contrib` if needed. We also provide **OpenCV** build from inside `<ring-daemon>/contrib` ! +If you want to use Tensorflow, we provide built libraries for GNU/Linux and Android with our docker images [here](https://hub.docker.com/repository/docker/sflagsantos/tensorflow-cuda) and [here](https://hub.docker.com/repository/docker/sflagsantos/tensorflowlite). +For more information about OpenCV and Tensorflow build, please refer to [jami-plugins](7.-Jami-plugins.md) technical documentation. There we have a step-by-step! +* If you're using cmake, your can set configuration definition in `defines` property. Exemple: if your configuration line is of the form `cmake -DCPU=TRUE ..` you may set `"defines": ["CPU=TRUE"]`. +* Any command directly related to the plugin build can be defined inside `custom_scripts`. Bellow we create the build folder to with the plugins project will be configured with `mkdir msvc` and we also set the build command as `cmake --build ./msvc --config Release`. Our example CMakeLists.txt may have pre and post build instruction that are not listed here. + +``` bash +{ + "name": "Hello World", + "version": "1.0.0", + "extractLibs": false, + "deps": [ + "ffmpeg" + ], + "defines": [], + "custom_scripts": { + "pre_build": [ + "mkdir msvc" + ], + "build": [ + "cmake --build ./msvc --config Release" + ], + "post_build": [] + } +} +``` + +To create a package.json, from inside Jami Plugins SDK shell, the user must call: +``` +(Jami Plugins SDK) package +``` + +The SDK will ask other informations needed to correctly accomplish the task. After the base package.json creation, the user must add or modify any information not previewed by the SDK process. + +### Create or delete a preference +A preference is a internal variable that will be used upon loading or while running the plugin. +There is limited types of preferences supported by the plugin system and each of them must contain generic and specific informations. Those informations must be placed under a certain structure that will form one preference and each preference must be listed inside a `preference.json` file. + +The generic properties of a preference are those that must be set by any type of preference: `category`, `type`, `key`, `title`, `summary`, `defaultValue`, and `scope`. The specific ones are linked to the type of the preference constructed. +For `EDITTEXT` preferences there is no other property to be set. +For `PATH` preferences we have: `mimeType`. +For `LIST` preferences we have: `entries` and `entryValues`. +A `LIST` preference must have a list of possible values to be used and a list of 'names' for these values. For example: +If you have two `entriesValues`: '0' and '1', these values may not be understandable by the user. Jami's UI will take the values from `entries` to be shown as 'names' for each one of these `entryValues`. +Then you can call '0' and '1' as 'sent' and 'received'. The UI will show these names that are more user friendly! +It is important to note that `entries` and `entryValues` must have the same number of items. + +Another important point to be noted for the preferences is that their values could be modified during runtime if, and only if, two conditions are satisfied: + +1. the code that applies the new value is within your functionality implementation and; +2. this functionality is listed in the preference's `scope` property. + +To better explain, we have to detail what is a scope and how a preference changement is implemented inside the plugin. + +* **Scope**: A scope is a list of functionalities that can modify a preference value even if the functionality is under use. It is, imagine you have a functionality called "Circle" that prints a colored circle to your video. Consider also that the color of that circle is set by a preference and that this preference lists "Circle" as one of it's scopes. In that scenario "Circle" will be able to modify the default circle color to another one. +* **Code implementation**: Continuing our example above, "Circle" also is the implementation of an abstract API class from Daemon (for more details check [#Create functionality](#create-functionality). When a user changes a preference value, the plugin system will call the `setPreferenceAttribute` implementation for each of the functionalities listed by the preference's scope. By it's turn, this function will match the preference unique `key` and call an internal function to apply the new value. + +For a pratical example, you can check the 'Foreground Segmentation' functionality of the GreenScreen plugin - [pluginMediaHandler.cpp](https://git.jami.net/savoirfairelinux/jami-plugins/blob/master/GreenScreen/pluginMediaHandler.cpp) and [preferences-onnx.json](https://git.jami.net/savoirfairelinux/jami-plugins/blob/master/GreenScreen/preferences-onnx.json). This plugin has both `LIST` and `PATH` preferences and also has one preference that can be modified during runtime. Can you tell wich one? + +To create (or delete) a preference, from inside Jami Plugins SDK shell, the user must call: +``` +(Jami Plugins SDK) preference (-del) +``` + +The SDK will ask other informations needed to correctly accomplish the task. If a preference is created from outside a functionality creation pipeline, any API implementation will not be changed in order to avoid overwrittings. Thus, if you want to allow values changement during runtime, you may need to manually modify your functionality implementation to fit running time changements conditions. + +### Create functionality +A Jami plugin may wrap one or multiple functionalities. It is, the same plugins may have a functionality to change background and to draw a circle to a video for example. +Each functionality must implement an abstract API class defined by Jami plugins System. That way, we can say that one functionality is one implementation of an API. +Currently we have the `MediaHandler` which allows access to audio or video frames and we have `ChatHandler` which allows acces to messages exchanged in a conversation. + +When defining a new a functionality, the SDK will create basic header and cpp files for you work on. +However, your new functionality can not be added to the scopes of a previously existing preference. For more information, please refer to [Create or delete a preference](#create-or-delete-a-preference). + +To create a functionality, from inside Jami Plugins SDK shell, the user must call: +``` +(Jami Plugins SDK) functionality +``` + +The SDK will ask other informations needed to correctly accomplish the task. + +### Create main +This option create plugin's `main.cpp`. A file that implements the plugin external loading function that Jami Plugin System +will call to initialize and register all functionalities for latter use. + +The SDK is set to rewrite the main file every time you create a new functionality. +Thus, if you want to manually create or delete a functionality, we recomend calling this option instead. + +To create a `main.cpp`, from inside Jami Plugins SDK shell, the user must call: +``` +(Jami Plugins SDK) main +``` + +The SDK will ask other informations needed to correctly accomplish the task. + +### Assemble files +The final plugin file is an archive compressed to a `JPL` format. This archive contains libraries, manifest.json, preferences.json, icons and other custom important files for your plugin. +OBS.: files added by the developper must be organized under the `data/` folder. + +The SDK assemble option has two different behaviours: + +* it creates a build folder and copies there all files that will be compressed to the final plugin archive. For linux host: `<plugins>/<HelloWorld>/build-local/jpl/` and for a windows host: `<plugins>/<HelloWorld>/msvc/jpl/`; +* it compresses all files inside the build folder to the jpl archive wich is output under `<plugins>/build`. + +Both process should be called from inside the CMakeLists.txt as POST_BUILD and PRE_BUILD commands, respectively. Also, the build.sh script should call them. For more information about CMakeLists.txt and build.sh files, please refere to [build](#build) and to our available plugins at [jami-plugins](https://git.jami.net/savoirfairelinux/jami-plugins). + + +To create a build folder and copy all important files there, from inside Jami Plugins SDK shell, the user must call: +``` +(Jami Plugins SDK) assemble -pre +``` + +To compress all assembled to a jpl archive, from inside Jami Plugins SDK shell, the user must call: +``` +(Jami Plugins SDK) assemble +``` + +The SDK will ask other informations needed to correctly accomplish the task. + +### Build +The SDK build option has two different behaviours: + +* it creates basic CMakeLists.txt and buils.sh files; +* it build the plugin library with default options defined at the files mentioned above. + +A description of thes CMakeLists.txt and buils.sh files are found further in this section. + +To create basic CMakeLists.txt and buils.sh files, from inside Jami Plugins SDK shell, the user must call: +``` +(Jami Plugins SDK) build -def +``` + +To build plugin library with default configuration, from inside Jami Plugins SDK shell, the user must call: +``` +(Jami Plugins SDK) build +``` + +The SDK will ask other informations needed to correctly accomplish the task. For the moment, the build option does not support cross-compilation neither non default builds. If you have build variables to be set differently than the default option, please directly use the `<plugins>/build-plugin.py` script. + +#### CMakeLists.txt +This file is used only by Jami's Windows build pipeline. + +If you need to pass any defines to cmake generation, your definitions can be in `package.json` as explained in the package section. The package.json created specifies the default configuration that calling the build from Jami Plugins SDK will consider. +If you want to build with no default configuration, you can: directly use the script mentioned above and pass non-default definitions as an argument or; you also can manually change your package.json file. For examples, you can refer to our available plugins at [jami-plugins](https://git.jami.net/savoirfairelinux/jami-plugins). + +Another important information about CMakeLists.txt is that it has to add custom commands. For PRE_BUILD, it must call the pre-assemble functionality process. For POST_BUILD, it must copy the built library to the build folder and call the assemble functionality process. In the end, your jpl archive may be found under `<plugins>/build`. The CMakeLists.txt file automatically created by our SDK, already respects these constraints. + +#### build.sh +This file is used by Jami to build plugins for GNU/Linux and Android platforms. + +The basic script consider the environment variable `DAEMON` that must point to the ring-daemon folder. Besides, you can pass an argument for the platform used like `-t android` if you want to cross-compile for Android. Further custom definitions and environment variables should be handled by the plugin developper. +If you want to build with no default configuration, you can modify the environment variables values and then call the build. +Ie: for android, you can set which ABI you want to build with `export ANDROID_ABI="arm64-v8a armeabi-v7a`. +For other examples, you can refer to our [technical documentation](https://git.jami.net/savoirfairelinux/ring-project/wikis/technical/7.-Jami-plugins) and to our [available plugins](https://git.jami.net/savoirfairelinux/jami-plugins). + +Another important information about build.sh is that it has to call pre and post assemble. Before the build, it must call the pre-assemble functionality process. After it, it must copy the built library to the build folder and call the assemble functionality process. In the end, your jpl archive may be found under `<plugins>/build`. The build.sh file automatically created by our SDK, already respects these constraints. + +### Merge jpls +If you have more than one jpl archives, like one build for Android and anothe for GNU/Linux platforms, you can merge them into one to easy it's distribution. However, you should know that merging two or more jpls may inccur orverwritting some of the files inside them if they are not equal for all archives. The only files that may not present conflicting contents are the ones that do not repeate themselves. If conflicts occur, files from the first jpl in the arguments will prevail over the others. + +To merge two or more jpls, from inside Jami Plugins SDK shell, the user must simple call: +``` +(Jami Plugins SDK) merge +``` + +The SDK will ask other informations needed to correctly accomplish the task. + +## Create my first plugin +Through this section we will present a step-by-step construction of a `HelloWorld` plugin using our SDK. +Our goal is to print a coloured circle in the midle of the video frames using **OpenCV**. +The color of that circle will be defined by a preference which will be changeable during runtime. +Also we can set a stream preferece to define if the plugin will modify the video sent or the one received, this time we don't want to allow a changement during runtime. +We can define a second functionality that will aways draw a circle in the right top corner, with the color defined by the same preference as the previous functionality but that cannot be changed during runtime. +At the end we will exemplify how to build your plugin with and without the SDK. + +### Step 1 - prepare developpment environment +The first step towards plugin development is to properly prepare the environment. + +``` bash +mkdir jami-plugins && cd jami-plugins +git clone https://review.jami.net/ring-daemon daemon +git clone https://review.jami.net/jami-plugins plugins +cd plugins +pip3 install -r SDK/requirements.txt +``` + +### Step 2 - create HelloWorld with one functionality + +* Use Jami Plugins SDK to create the plugin skeleton + +``` python +python3 SDK/pluginMainSDK.py +``` +> (Jami Plugins SDK) plugin + +> Tell us who you are? + +> What's your name? Aline Gondim Santos + +> What's your e-mail? aline.gondimsantos@savoirfairelinux.com + +> Leave Empty or Press 'q' to quit this option. + +> Now, you need to tell how you want yout plugin to be called. + +> Choose a cool name for your plugin: Hello World + +> Nice! Your HelloWorld will be awesome! + +> Defining a manifest for "HelloWorld" plugin, we gonna need more information.. + +> Press 'q' to quit this option. + +> Tell us a description: HelloWorld draws a circle in the center of a call's video + +> Version must be of the form X.Y.Z +Set plugin version (default 0.0.0): 1.0.0 + +> Chose a functionality name: CenterCircle + +> Choose a API for functionality "CenterCircle". + +> Available APIs: +(1) video during a call (Media Handler API) +(2) audio during a call (Media Handler API) +(3) chat messages (Chat Handler API) +For more information about the API, call help preferences. + +> Enter a data type number: 1 + +> Add another functionaliy? [y/N] + +> Would you like to add a preference? [y/n] y + +> Your preferences options available are: +(0) List; +(1) Path; +(2) EditText; + +> Which preference type do you choose: 0 +Type a value for category: stream +Type a value for key: videostream +Type a value for title: Video stream +Type a value for summary: select a video direction +Type a value for defaultValue: 0 + +> Would you like to add a scope? [y/n] n + +> Do you want to add a new entry Value? [y/n] y +Type one new entry: 0 + +> Do you want to add a new entry Value? [y/n] y +Type one new entry: 1 + +> Do you want to add a new entry Value? [y/n] n +Type an entry name for '0': sent +Type an entry name for '1': received + +> Would you like to add a preference? [y/n] y + +> Your preferences options available are: +(0) List; +(1) Path; +(3) EditText; + +> Which preference type do you choose: 0 +Type a value for category: color +Type a value for key: color +Type a value for title: Circle color +Type a value for summary: select a color +Type a value for defaultValue: ##00FF00 + +> Would you like to add a scope? [y/n] y + +> Possible values for scope: +(0) centerCircle; + +> Which scope do you choose: 0 + +> Do you want to add a new entry Value? [y/n] y +Type one new entry: #0000FF + +> Do you want to add a new entry Value? [y/n] y +Type one new entry: #00FF00 + +> Do you want to add a new entry Value? [y/n] y +Type one new entry: #FF0000 + +> Do you want to add a new entry Value? [y/n] n +Type an entry name for '#0000FF': blue +Type an entry name for '#00FF00': green +Type an entry name for '#FF0000': red + +> Would you like to add a preference? [y/n] n + +> The preference Circle color will be changeable during running time +for centerCircle functionality? [y/n] y + +> Package ok. + +> CMakeLists.txt and build.sh ok. + + +* modify CenterCircleMediaHandler.cpp, CenterCircleVideoSubscriver.h, CenterCircleVideoSubscriber.cpp; +* You will need to modify package.json, CMakeLists.txt and build.sh to add OpenCV dependencies. + +All final files can be found [here](https://git.jami.net/savoirfairelinux/jami-plugins) + + +### Step 3 - build +Before building the HelloWorld, you should compile ffmpeg and OpenCV dependencies. This can be achieved by following the build instructions for OpenCV [here](7.-Jami-plugins). Ffmpeg will be automatically build if you follow those instructions. +To build you plugin, you can either: +* Call the build from Jami Plugins SDK (works for GNU/Linux and Windows): + +``` python +python3 SDK/pluginMainSDK.py +``` +> (Jami Plugins SDK) build + +> Leave Empty or Press 'q' to quit this option. + +> Plugin to pass build related step: HelloWorld + +> DAEMON not provided, building with ./../../daemon + +* Call plugin-build.py script (works for GNU/Linux, Windows and Android): + * GNU/Linux or Windows: +> python3 build-plugin.py --projects=HelloWorld + + * Android: +> python3 build-plugin.py --projects=HelloWorld --distribution=android + +OBS: For Android, you can set `ANDROID_ABI` environment variable to the ABI you want to build. Currently Jami supports `x86_64`, `armeabi-v7a`, and `arm64-v8a`. Plugins will be build for the three options by default. + +### Step 4 - create another functionality for HelloWorld and rebuild +Now that you tested and your HelloWorld is working, you can try to do add another functionality to it. + +``` python +python3 SDK/pluginMainSDK.py +``` +> (Jami Plugins SDK) functionality + +> Tell us who you are? + +> What's your name? Aline Gondim Santos + +> What's your e-mail? aline.gondimsantos@savoirfairelinux.com + +> Leave Empty or Press 'q' to quit this option. + +> Plugin to be modified or created: HelloWorld + +> Chose a functionality name: CoinCircle + +> Choose a API for functionality "CoinCircle". + +> Available APIs: +(1) video during a call (Media Handler API) +(2) audio during a call (Media Handler API) +(3) chat messages (Chat Handler API) +For more information about the API, call help preferences. + +> Enter a data type number: 1 + +> Add another functionaliy? [y/N] n + +> Would you like to add a preference? [y/n] n + +> CMakeLists.txt and build.sh ok. + +Again, all code modifications are available [here](https://git.jami.net/savoirfairelinux/jami-plugins) + +Optionally, you can change your plugin version and description: + +``` python +python3 SDK/pluginMainSDK.py +``` +> (Jami Plugins SDK) manifest +Leave Empty or Press 'q' to quit this option. + +> Plugin to be modified or created: HelloWorld +New description for your plugin (ignore to keep previous description): HelloWorld can draw a circle at the center or at the top-left of a call's video +New plugin version (ignore to keep previous version): 2.0 + +Finally, you can rebuild the plugin using the same steps as before! + + +Now you can start your own creations! +Do not forget to tag Jami or Savoir-Faire Linux in your repository, this way we can get to know how the community is developping their own plugins! diff --git a/technical/advanced-features/index.rst b/technical/advanced-features/index.rst @@ -0,0 +1,10 @@ +########## +Advanced Features +########## + +.. toctree:: + :maxdepth: 1 + + plugins + create-a-plugin + tensorflow-plugin diff --git a/technical/advanced-features/plugins.md b/technical/advanced-features/plugins.md @@ -0,0 +1,297 @@ +# Installing and Using Plugins + +**NOTE: this page describes what is a Jami Plugin and how to install and use them.** + +As from September of 2020, Jami team has added plugins as a call and chat feature for Linux, Windows, and Android users. +This meaning that now you can personalize your call/chat experience by using one of our available plugins. +But that is not all, you can also transform your awesome ideas into a brand new plugin! + +* To properly setup a plugin you must follow the steps in [#How to use it?](#how-to-use-it). +* To build a available plugin, please refer to [#How to build?](#how-to-build) instructions. +* To create your own plugin, please refer to [Create Plugin](7.1 - Create Plugin) instructions. + +## How it works? +Jami can be break down to three main components that interact together: Daemon, LRC and clients. +Daemon is the core of Jami, and although it does not interact with users, it is involved in every +command. Therefore, Daemon has a `JamiPluginManager` class that among other actions perfoms install/uninstall, load/unload, edit preferences and control plugins' usage. +Despite Daemon importance, what a plugin effectivelly does to a call video/audio or to a chat message is unknown to it the same way Daemon does not know what is effectivelly done by LRC or the clients interfaces. +Plugins then can be seen as a forth interacting component in Jami. + +The plugin system inside Jami exposes different APIs that can be used by the plugins. For instance, the ChatHandler and the Media Handler API. The latter enables the plugins to modify audio and video streams from Jami calls and is used by our GreenScreen plugin but could also be used to build a YouTube streaming system, various instagram-style filters, a real time translation service, etc. + +Plugins can be composed by one or multiple media and chat handlers that are responsible for attaching/detaching a data stream from Jami and a data process. Each handler represents a functionality that can be totally different between them or can be a modified versions of the same core process. In our example, we have only one functionality, it being, the GreenScreen plugin has one media handler which data process is responsible for segmenting the foreground from a video frame and applying another image to the background, just like it is done with the green screens in movies! + +To use one custom functionality, it is necessary that Jami knows all plugins' handlers, which one is going to be used and the data that should be processed. Plugin's handlers are created once a plugin is loaded and are shared with Daemon's Plugin Manager. The data is inside Jami flow (for a call plugin, in the event of a new call, Jami creates and stores the corresponding media stream subjects). Finally, once a user puts a plugin functionality in action Jami tells this handler to attach the available data. When deactivated, Jami tells the handler to dettach. + +## How to use it? + +### Setup +A Jami plugin is a `pluginname.jpl` file, and it must be installed to your Jami. +Once installed, Jami will add your new plugin to the available plugins list but they will not be ready for use yet. Plugins are libraries and must be loaded if you want to expose them. +Moreover, plugin may have preferences and besides install/uninstall and load/unload actions, it is possible to modify those preferences. For example, our GreenScreen plugin allows the user to change the background image displayed. + +#### Android +To setup a plugin for Android you must go under Setting, enable plugins if they are disabled, and select a plugin file from your phone. +After installed it is automaticaly loaded. +Optionally, you can manually perform load/unload using the checkbox button on the plugin list. + + + +For Android uninstall, you must click on the plugin and a uninstall option will appear allong with the preferences and a reset preferences option. +In order to a preference modification can take effect the plugin has to be reloaded. + + + +#### Linux/Windows +Similarly, for the client-qt available on Linux and Windows and for the client-gnome available only on Linux, you must go to Prefences, enable plugins if it is disabled, and select a plugins file from your computer. +Each plugin in the shown list is linked to two buttons beeing: + +- Client-qt: a load/unload button and a preferences button; +- Client-gnome: a load/unload button and a uninstall button; +For client-gnome it is not possible to change plugin's preferences. + + + + +### Use! +A media handler functionality only takes place if you turn them on during a call. +For either Android, Linux or Windows you can do so by clicking on the plugins icon on your call screen. + + + + +Similarly, for chat handler functionality, you will see a plugin icon in the chat window as in the images bellow. + +## How to build? +If you want to make something with your video call, it is possible that you will do so with OpenCV and/or deep learning models (Tensorflow, PyTorch, etc). +So, before going to the plugin, it is necessary to build plugin's dependencies. + +### Dependencies +Here we give you the steps to build [OpenCV](https://opencv.org/) and [ONNX](https://www.onnxruntime.ai/) but do not feel limited to these libraries. We offer a [page](7.2 - Tensorflow Plugin) with detailled explanation of how to build tensorflow C++ API for Windows, Linux and Android. +Other libraries should work as long they and the plugin are correctly built! + +#### OpenCV 4.1.1 +We kindly added OpenCV 4.1.1 as a contrib in [daemon](https://git.jami.net/savoirfairelinux/ring-daemon/tree/master/contrib). +This way you can easily build OpenCV for Android, Linux, and Windows. You only have to follow the corresponding instructions. + +##### Windows +```bash +set DAEMON=<path/to/daemon> +cd ${DAEMON}/compat/msvc +python3 winmake.py -fb opencv +``` + +##### Linux +With Docker (recommended): +```bash +export DAEMON=<path/to/daemon> +cd ${DAEMON}/../ +docker build -f plugins/docker/Dockerfile_ubuntu_18.04_onnxruntime -t plugins-linux . +docker run --rm -it -v ${DAEMON}/../:/home/plugins/jami:rw plugins-linux:latest /bin/bash +cd jami/plugins/contrib +cd ../../daemon/contrib +mkdir native +cd native +../bootstrap --disable-argon2 --disable-asio --disable-fmt --disable-gcrypt --disable-gmp --disable-gnutls --disable-gpg-error --disable-gsm --disable-http_parser --disable-iconv --disable-jack --disable-jsoncpp --disable-libarchive --disable-libressl --disable-msgpack --disable-natpmp --disable-nettle --enable-opencv --disable-opendht --disable-pjproject --disable-portaudio --disable-restinio --disable-secp256k1 --disable-speexdsp --disable-upnp --disable-uuid --disable-yaml-cpp --disable-zlib +make list +make fetch opencv opencv_contrib +make +``` + +Using your own system: +```bash +export DAEMON=<path/to/daemon> +cd ${DAEMON}/contrib/native +../bootstrap --enable-ffmpeg --disable-argon2 --disable-asio --disable-fmt --disable-gcrypt --disable-gmp --disable-gnutls --disable-gpg-error --disable-gsm --disable-http_parser --disable-iconv --disable-jack --disable-jsoncpp --disable-libarchive --disable-libressl --disable-msgpack --disable-natpmp --disable-nettle --enable-opencv --disable-opendht --disable-pjproject --disable-portaudio --disable-restinio --disable-secp256k1 --disable-speexdsp --disable-upnp --disable-uuid --disable-yaml-cpp --disable-zlib +make list +make fetch opencv opencv_contrib +make +``` + +##### Android +Using Docker (recommended): +```bash +export DAEMON=<path/to/daemon> +cd ${DAEMON}/../ +docker build -f plugins/docker/Dockerfile_android_onnxruntime -t plugins-android . +docker run --rm -it ${DAEMON}/:/home/gradle/src:rw plugins-android:latest /bin/bash +cd plugins/contrib +ANDROID_ABI="arm64-v8a" sh build-dependencies.sh +``` + +Using your own system: +```bash +export DAEMON=<path/to/daemon> +cd ${DAEMON} +export ANDROID_NDK=<NDK> +export ANDROID_ABI=arm64-v8a +export ANDROID_API=29 +export TOOLCHAIN=$ANDROID_NDK/toolchains/llvm/prebuilt/linux-x86_64 +export TARGET=aarch64-linux-android +export CC=$TOOLCHAIN/bin/$TARGET$ANDROID_API-clang +export CXX=$TOOLCHAIN/bin/$TARGET$ANDROID_API-clang++ +export AR=$TOOLCHAIN/bin/$TARGET-ar +export LD=$TOOLCHAIN/bin/$TARGET-ld +export RANLIB=$TOOLCHAIN/bin/$TARGET-ranlib +export STRIP=$TOOLCHAIN/bin/$TARGET-strip +export PATH=$PATH:$TOOLCHAIN/bin +cd contrib +mkdir native-${TARGET} +cd native-${TARGET} +../bootstrap --build=x86_64-pc-linux-gnu --host=$TARGET$ANDROID_API --enable-opencv --enable-opencv_contrib +make +``` + +#### Onnxruntime 1.6.0 +A difficulty for a lot of people working with deep learning models is how to deploy them. +With that in mind we provide the user the possibility of using onnxruntime. +There are several development libraries to train and test but, they are usually too heavy to deploy. Tensorflow with cuda support, for instance, can easily surpass 400MB. In our GreenScreen plugin We chose to use onnxruntime because it's lighter (library size of 140Mb for cuda support) and supports model convertion from several development libraries (Tensorflow, PyTorch, Caffe, etc.). + +* For more advanced and curious third-party developpers, we also [provide instructions](7.2 - Tensorflow Plugin) to build Tensorflow C++ API for Windows and Linux, and the TensorflowLite C++ API for Android. + +To build onnxruntime based plugins for Linux and Android, we strongly recommend using docker files available under `<plugins>/docker/`. We don't offer Windows docker, but here we carefully guide you through the proper build of this library for our three supported platforms. + +If you want to build onnxruntime with Nvidia GPU suport, be sure to have a CUDA capable GPU and that you have +followed all installation steps for the Nvidia drivers, CUDA Toolkit, CUDNN, and that their versions +match. + +The following links may be very helpfull: + +* https://developer.nvidia.com/cuda-gpus +* https://developer.nvidia.com/cuda-toolkit-archive +* https://developer.nvidia.com/cudnn + +#### Linux and Android +We kindly added onnxruntime as a contrib in [daemon](https://git.jami.net/savoirfairelinux/ring-daemon/tree/master/contrib). +This way you can easily build onnxruntime for Android, and Linux. + +* Linux - Without acceleration: + +```bash +export DAEMON=<path/to/daemon> +cd ${DAEMON}/contrib/native +../bootstrap +make .onnx +``` + +* Linux - With CUDA acceleration (CUDA 10.2): + +```bash +export CUDA_PATH=/usr/local/cuda/ +export CUDA_HOME=${CUDA_PATH} +export CUDNN_PATH=/usr/lib/x86_64-linux-gnu/ +export CUDNN_HOME=${CUDNN_PATH} +export CUDA_VERSION=10.2 +export USE_NVIDIA=True +export DAEMON=<path/to/daemon> +cd ${DAEMON}/contrib/native +../bootstrap +make .onnx +``` + +* Android - With NNAPI acceleration: + +```bash +export DAEMON=<path/to/daemon> +cd ${DAEMON} +export ANDROID_NDK=<NDK> +export ANDROID_ABI=arm64-v8a +export ANDROID_API=29 +export TOOLCHAIN=$ANDROID_NDK/toolchains/llvm/prebuilt/linux-x86_64 +export TARGET=aarch64-linux-android +export CC=$TOOLCHAIN/bin/$TARGET$ANDROID_API-clang +export CXX=$TOOLCHAIN/bin/$TARGET$ANDROID_API-clang++ +export AR=$TOOLCHAIN/bin/$TARGET-ar +export LD=$TOOLCHAIN/bin/$TARGET-ld +export RANLIB=$TOOLCHAIN/bin/$TARGET-ranlib +export STRIP=$TOOLCHAIN/bin/$TARGET-strip +export PATH=$PATH:$TOOLCHAIN/bin +cd contrib +mkdir native-${TARGET} +cd native-${TARGET} +../bootstrap --build=x86_64-pc-linux-gnu --host=$TARGET$ANDROID_API +make .onnx +``` + +#### Windows + +* Pre-build: + +```bash +mkdir pluginsEnv +export PLUGIN_ENV=<full-path/pluginsEnv> +cd pluginsEnv +mkdir onnxruntime +mkdir onnxruntime/cpu +mkdir onnxruntime/nvidia-gpu +mkdir onnxruntime/include +git clone https://github.com/microsoft/onnxruntime.git onnx +cd onnx +git checkout v1.6.0 && git checkout -b v1.6.0 +``` + +* Without acceleration: + +``` +.\build.bat --config Release --build_shared_lib --parallel --cmake_generator "Visual Studio 16 2019" +cp ./build/Windows/Release/Release/onnxruntime.dll ../onnxruntime/cpu/onnxruntime.dll +``` +* With CUDA acceleration (CUDA 10.2): + +``` +.\build.bat --config Release --build_shared_lib --parallel --cmake_generator "Visual Studio 16 2019" +--use_cuda --cudnn_home <cudnn home path> --cuda_home <cuda home path> --cuda_version 10.2 +cp ./build/Windows/Release/Release/onnxruntime.dll ../onnxruntime/nvidia-gpu/onnxruntime.dll +``` +* Post-build: + +```bash +cp -r ./include/onnxruntime/core/ ../onnxruntime/include/ +``` + +For further build instructions, please refer to onnxruntime official [GitHub](https://github.com/microsoft/onnxruntime/blob/master/BUILD.md). + +### Plugin +To exemplify a plugin build, we will use the GreenScreen plugin available [here](https://git.jami.net/savoirfairelinux/jami-plugins). + +#### Linux/Android +First you need to go to the plugins repository in your cloned ring-project: + +```bash +cd <ring-project>/plugins +``` + +- Linux - Nvidia GPU + +`PROCESSOR=NVIDIA python3 build-plugin.py --projects=GreenScreen` +- Linux - CPU + +`python3 build-plugin.py --projects=GreenScreen` +- Android + +```bash +export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-amd64/jre +export ANDROID_HOME=/home/${USER}/Android/Sdk +export ANDROID_SDK=${ANDROID_HOME} +export ANDROID_NDK=${ANDROID_HOME}/ndk/21.1.6352462 +export ANDROID_NDK_ROOT=${ANDROID_NDK} +export PATH=${PATH}:${ANDROID_HOME}/tools:${ANDROID_HOME}/platform-tools:${ANDROID_NDK}:${JAVA_HOME}/bin +ANDROID_ABI="arm64-v8a armeabi-v7a x86_64" python3 build-plugin.py --projects=GreenScreen --distribution=android +``` + +The GreenScreen.jpl file will be available under `<ring-project/plugins/build/>`. + +#### Windows +Windows build of plugins are linked with the daemon repository and its build scripts. So to build our example plugins you have to: + +```bash +cd <ring-project>/daemon/compat/msvc +python3 winmake.py -fb GreenScreen +``` + +The GreenScreen.jpl file will be available under `<ring-project/plugins/build/>`. + + +Related articles: + +- https://jami.net/plugins-sdk/ diff --git a/technical/advanced-features/tensorflow-plugin.md b/technical/advanced-features/tensorflow-plugin.md @@ -0,0 +1,104 @@ +# Build Tensorflow + +**NOTE: this page describes to build Tensorflow and TensorflowLite C++ API for Linux Android and Windows.** + +## Tensorflow 2.1.0 + +A difficulty for a lot of people working with tensorflow is how to properly build it. +With that in mind we created docker images with cuda and tensorflow libraries available for GNU/Linux builds [here](https://hub.docker.com/repository/docker/sflagsantos/tensorflow-cuda) and for Android builds [here](https://hub.docker.com/repository/docker/sflagsantos/tensorflowlite). These docker can be used to build plugins for Linux and Android, however they cannot handle Windows. +Here we carefully guide you through the proper build of tensorflow LITE Native and Tensorflow C++ API for our three supported platforms. + +You will need: + +* Python 3 +* Bazel 0.29.1 +* Tensorflow 2.1.0 repository: + + ```bash + git clone https://github.com/tensorflow/tensorflow.git + cd tensorflow + git checkout v2.1.0 + ``` +We assembled Tensorflow headers needed to build plugins, to access them you only have to extract `libs.tar.gz` file found under `jami-project/plugins/contrib`. However, if you are using another tensorflow version or you want to do it by yourself, you can follow the assemble instructions for Tensorflow LITE Native and C++ API are available under [jami-plugins](https://git.jami.net/savoirfairelinux/jami-plugins) README_ASSEMBLE file. + +#### Linux +Tensorflow LITE does not support desktops GPU. If you want to use them, please consider using C++ API + +If you want to build Tensorflow C++ API with GPU suport, be sure to have a CUDA capable GPU and that you have +followed all installation steps for the Nvidia drivers, CUDA Toolkit, CUDNN, Tensor RT, that their versions +matches and that they are correct for the Tensorflow version you want to build. + +The following links may be very helpfull: + +* https://www.tensorflow.org/install/source +* https://developer.nvidia.com/cuda-gpus +* https://developer.nvidia.com/cuda-toolkit-archive +* https://developer.nvidia.com/cudnn + +Setup your build options with `./configure`. + +* Tensorflow LITE Native + + ```bash + bazel build //tensorflow/lite:libtensorflowlite.so + ``` +* Tensorflow C++ API + + ```bash + bazel build --config=v1 --define framework_shared_object=false --define=no_tensorflow_py_deps=true //tensorflow:libtensorflow_cc.so + ``` + +#### Windows +Tensorflow LITE does not support desktops GPU. If you want to use them, please consider using C++ API + +If you want to build Tensorflow C++ API with GPU suport, be sure to have a CUDA capable GPU and that you have +followed all installation steps for the Nvidia drivers, CUDA Toolkit, CUDNN, Tensor RT, that their versions +matches and that they are correct for the Tensorflow version you want to build. + +The following links may be very helpfull: + +* https://www.tensorflow.org/install/source +* https://developer.nvidia.com/cuda-gpus +* https://developer.nvidia.com/cuda-toolkit-archive +* https://developer.nvidia.com/cudnn + +Setup your build options with `python3 configure.py`. + +* Tensorflow LITE Native + + ```bash + bazel build //tensorflow/lite:tensorflowlite.dll + ``` +* Tensorflow C++ API + + ```bash + bazel build --config=v1 --define framework_shared_object=false --config=cuda --define=no_tensorflow_py_deps=true //tensorflow:tensorflow_cc.dll + ``` +There may be some missign references while compilling a plugin with Tensorflow C++ API. If that happens you have to rebuild you tensorflow and explicitly export the missing symbols. Fortunatelly Tensorflow now has a easy workaround to do so, you only have to feed [this]("https://github.com/tensorflow/tensorflow/blob/v2.2.0/tensorflow/tools/def_file_filter/def_file_filter.py.tpl") file with the desired symbols. + +#### Android - Tensorflow LITE Native +For mobile applications Tensorflow LITE is the only option you want to consider and to succesfully build it you will also need: + +* Android NDK 18r + +Setup your build options with: + +```bash +./configure + >> Do you wish to build TensorFlow with XLA JIT support? [Y/n]: n + >> Do you wish to download a fresh release of clang? (Experimental) [y/N]: y + >> Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: y + >> Please specify the home path of the Android NDK to use. [Default is /home/<username>/Android/Sdk/ndk-bundle]: put the right path to ndk 18r +``` +And build as desired: + +* armeabi-v7a + + ```bash + bazel build //tensorflow/lite:libtensorflowlite.so --crosstool_top=//external:android/crosstool --cpu=armeabi-v7a --host_crosstool_top=@bazel_tools//tools/cpp:toolchain --cxxopt="-std=c++11" + ``` +* arm64-v8a + + ```bash + bazel build //tensorflow/lite:libtensorflowlite.so --crosstool_top=//external:android/crosstool --cpu=arm64-v8a --host_crosstool_top=@bazel_tools//tools/cpp:toolchain --cxxopt="-std=c++11" + ``` diff --git a/technical/basic-features/file-transfer.md b/technical/basic-features/file-transfer.md @@ -0,0 +1,200 @@ +# File Transfers + +## How to use it? + + +#### Gnome + +For sending a file on gnome, in a conversation you need to click to the +send file icon at the bottom right of the conversation: + + + +Then you will see your image as soon as the transfer is finished (and +show images is activated) + + + +On the contrary if you receive a file (and if it's not a picture < 20 +Mb), you will have to accept the transfer: + + + +And then the file will be sent. You have the possibility to cancel in a +middle of a transfer. + + + +#### Android + +When you are talking to somebody on Android, you have the possibility to +send a picture on your device or take a photo with these buttons: + + + +Note: when you send a file, the other has to accept it. At this moment +you will see 'awaiting peer': + + + +## How it works? (technical) + + +### How it works + +#### Introduction + +Jami is a distributed application and has to work without any internet connectivity. So, file transfer too! Basically, we use the same method to perform file transfer and calls, but in TCP. To summarize how it works, we can imagine a situation where Alice (A) wants to transfer a file to Bob (B). + +First, Alice will request a connection to Bob. To do that, Jami is using ICE (RFC 6544), a protocol used to negotiate links between peers. Alice will send, into an encrypted packet via the DHT the ip of its device. So, when Bob receives the ips of Alice, they will be able to negotiate a transport where Bob will be able to send packets to Alice. The negotiation can be successful, but if it fails, a TURN server will be used (the one configured into the settings) to perform the transfer. If the negotiation succeeds, Bob will send its ips to Alice to perform the negotiation in the other direction. Note that the link is still not secure, so Bob will send the ips through the DHT in an encrypted message. If the second negotiation fails, the TURN will be used as a fallback. + +Now that the bidirectionnal TCP link is here, the next step will be to negotiate a TLS 1.3 (generally a (TLS1.3)-(DHE-FFDHE8192)-(RSA-PSS-RSAE-SHA384)-(AES-256-GCM) when I write these lines) between Alice an Bob, then Alice will start to transfer the file. + +The first part will be a small header to describe the content of the file. Then, after Bob accepts the transfer, the full file will be transmitted. + +#### Process + +##### Sending a file + +The following method is used: + +1\. A client will call `DataTransferFacade::sendFile()`. `DataTransferFacade` is the class corresponding to the API exposed for the clients. It is used to manage a view of the file transfers (the corresponding classes are `DataTransfer`, `IncomingFileTransfer`, `OutgoingFileTransfer` and `SubOutgoingFileTransfer`). This method will ask the linked `JamiAccount` to request a connection. + + + +2\. The method `DhtPeerConnector: requestConnection()` is triggered and creates a connection between all connected devices of the peer (found on the DHT). `DhtPeerConnector` is used to manage the main event loop which manage connections. When a device is found, the *event loop* will create a `ClientConnector` (which manage the connection for one device) and launch the `process()` method. + +3\. This method is used to initialize the ICE transport and put a *PeerConnectionMsg* (which contains the SDP message, see below) on the DHT and waits for a response (`DhtPeerConnector::Impl::onResponseMsg`). + +4\. Then a response is received from the DHT, which contains public addresses of the peer device. We can now negotiate a TLS link (directly via ICE, or via TURN as a fallback). This `TlsSocketEndpoint` is given to the `PeerConnection` object as an output and the transfer can start. + +5.\ When the TLS socket is ready, the callback `DataTransferFacade::Impl::onConnectionRequestReply` is called, and a `OutgoingFileTransfer` is linked to the `PeerConnection` as an input. This `OutgoingFileTransfer` contains a list of `SubOutgoingFileTransfer` (one per device) where each sub transfer is a transfer to one device. We do that to be able to furnish the most optimistic view of the transfer (if a contact as 3 devices, where the contact cancel the transfer on one device, but accepted the transfer on the two others, the most advanced transfer will be shown). + +6\. The `SubOutgoingFileTransfer` will first transfer the header of the file, wait the peer acceptance (A "GO\n" message on the socket) and then will send the file. + +7\. If a cancel is received from the peer or the client or if the file transfer finish, the connection will be closed via a `CANCEL` message on the `DhtPeerConnector::eventLoop()` and the resources will be released. + + + +##### Receiving a file + + +The same structure is used to receive files, but the method changes a bit: + +1. The `JamiAccount` class is used to receives messages from the DHT, because the first thing received will be the DHT request. +2. Then, this message is given to `DhtPeerConnector: onRequestMessage()` through the eventLoop. +3. The `DhtPeerConnector::Impl::answerToRequest` will try to connect to the TURN server (if not connected) and initialize the ICE transport. This method open 2 control connections to a TURN server (one to authorize IPv4 peers, another one for IPv6 peers, due to **RFC 6156**) if it's not already open and permit Peer public addresses to connect. Then, if the SDP received doesn't contains ICE candidates, will use the TURN and craft the SDP answer to wait for the peer. If the SDP contains ICE candidates, the method will try to negotiate the link (or fallback on the TURN) and then answer the SDP (with ICE candidates or not). +4. Once the links are ready, like the sender, a TLS link is negotiated and given to the `PeerConnection` given to the `IncomingFileTransfer` as an input. The headers of the file will come and the client is now able to accept or cancel the transfer. + +##### Schema + + + +##### SDP sent over the DHT + +``` +0d04b932 +7c33834e7cf944bf0e367b47 +H6e6ca682 1 TCP 2130706431 2607:fad8:4:6:9eb6:d0ff:dead:c0de 50693 typ host tcptype passive +H6e6ca682 1 TCP 2130706431 2607:fad8:4:6:9eb6:d0ff:dead:c0de 9 typ host tcptype active +H42c1b577 1 TCP 2130706431 fe80::9eb6:d0ff:fee7:1412 50693 typ host tcptype passive +H42c1b577 1 TCP 2130706431 fe80::9eb6:d0ff:fee7:1412 9 typ host tcptype active +Hc0a8007e 1 TCP 2130706431 192.168.0.123 42751 typ host tcptype passive +Hc0a8007e 1 TCP 2130706431 192.168.0.123 9 typ host tcptype active +Sc0a8007e 1 TCP 1694498815 X.X.X.X 42751 typ srflx tcptype passive +Z.Z.Z.Z:YYYY +A.A.A.A:YYYY +``` + +Where `0d04b932` is the ufrag and `7c33834e7cf944bf0e367b47` the password of the ICE session. `2130706431` and `1694498815` are the priority of the candidates. `192.168.0.126 42751 typ host tcptype passive` is a passive host candidate and `1694498815 X.X.X.X 42751 typ srflx tcptype passive` a passive host reflecting the public ip (mapped via UPnP for example). + +#### PJSIP related patches. + +3 patches will be included into the PJSIP project: + +1. RFC 6062, used to perform TURN over TCP (merged upstream: pjproject - fa6616c43c7e19797084f4e02a67d1fb6fd99473) +2. RFC 6544, used to perform ICE over TCP +3. A fix for pj_activesock + +Note that the stack for the file transfer is: + +``` ++-----------+----------------+-----------------------+ ++ STUN + TLS data + TLS data + ++-----------+----------------+-----------------------+ ++ RFC 4571 (RTP) + RFC 6062 (TURN TCP) + ++----------------------------+-----------------------+ ++ TCP + ++----------------------------------------------------+ +``` + +### Multi devices + +A RING user can link its account to several devices. So, we need to +implement the transfer when a user send a file to a contact who have +multiple devices linked to this account. + +#### First approach + +The first approach was to send a request through the DHT to all devices +and the first devices which answers get the file to transfer. This is +bad for your contact because they will not know which device will +receives will get the transfer. + +#### Current approach + +Now, we still send a request to all devices. The difference is that all +devices will have the notification for receiving a file and can +accept/refuse the transfer. The major part of the code for that is in +*data\_transfer.cpp*. + +Now (since <https://gerrit-ring.savoirfairelinux.com/#/c/9327/>), when a +user send a file, it will request a *PeerConnection* with all peer +devices. And for all connections, we attach a new input stream to have +the ability to accept/refuse/cancel each transfer separately. + +In *data\_transfer.cpp* we define the *OptimisticMetaOutgoingInfo* class +which represent the optimistic view to show to the client. It's +optimistic because if a contact accept a transfer on one device and +refuse on others, this class will show the ongoing file transfer. And it +will only show an error if all devices refuse the transfer. + +This class is linked to *SubOutgoingFileTransfer* which represent the +state of a transfer with one device. Clients will have the ability to +show a sub transfer instead the optimistic later (see TODO list). + +### Using another TURN server + +Actually the default TURN server is *turn.ring.cx*. But you can host +your own TURN server. For example by running a +[coturn](https://github.com/coturn/coturn) server. + +`sudo turnserver -a -v -n -u user:password -r "realm"` + +Then, you can configure the TURN server in the advanced settings of +RING. + +Note: this needs some technical knowledges. Moreover, the TURN server +should see the same ip address of your node as the destination node or +the peer connection will fail (because the authorization will be +incorrect) + +## Future + + +For now, if a file transfer fails when ongoing, the sender can't resume +the transfer and must relaunch the whole transfer. In the future, there +will be a retry button for resuming the transfer. + +Finally, because Ring do not support text conferences (just video +conferences, where there is one master merging slaves SIP calls), there +is no real file transfer in conferences. For now, when you are in a +conference on the gnome client for example: A master, B and C slave. B +will be able to send a file to A the master (C same) A will be able to +send a file to B or to C (just have to select the correct conversation). + +### TODO List + +1. Add unit-tests (https://gerrit-ring.savoirfairelinux.com/\#/c/9365/) +2. Show subtransfers status for outgoing files +3. Offset resume (for failed transfer) diff --git a/technical/basic-features/index.rst b/technical/basic-features/index.rst @@ -0,0 +1,12 @@ +########## +Basic Features +########## + +.. toctree:: + :maxdepth: 1 + + manage-accounts + manage-contacts + swarm + let's-do-a-call + file-transfer diff --git a/technical/basic-features/let's-do-a-call.md b/technical/basic-features/let's-do-a-call.md @@ -0,0 +1,77 @@ +# Calling + +**NOTE: this page detail the principle for Jami accounts. For SIP accounts, the SIP protocol is used.** + +## Daemon side + +When creating a call between two peers, Jami mainly uses known protocols such as ICE, SIP or TLS. However, to make it distributed, the process of creating a call is a bit different. To summarize, when someone wants to contact one of its contact, this is what they will do: + +1. Search the contact presence on the DHT (for more details, see https://git.jami.net/savoirfairelinux/ring-project/wikis/technical/2.2.%20Manage%20contacts) +2. Once the contact is found, send a call request, announcing the known candidates (the ip of each network interfaces + relay addresses (TURN) + reflexives addresses (UPnP, public ones). +3. Wait for the response of the contact (they will respond their known addresses). +4. Negotiate the socket via ICE. In fact, two ICE sessions are negotiated. One (preferred) in TCP, one in UDP (as a fallback). +5. Then, the socket is encrypted in TLS (if TCP) or DTLS (if UDP). +6. The contact is now able to accept or decline the call. When they accept, a ICE transport (UDP only for now) is negotiated to create 4 new sockets for the medias (2 for audio, 2 for video). +7. The call is now alive! + +### Exchange ICE candidates + +Everything really starts in `jamiaccount.cpp` (`JamiAccount::startOutgoingCall`). Once both ICE objects are ready and when the contact is found via the DHT, the call request for the contact is crafted. This request contains all the informations necessary for the remote ICE session defined by: + +```cpp +dht::IceCandidates(callvid, blob) +``` + +where `callvid` is a random number used to identify the call and blob contains two concatened ICE messages (`IceTransport::packIceMsg` in `ice_transport.cpp`) containing the password of the session, the *ufrag* and ICE candidates.) like: + +``` +0d04b935 +7c33834e7cf944bf0e367b42 +H6e6ca382 1 UDP 2130706431 2607:fad8:4:6:9eb6:d0ff:dead:c0de 14133 typ host +H42c1g477 1 UDP 2130706431 fe80::9eb6:d0ff:fee7:1412 14133 typ host +Hc0a8027e 1 UDP 2130706431 192.168.0.123 34567 typ host +Sc0a8027e 1 UDP 1694498815 X.X.X.X 32589 typ srflx +0d04b932 +7c33834e7cf944bf0e367b47 +H6e6ca682 1 TCP 2130706431 2607:fad8:4:6:9eb6:d0ff:dead:c0de 50693 typ host tcptype passive +H6e6ca682 1 TCP 2130706431 2607:fad8:4:6:9eb6:d0ff:dead:c0de 9 typ host tcptype active +H42c1b577 1 TCP 2130706431 fe80::9eb6:d0ff:fee7:1412 50693 typ host tcptype passive +H42c1b577 1 TCP 2130706431 fe80::9eb6:d0ff:fee7:1412 9 typ host tcptype active +Hc0a8007e 1 TCP 2130706431 192.168.0.123 42751 typ host tcptype passive +Hc0a8007e 1 TCP 2130706431 192.168.0.123 9 typ host tcptype active +Sc0a8007e 1 TCP 1694498815 X.X.X.X 42751 typ srflx tcptype passive +``` + +and is sent via the DHT in an encrypted message for the device to `hash(callto:xxxxxx)` where `xxxxxx` is the device id. The peer will answer at the exact same place (but encrypted for the sender device) its own `dht::IceCandidates`. See `JamiAccount::replyToIncomingIceMsg` for more details. + +The ICE session is created both side when they have all the candidates (so for the sender, when the reply from the contact is received). + +### ICE negotiation + +Pending calls are managed by `JamiAccount::handlePendingCallList()`, which first wait that the TCP negotiation finish (and if it fails, wait for the UDP one). The code for the ICE negotiation is mainly managed by [pjproject](https://github.com/pjsip/pjproject) but for Jami, the interesting part is located in `ice_transport.cpp`. Moreover, we add some important patches/features on top of *pjproject* not merged upstream for now (for example, ICE over TCP). These patches are present in `contrib/src/pjproject`. + +### Encrypt the control socket + +Once the socket is created and managed by an **IceTransport** instance, it is then wrapped in a **SipTransport** corresponding to a *TlsIceTransport*. The main code is located into `JamiAccount::handlePendingCall()` and the wrapping is done into `SipTransportBroker::getTlsIceTransport`. Finally, our session is managed by **TlsSession** in `daemon/src/security/tls_session.cpp` and uses the GnuTLS library. + +So, the control socket will be a TLS (1.3 if your and your peer gnutls version support it) if a TCP socket is negotiated. If a UDP socket is negotiated instead (due to firewall restrictions/problem in the negotiation/etc), the socket will use DTLS (still managed by the same parts). + +The control socket is used to transmit SIP packets, like invites, custom messages (Jami sends the VCard of your profile on this socket at the start of the call, or the rotation of the camera), text messages. + +Related articles: + ++ https://jami.net/improved-video-rotation-support/ ++ https://jami.net/peer-to-peer-file-sharing-support-in-jami/ + +### Media sockets + +Media sockets are SRTP sockets where the key is negotiated through the TLS Session previously created. +**TODO** + +### Architecture + + + +## Client APIs + +## LRC diff --git a/technical/basic-features/manage-accounts.md b/technical/basic-features/manage-accounts.md @@ -0,0 +1,138 @@ +# Managing Accounts +In this part, we will learn how to manage a Ring account. This means, how to create a Ring account, modify the basic settings and delete the account. This part will *NOT* explain what all the settings mean or how we can use the account to do any action like adding a contact. + +## Create a new account + +### Client side + +#### Gnome + +The code related to this feature is located in `src/accountcreationwizard.*` + +#### LRC + +The account creation is mainly managed by the *NewAccountModel* in `src/api/newaccountmodel.h` and `src/newaccountmodel.cpp` + +### Daemon side + +#### API + +In cx.ring.Ring.ConfigurationManager: + +```xml +<method name="addAccount" tp:name-for-bindings="addAccount"> + <tp:docstring> + Add a new account. When created, the signal <tp:member-ref>accountsChanged</tp:member-ref> is emitted. The clients must then call <tp:member-ref>getAccountList</tp:member-ref> to update their internal data structure. + <tp:rationale>If no details are specified, the default parameters are used.</tp:rationale> + <tp:rationale>The core tries to register the account as soon it is created.</tp:rationale> + </tp:docstring> + <annotation name="org.qtproject.QtDBus.QtTypeName.In0" value="MapStringString"/> + <arg type="a{ss}" name="details" direction="in" tp:type="String_String_Map"> + <tp:docstring> + The new account settings + </tp:docstring> + </arg> + <arg type="s" name="createdAccountId" direction="out"> + <tp:docstring> + A new account ID + </tp:docstring> + </arg> +</method> +``` + +The details can be retrieven from the method `getAccountTemplate(type)` with `type=RING` or `type=SIP`. For example, this is the following code used in LRC. + +```cpp +std::string +NewAccountModel::createNewAccount(profile::Type type, + const std::string& displayName, + const std::string& archivePath, + const std::string& password, + const std::string& pin) +{ + + MapStringString details = type == profile::Type::SIP? + ConfigurationManager::instance().getAccountTemplate("SIP") : + ConfigurationManager::instance().getAccountTemplate("RING"); + using namespace DRing::Account; + details[ConfProperties::TYPE] = type == profile::Type::SIP? "SIP" : "RING"; + details[ConfProperties::DISPLAYNAME] = displayName.c_str(); + details[ConfProperties::ALIAS] = displayName.c_str(); + details[ConfProperties::UPNP_ENABLED] = "true"; + details[ConfProperties::ARCHIVE_PASSWORD] = password.c_str(); + details[ConfProperties::ARCHIVE_PIN] = pin.c_str(); + details[ConfProperties::ARCHIVE_PATH] = archivePath.c_str(); + QString accountId = ConfigurationManager::instance().addAccount(details); + return accountId.toStdString(); +} +``` + +When a new account is added, the signal `accountsChanged` will be emitted. The client should update its internal structure after this signal with other methods in ConfigurationManager. + +#### Core + +The main logic to create a new account is located in `src/ringdht/ringaccount.cpp`, in `RingAccount::createAccount` + +### How it works, from scratch + +A Ring account is in fact represented by some files stored in a gzip archive. If a password is provided during the account creation, the archive will be encrypted as following: `dht::crypto::aesEncrypt(archive, password)` (`dht::crypto::aesEncrypt` is defined in OpenDHT and use `nettle/{aes,gcm}`). This is what the archive will contain a big JSON file with: + +1. The private key `ringAccountKey` and certificate chain `ringAccountCert` (base64 encoded) +2. Generated CA key (for self-signed certificates) `ringCAKey` +3. Revocated devices `ringAccountCRL` +4. The ethereum private key `ethKey` for the device. It's only used when you register your name on `ns.ring.cx`. Not mandatory. +5. The contacts +6. The account settings + +So let's generate it! + +**TODO** + + + +## Delete the account + +Deleting a Ring account is pretty simple. Because the keys are only on the device, if the keys are deleted... the account is deleted! The only thing outside the device is the username, on the nameserver. To remove this info, it depends how the nameserver work. For example, it's not possible with https://ns.ring.cx + +### Client side + +#### Gnome + +The code related to this feature is located in `newaccountsettingsview.cpp` + +#### LRC + +The account deletion is in the *NewAccountModel* in `src/api/newaccountmodel.h` and `src/newaccountmodel.cpp` (`removeAccount`) + +### Daemon side + +#### API + +In cx.ring.Ring.ConfigurationManager: + +```xml +<method name="removeAccount" tp:name-for-bindings="removeAccount"> + <tp:docstring> + Remove an existing account. When removed, the signal <tp:member-ref>accountsChanged</tp:member-ref> is emitted. The clients must then call <tp:member-ref>getAccountList</tp:member-ref> to update their internal data structure. + </tp:docstring> + <arg type="s" name="accoundID" direction="in"> + <tp:docstring> + The account to remove, identified by its ID + </tp:docstring> + </arg> +</method> +``` + +When the account is deleted, the signal `accountsChanged` will be emitted. The client should update its internal structure after this signal with other methods in ConfigurationManager. + +#### Core + +The main logic to create a new account is located in `src/manager.cpp`, in `Manager::removeAccount`. It removes the accounts files and update the config (`dring.yml`). + +## Update the settings + +**TODO** + +## Add and revoke devices + +**TODO** diff --git a/technical/basic-features/manage-contacts.md b/technical/basic-features/manage-contacts.md @@ -0,0 +1,215 @@ +# Managing Contacts + +This section will present how to find and add a contact from the DHT to the client. The usage of a name server will not be explained here. If you want details about that, please read: https://git.ring.cx/savoirfairelinux/ring-project/wikis/technical/Name-Server-Protocol + +## Presence on the network + +### Announce the presence on the DHT + +The presence is pretty simple to announce on the DHT. In fact, it's +just a value containing the device hash (see [previous +section](https://git.ring.cx/savoirfairelinux/ring-project/wikis/technical/2.1.-Manage-Accounts)) +on the hash corresponding to the Ring ID. So, if we have the account +`bf5f1e21d3eb2c1246946aa49d5dcf3e5b9cb1b9` with the device +`62fbdff0ce86f368c7d3c2682539e5ba9e06404f`, the following defined +value will be sent over the DHT: + +```cpp +/** + * Device announcement stored on DHT. + */ +struct RingAccount::DeviceAnnouncement : public dht::SignedValue<DeviceAnnouncement> +{ +private: + using BaseClass = dht::SignedValue<DeviceAnnouncement>; +public: + static const constexpr dht::ValueType& TYPE = dht::ValueType::USER_DATA; + dht::InfoHash dev; + MSGPACK_DEFINE_MAP(dev); +}; +``` + +(This value can be put with `dht_.put(h, VALUE, dht::DoneCallback{}, {}, true);`, as a permanent put). If the device is announced, the device is present. For now, there is no way to delete or edit a value on the DHT (this will come when OpenDHT will supports ECC). So, the presence always have a delay for now (mean delay: expire-time/2, so 2min30 for now). + +### Get if a contact is present + +Now our presence on the network, it's time to get if somebody is present on the DHT. With the previous section, it's easy to do the reverse process. To know if somebody is present on the DHT (ex: `bf5f1e21d3eb2c1246946aa49d5dcf3e5b9cb1b9`), we have to get value at `bf5f1e21d3eb2c1246946aa49d5dcf3e5b9cb1b9` and retrieve the `DeviceAnnouncement` on this hash. The related code in the ring daemon is in `ringaccount.cpp`: + +```cpp +auto shared = std::static_pointer_cast<RingAccount>(shared_from_this()); +auto treatedDevices = std::make_shared<std::set<dht::InfoHash>>(); +dht_.get<dht::crypto::RevocationList>(to, [to](dht::crypto::RevocationList&& crl){ + tls::CertificateStore::instance().pinRevocationList(to.toString(), std::move(crl)); + return true; +}); +dht_.get<DeviceAnnouncement>(to, [shared,to,treatedDevices,op](DeviceAnnouncement&& dev) { + if (dev.from != to) + return true; + if (treatedDevices->emplace(dev.dev).second) + op(shared, dev.dev); + return true; +}, [=](bool /*ok*/){ + { + std::lock_guard<std::recursive_mutex> lock(shared->buddyInfoMtx); + auto buddy_info_it = shared->trackedBuddies_.find(to); + if (buddy_info_it != shared->trackedBuddies_.end()) { + if (not treatedDevices->empty()) { + for (auto& device_id : *treatedDevices) + shared->onTrackedBuddyOnline(buddy_info_it, device_id); + } else + shared->onTrackedBuddyOffline(buddy_info_it); + } + } + RING_DBG("[Account %s] found %lu devices for %s", + getAccountID().c_str(), treatedDevices->size(), to.to_c_str()); + if (end) end(shared, not treatedDevices->empty()); +}); +``` + +And that's all. + +### Pending request + +### Send a request + +**TODO craft request** + +Finally, once the trust request is crafted, we can push the request to the following hash: `InfoHash("inbox:" + deviceId)` + +The following code is used in the daemon: +```cpp +dht_.putEncrypted(dht::InfoHash::get("inbox:"+dev.toString()), dev, dht::TrustRequest(DHT_TYPE_NS, payload)); +``` + +### Receiving a request + +**TODO** + +(Accept/Block/Discard) + +### Daemon API + +All methods to follow the presence of a buddy is located in the `PresenceManager` such as: + +```xml +<signal name="newBuddyNotification" tp:name-for-bindings="newBuddyNotification"> + <tp:added version="1.3.0"/> + <tp:docstring> + Notify when a registered presence uri presence informations changes + </tp:docstring> + <arg type="s" name="accountID"> + <tp:docstring> + The associated account + </tp:docstring> + </arg> + <arg type="s" name="buddyUri"> + <tp:docstring> + The registered URI + </tp:docstring> + </arg> + <arg type="b" name="status"> + <tp:docstring> + Is the URI present or not + </tp:docstring> + </arg> + <arg type="s" name="lineStatus"> + <tp:docstring> + A string containing informations from the user (human readable) + </tp:docstring> + </arg> +</signal> +``` + +All methods and signals used to manage trust requests and contacts are in the `ConfigurationManager` such as: + +```xml +<method name="getTrustRequests" tp:name-for-bindings="getTrustRequests"> + <tp:added version="2.2.0"/> + <arg type="s" name="accountID" direction="in"> + </arg> + <annotation name="org.qtproject.QtDBus.QtTypeName.Out0" value="VectorMapStringString"/> + <arg type="aa{ss}" name="requests" direction="out" > + <tp:docstring> + A list of contact request details. Details: + - from: account ID of sender + - received: UNIX timestamp of reception date + - payload: attached payload + </tp:docstring> + </arg> +</method> + +<method name="acceptTrustRequest" tp:name-for-bindings="acceptTrustRequest"> + <tp:added version="2.2.0"/> + <arg type="s" name="accountID" direction="in"> + </arg> + <arg type="s" name="from" direction="in"> + </arg> + <arg type="b" name="success" direction="out" tp:type="Boolean"> + <tp:docstring> + True if the operation succeeded. + </tp:docstring> + </arg> +</method> + +<method name="discardTrustRequest" tp:name-for-bindings="discardTrustRequest"> + <tp:added version="2.2.0"/> + <arg type="s" name="accountID" direction="in"> + </arg> + <arg type="s" name="from" direction="in"> + </arg> + <arg type="b" name="success" direction="out" tp:type="Boolean"> + <tp:docstring> + True if the operation succeeded. + </tp:docstring> + </arg> +</method> + +<signal name="incomingTrustRequest" tp:name-for-bindings="incomingTrustRequest"> + <tp:added version="2.2.0"/> + <tp:docstring> + Notify clients that a new contact request has been received. + </tp:docstring> + <arg type="s" name="accountID"> + </arg> + <arg type="s" name="from"> + </arg> + <arg type="ay" name="payload"> + </arg> + <arg type="t" name="receiveTime"> + </arg> +</signal> + +<method name="sendTrustRequest" tp:name-for-bindings="sendTrustRequest"> + <tp:added version="2.2.0"/> + <arg type="s" name="accountID" direction="in"> + </arg> + <arg type="s" name="to" direction="in"> + </arg> + <arg type="ay" name="payload" direction="in"> + </arg> +</method> + +<method name="addContact" tp:name-for-bindings="addContact"> + <tp:added version="3.0.0"/> + <arg type="s" name="accountID" direction="in"> + </arg> + <arg type="s" name="uri" direction="in"> + </arg> +</method> + +<method name="removeContact" tp:name-for-bindings="removeContact"> + <tp:added version="3.0.0"/> + <arg type="s" name="accountID" direction="in"> + </arg> + <arg type="s" name="uri" direction="in"> + </arg> + <arg type="b" name="ban" direction="in" tp:type="Boolean"> + <tp:docstring> + True if the the contact should be banned. + If false, the contact is removed from the contact list (banned or not). + </tp:docstring> + </arg> +</method> +``` + +If you want some examples, these methods are used into `contactmodel.cpp` in LRC. diff --git a/technical/basic-features/swarm.md b/technical/basic-features/swarm.md @@ -0,0 +1,492 @@ +# Understanding Swarms + +## Synospis + +The goal of this document is to describe how group chats +(a.k.a. **swarm chat**) will be implemented in Jami. + +A *swarm* is a group able to discuss without any central authority in +a resilient way. Indeed, if two person doesn't have any connectivity +with the rest of the group (ie Internet outage) but they can contact +each other (in a LAN for example or in a sub network), they will be +able to send messages to each other and then, will be able to sync +with the rest of the group when it's possible. + +So, the *swarm* is defined by: +1. Ability to split and merge following the connectivity. +2. Syncing of the history. Anyone must be able to send a message to the whole group. +3. No central authority. Can't rely on any server. +4. Non-repudiation. Devices must be able to verify old messages validity and to replay the whole history. +5. PFS on the transport. Storage is managed by the device. + +Main idea is to get a synchronized merkle tree with the participants. + +We identified four modes for swarm chat that we want to implement: ++ **ONE_TO_ONE**, basically the case we have today, when you discuss to a friend ++ **ADMIN_INVITES_ONLY** generally a class where the teacher can invite people, but not students ++ **INVITES_ONLY** a private group of friends ++ **PUBLIC** basically an opened forum + +## Scenarios + +### Create a Swarm + +*Bob wants to create a new swarm* + +1. Bob create a local git repository. +2. Then, he adds an initial signed commit and adds the following: + + His public key in `/admins` + + His device certificate in ̀ /devices` + + His CRL in ̀ /crls` +3. The hash of the first commit becomes the **ID** of the conversation +4. Bob announce to his other devices that he creates a new conversation. This is done via an invite to join the swarm sent through the DHT to other devices linked to that account. + +### Adding someone + +*Alice adds Bob* + +1. Alice adds Bob to the repo: + + Adds the invited uri in `/invited` + + Adds the CRL into `/crls` +2. Alice sends a request on the DHT + +### Receiving an invite + +*Alice gets the invite to join the previously create swarm* + +1. She accepts the invite (if decline, do nothing, it will just stay into invited and Alice will never receives any message) +2. A peer to peer connection between Alice and Bob is done. +3. Alice pull the git repo of Bob. **WARNING this means that messages needs a connection, not from the DHT like today** +4. Alice validates commits from Bob +5. To validate that Alice is a member, she removes the invite from `/invited` directory, then adds her certificate into the `/members` directory +6. Once all commits validated and on her device, other members of the group are discovered by Alice. with these peers, she will construct the **DRT** (explained below) with Bob as a bootstrap. + +### Sending a message + +*Alice sends a message* + +Sending a message is pretty simple. Alice write a commit-message with the following format: + +**TODO format unclear** + +and adds her device and CRL to the repository if missing (others must be able to verify the commit). Merge conflicts are avoided because we are mostly based on commit messages, not files (unless CRLS + certificates but they are located). then she announce the new commit via the **DRT** with a service message (explained later) and ping the DHT for mobile devices (they must receive a push notification). + +For pinging other devices, the sender sends to other members a SIP messages with mimetype = "application/im-gitmessage-id" containing a JSON with the "deviceId" which sends the message, the "id" of the conversation related and the "commit" + +### Receiving a message + +*Bob receives the message from Alice* + +1. *Bob* do a git pull on *Alice* +2. Commits MUST be verified via a hook +3. If all commits are valid, commits are stored and displayed. Then *Bob* announce the message via the DRT for other devices. +4. If all commits are not valid, pull is cancelled. *Alice* must restablish her state to a correct state. **TODO process* + +### Validating a commit + +To avoid users to push some unwanted commits (with conflicts, false messages, etc), this is how each commits (from the oldest to the newest one) MUST be validated before merging a remote branch: + +Note: if the validation fails, the fetch is ignored and we do not merge the branch (and remove the datas), and the user should be notified +Note2: If a fetch is too big, it's not done (**TODO**) + ++ For each commits, check that the device who try to send the commit is authorized at this moment, and that the certificates are present (in /devices for the device, and in /members or /admins for the issuer). ++ 3 cases. The commit has 2 parents, so it's a merge, nothing more to validate here ++ The commit has 0 parent, it's the initial commit: + + Check that admin cert is added + + Check that device cert is added + + Check CRLs added + + Check that no other file is added ++ The commit has 1 parent, commit msg is a JSON with a type: + + If text (or other mimetype that doesn't change files) + + Check signature from certif in repo + + Check that no weird file is added outside device cert nor removed + + If vote + + Check that vote is for user that signs the commit + + Check that vote is from an admin and device present & not banned + + Check that no weird file is added nor removed + + If member + + If adds + + Check that commit is correctly signed + + Check that certificate is added in /invited + + Check that no weird file is added nor removed + + If ONE_TO_ONE, check that we only have one admin, one member + + If ADMIN_INVITES_ONLY, check that invite is from an admin + + If joins + + Check that commit is correctly signed + + Check that device is added + + Check that invitation is moved to members + + Check that no weird file is added nor removed + + If kickban + + Check that vote is valid + + Check that the user is ban via an admin + + Check that member or device certificate is moved to banned/ + + Check that only files related to the vote is removed + + Check that no weird file is added nor removed + + else fail. Notify the user that they may be with an old version or that peer tried to submit unwanted commits + + +### Ban a device + +*Alice, Bob, Carla, Denys are in a swarm. Alice bans Denys* + +This is one of the most difficult scenario in our context. Without central authority we can't trust: + +1. Timestamps of generated commits +2. Conflicts with banned devices. If multiple admin devices are present and if Alice can speak with Bob but not Denys and Carla ; Carla can speak with Denys ; Denys bans Alice, Alice bans Denys, what will be the state when the 4 members will merge the conversations. +3. A device can be compromised, stolen or its certificate can expire. We should be able to ban a device and avoid that it lies about its expiration or send messages in the past (by changing its certificate or the timestamp of its commit). + +Similar systems (with distributed group systems) are not so much, but this is some examples: + ++ [mpOTR doesn't define how to ban someone](https://www.cypherpunks.ca/~iang/pubs/mpotr.pdf) ++ Signal, without any central server for group chat (EDIT: they recently change that point) doesn't give the ability to ban someone of a group. + +This voting system need a human action to ban someone or must be based on the CRLs infos from the repository (because we can't trust external CRLs) + +### Remove a device from a conversation + +This is the only part that MUST have a consensus to avoid conversation's split, like if two members kick each others from the conversation, what will see the third one? + +This is needed to detect revoked devices, or simply avoid to get unwanted people present in a public room. The process is pretty similar between a member and a device: + +*Alice removes Bob* + +Note: Alice MUST be admins to vote + ++ First, she votes for ban Bob. To do that, she creates the file in /votes/members/uri_bob/uri_alice (members can be replaced by devices for a device) and commit ++ Then she checks if the vote is resolved. This means that >50% of the admins agree to ban Bob (if she is alone, it's sure it's more than 50%). ++ If the vote is resolved, files into /votes can be removed, all files for Bob in /members, /admins, /CRLs, /devices can be removed (or only in /devices if it's a device that is banned) and Bob's certificate can be placed into /banned/members/bob_uri.crt (or /banned/devices/uri.crt if a device is banned) and committed to the repo ++ Then, Alice informs other users (outside Bob) + +### Remove a conversation + +1. Save in convInfos removed=time::now() (like removeContact saves in contacts) that the conversation is removed and sync with other user's devices +2. Now, if a new commit is received for this conversation it's ignored +3. Now, if Jami startup and the repo is still present, the conversation is not announced to clients +4. Two cases: + a. If no other member in the conversation we can immediately removes the repository + b. If still other members, commit that we leave the conversation, and now wait that at least another device sync this message. This avoid the fact that other members will still detect the user as a valid member and still sends new messages notifications. +5. When we are sure that someone is synched, remove erased=time::now() and sync with other user's devices +6. All devices owned by the user can now erase the repository and related files + +## How to specify a mode + +Modes can't be changed through the time. Or it's another conversation. So, this data is stored into the initial commit message. +The commit message will be the following: + + +```json +{ + "type": "initial", + "mode": 0, +} +``` + +For now, "mode" accepts values 0 (ONE_TO_ONE), 1 (ADMIN_INVITES_ONLY), 2 (INVITES_ONLY), 3 (PUBLIC) + +### Processus for 1:1 swarms + +The goal there is to keep the old API (addContact/removeContact, sendTrustRequest/acceptTrustRequest/discardTrustRequest) to generate swarm with a peer and its contact. This still imply some changes that we cannot ignore: + +The process is still the same, an account can add a contact via addContact, then send a TrustRequest via the DHT. But two changes are necessary: +1. The TrustRequest embeds a "conversationId" to inform the peer what conversation to clone when accepting the request +2. TrustRequest are retried when contact come backs online. It's not the case today (as we don't want to generate a new TrustRequest if the peer discard the first). So, if an account receives a trust request, it will be automatically ignored if the request with related conversation is declined (as convRequests are synched) + +Then, when a contact accepts the request, a period of sync is necessary, because the contact now needs to clone the conversation. + +removeContact() will remove the contact and related 1:1 conversations (with the same processus as "Remove a conversation"). The only note there is that if we ban a contact, we don't wait for sync, we just remove all related files. + +#### Tricky scenarios + +There is some cases were two conversations can be created. This is at least two of those scenarios: + +1. Alice adds Bob +2. Bob accepts +3. Alice removes Bob +4. Alice adds Bob + +or + +1, Alice adds Bob & Bob adds Alice at the same time, but both are not connected together + +In this case, two conversations are generated. We don't want to removes messages from users or choose one conversation here. So, sometimes two 1:1 swarm between the same members will be shown. It will generate some bugs during the transition time (as we don't want to break API, the infered conversation will be one of the two shown conversation, but for now it's "ok-ish", will be fixed when clients will fully handle conversationId for all APIs (calls, file transfer, etc)). + +### Conversations requests specification + +Conversations requests are represented by a **Map<String, String>** with the following keys: + ++ id: the conversation id ++ from: uri of the sender ++ received: timestamp ++ title: (optional) name for the conversation ++ description: (optional) ++ avatar: (optional) + +### Conversation's profile synchronization + +To be identifiable, a conversation generally needs some metadatas, like a title (eg: Jami), a description (eg: some links, what is the project, etc), and an image (the logo of the project). Those metadatas are optional, but shared across all members, so need to be synced and incorporated in the requests. + +#### Storage in the repository + +The profile of the conversation is stored in a classic vCard file at the root (`/profile.vcf`) like: + +``` +BEGIN:VCARD +VERSION:2.1 +FN:TITLE +DESCRIPTION:DESC +END:VCARD +``` + +#### Synchronization + +To update the vCard, an user with enough permissions (by default: =ADMIN) needs to edit `/profile.vcf`. and will commit the file with the mimetype `application/update-profile`. The new message is sent via the same mechanism and all peers will receive the **MessageReceived** signal from the daemon. The branch is dropped if the commit contains other files or too big or if done by a non authorized member (by default: <ADMIN). + +#### Merge conflicts management + +Because two admins can change the description at the same time, a merge conflict can occurs on `profile.vcf`. In this case, the commit with the higher hash (eg ffffff > 000000) will be chosen. + +#### APIs + +The user got 2 methods to get and set conversation's metadatas: + +```xml + <method name="updateConversationInfos" tp:name-for-bindings="updateConversationInfos"> + <tp:added version="10.0.0"/> + <tp:docstring> + Update conversation's infos (supported keys: title, description, avatar) + </tp:docstring> + <arg type="s" name="accountId" direction="in"/> + <arg type="s" name="conversationId" direction="in"/> + <annotation name="org.qtproject.QtDBus.QtTypeName.In2" value="VectorMapStringString"/> + <arg type="a{ss}" name="infos" direction="in"/> + </method> + + <method name="conversationInfos" tp:name-for-bindings="conversationInfos"> + <tp:added version="10.0.0"/> + <tp:docstring> + Get conversation's infos (mode, title, description, avatar) + </tp:docstring> + <annotation name="org.qtproject.QtDBus.QtTypeName.Out0" value="VectorMapStringString"/> + <arg type="a{ss}" name="infos" direction="out"/> + <arg type="s" name="accountId" direction="in"/> + <arg type="s" name="conversationId" direction="in"/> + </method> +``` + +where `infos` is a `map<str, str>` with the following keys: + ++ mode: READ-ONLY ++ title ++ description ++ avatar + +#### Re-import an account (link/export) + +The archive MUST contains conversationId to be able to be able to retrieve conversations on new commits after a re-import (because there is no invite at this point). If a commit come for a conversation not present there is two possibilities: + ++ The conversationId is there, in this case the daemon is able to re-clone this conversation ++ The conversationId is missing, so the daemon asks (via a message `{{"application/invite", conversationId}}`) a new invite that the user needs to (re)accepts + +Note, a conversation can only be retrieven if a contact or another device is there, else it will be lost. There is no magic. + +### Other mime types + ++ `application/call-history+json` with a JSON containing the `id` and the `duration` of the call ++ `application/data-transfer+json` with a JSON containing the `tid` of the file + +## Used protocols + +### Git + +#### Why this choice + +Each conversation will be a git repository. This choice is motivated by: + +1. We need to sync and ordering messages. The Merkle Tree is the perfect structure to do that and can be linearized by merging branches. Moreover, because it's massively used by Git, it's easy to sync between devices. +2. Distributed by nature. Massively used. Lot of backends and plugguable. +3. Can verify commits via hooks and massively used crypto +4. Can be stored in a database if necessary +5. Conflicts are avoided by using commit messages, not files. + +#### What we have to validate + ++ Performance? `git.lock` can be low ++ Hooks in libgit2 ++ Multiple pulls at the same time? + +#### Limits + +History can't be deleted. To delete a conversation, the device has to leave the conversation and create another one. + +However, non permanent messages (like messages readable only for some minutes) can be sent via a special message via the DRT (like Typing or Read notifications). + +Moreover editing messages will be possible! (`commit --fixup`) + +#### Structure + +``` +/ + - invited + - admins (public keys) + - members (public keys) + - devices (certificates of authors to verify commits) + - banned + - devices + - members + - votes + - members + - uri + - devices + - uri + - CRLs +``` + +#### Attacks? + ++ Avoid git bombs + +#### Notes + +Timestamp of a commit can be trusted because it's editable. Only the user's timestamp can be trusted. + +### TLS + +Git operations, control messages, files and other things will use a p2p TLS v1.3 link with only cyphers which garanty PFS. So each key is renegotiated for each new connexion. + +### DHT (udp) + +Used to send messages for mobiles (to trigger push notifications) and to initiate TCP connexions. + +### DRT (name will change) + +The DRT is a new concept used in swarm to maintain p2p connections. Indeed, group members define a graph of nodes (identified by a hash) en must be connected. + +So we need a structure that: + +1. Maximize the connected nodes at every time +2. Minimize the time for message transmission +3. Minimize links between peers +4. Needs low calculation + +Several solutions exists: + +1. Each node has a connection to the next node. So we only need $N$ connections, but it's not effective to transmit a message, because the message will go though all peers, one by one. +2. Each nodes is connected to all other nodes $N\timesN$ connections. Effective to transmit, but need more resources **WILL BE CHOSEN FOR THE FIRST VERSION** +3. *Maximizing the Coverage of Roadmap Graph for Optimal Motion Planning* +https://www.hindawi.com/journals/complexity/2018/9104720/. But need calculations +4. Use the algorithm of the DHT for the routing table. The 4 points are basically solved and already used by Jami in UDP. + +Note: to optimize the socket numbers, a socket will be given by a **ConnectionManager** to get multiplexed sockets with a given hash. This means that if we need to transmit several files, and chat with someone, only one socket will be used. + +### File transfer (libtorrent?) + +**TODO** + +### Network activity + +#### Process to invite someone + +Alice wants to invite Bob: + +1. Alice adds bob to a conversation +2. Alice generates an invite: { "application/invite+json" : { + "conversationId": "$id", + "members": [{...}] +}} +3. Two possibilites for sending the message + a. If not connected, via the DHT + b. Else, Alice sends on the SIP channel +4. Two possibilities for Bob + a. Receives the invite, a signal is emitted for the client + b. Not connected, so will never receives the request cause Alice must not know if Bob just ignored or blocked alice. The only way is to regenerate a new invite via a new message (cf next scenario) + +#### Process to send a message to someone + +Alice wants to send a message to Bob: + +1. Alice adds a message in the repo, giving an ID +2. Alice gets a message received (from herself) if successful +3. Two possibilities, alice and bob are connected, or not. In both case a message is crafted: { "application/im-gitmessage-id" : "{"id":"$convId", "commit":"$commitId", "deviceId": "$alice_device_hash"}"}. + a. If not connected, via the DHT + b. Else, Alice sends on the SIP channel +4. Four possibilities for Bob: + a. Bob is not connected to Alice, so if he trusts alice, ask for a new conection and go to b. + b. If connected, fetch from alice and announce new messages + c. Bob doesn't know that conversation. Ask through the DHT to get an invite first to be able to accept that conversation ({"application/invite", conversationId}) + d. Bob is disconnected (no network, or just closed). He will not receive the new message, but will try to sync when the next connection will occurs + + +### Implementation + + +--------------- + +**!! OLD DRAFT !!** + +Note: Following notes are not organized yet. Just some line of thoughts. + +## Crypto improvements. + +For a serious group chat feature, we also need serious crypto. With the current design, if a certificate is stolen as the previous DHT values of a conversation, the conversation can be decrypted. Maybe we need to go to something like **Double ratchet**. + +Note: a lib might exists to implement group conversations. TODO, investigate. + +Needs ECC support in OpenDHT + +## Usage + +### Add Roles? + +There is two major use case for group chats: + +1. Something like a Mattermost in a company, with private channels, and some roles (admin/spectator/bot/etc) or for educations (where only a few are active). +2. Horizontal conversations like a conversation between friends. + +Ring will be for which one? + +#### Implementation idea + +A certificate for a group which sign user with a flag for a role. Adding or revoking can also be done. + +### Join a conversation + ++ Only via a direct invite ++ Via a link/QR Code/whatever ++ Via a room name? (a **hash** on the DHT) + +## What we need + ++ Confidentiality: members outside of the group chat should not be able to read messages in the group ++ Forward secrecy: if any key from the group os compromised, previous messages should remain confidential (as much as possible) + ++ Message ordering: There is a need to have messages in the right order ++ Synchronization: There is also a need to be sure to have all messages at soon as possible. ++ Persistence: Actually, a message on the DHT lives only 10 minutes. Because it's the best timing calculated for this kind of DHT. To persist datas, the node must re-put the value on the DHT every 10 minutes. Another way to do when the node is offline is to let nodes re-put the data. But, if after 10 minutes, 8 nodes are still here, they will do 64 requests (and it's exponential). The current way to avoid spamming for that is queries. This will still do 64 requests but limit the max redundency to 8 nodes. + +## Other distributed ways + ++ IPFS: Need some investigation ++ BitMessage: Need some investigation ++ Maidsafe: Need some investigation + +### Based on current work we have + +Groups chat can be based on the same work we already have for multi devices (but here, with a group certificate). Problems to solve: + +1. History sync. This need to move the database from the client into the daemon. +2. If nobody is connected, the synchronization can't be done, and the person will never see the conversation + +### Another dedicated DHT + +Like a DHT with super user. (Not convinced) + +## File transfer + +Currently the file transfer algorithm is based on a TURN connection (See https://git.ring.cx/savoirfairelinux/ring-project/wikis/tutorials/File-transfer). In case of a big group, this will be bad. We first need a p2p implem for the file transfer. Implement the RFC for p2p transfer. + +Other problem: currently there is no implementation for TCP support for ICE in PJSIP. This is mandatory for this point (in pjsip or home made) + +## Resources + ++ https://eprint.iacr.org/2017/666.pdf ++ Robust distributed synchronization of networked linear systems with intermittent information (Sean Phillips and Ricardo G.Sanfelice) diff --git a/technical/certificates.rst b/technical/certificates.rst @@ -0,0 +1,63 @@ +Certificates +===================== + + + + + + + + + +https://www.linuxjournal.com/content/understanding-public-key-infrastructure-and-x509-certificates + + + + + + + +encoding is important: if there were multiple ways to encode the data +in the certificate, as there might be using BER, the hash might assume +several different values. By using DER, you guarantee that the values +are encoded and decoded consistently into the same series of bytes. If +a single byte changes, a different hash would be created and the +verification + + + + + + + + +signature from a +certificate authority, which is most often a self-signature. + + + +account: + + +- Signed by a CA (from an organization or self-signed). +- The subject UID field must be the hexadecimal form of the JamiId. +- The issuer UID field must be the hexadecimal form of the issuer + public key fingerprint (CA). +- Random RSA key-pair of at least 4096-bits long. + + +The subject UID field of the account certificate must be the hexadecimal +form of the public key fingerprint. The issuer UID field must be the +hexadecimal form of the issuer public key fingerprint. + +device: + + +- This is the identity of one specific device used to run Jami. +- One per device. +- Random and 4096-bits long. +- The SHA-1 fingerprint of the public key becomes the **DeviceId**. +- Must be signed by the private key that created the Jami certificate. +- The subject UID field must be the hexadecimal form of the DeviceId. +- The issuer UID field must be the hexadecimal form of the issuer + public key fingerprint (JamiId). diff --git a/technical/chatview-i18n-(design-draft).md b/technical/chatview-i18n-(design-draft).md @@ -0,0 +1,116 @@ +The GNOME client's chatview lacks proper i18n support. + +Related bug report: https://git.jami.net/savoirfairelinux/ring-client-gnome/issues/900 + +first ideas: + +* i18n functions provided by separate js library +* i18n not trivial, overhead of custom code extremely high => use existing js lib +* library will be embedded into the client => code base should be as mature as possible +* ideally the translation process would be exactly the same as for the C++ code + +# in short + +(1) either + +C++ tells JS code which language is currently used and JS code loads translations + +(2) or + +C++ loads translations and passes them to JS code together with info about currently used language + +# review exisiting js i18n libs + +## [i18next](https://www.i18next.com/) + +**The good** + +* very mature +* huge user base +* lots of documentation +* No runtime dependencies + +**The bad** + +* huge, overkill ? +* uses own JSON format not gettext .mo + +This will require translation of .po files to JSON format (using [i18next-gettext-converter](https://github.com/i18next/i18next-gettext-converter)). + +**The ugly** + +* keys do not have the same meaning as gettext keys (see [this thread](https://stackoverflow.com/questions/19403787/gettext-style-keys-with-i18next-and-general-workflow)), this might be very confusing for the translators +* i18next-gettext-converter requires nodeJS as build dependency + +## [jed](http://messageformat.github.io/Jed/) + +**The good** + +* looks quite mature +* good user base +* provides gettext API (will be very intuitive to translators) +* No runtime dependencies + +**The bad** + +* not very active currently (but it might very well just be stable) +* uses own JSON format not gettext .mo + +This will require translation of .po files to JSON format (e.g. using [po2json](https://github.com/mikeedwards/po2json)). +po2json also requires nodeJS as dependency, **but** it is [available as Debian package](https://packages.debian.org/stable/javascript/node-po2json), which makes it slightly less annoying + + + +Source: [Internationalization in JavaScript](http://www.jeromesteunou.net/internationalisation-in-javascript.html) + +## [Polyglot](http://airbnb.io/polyglot.js/) + +**The good** + +* developed by airbnb, well maintained +* good user base + +**The bad** + +* uses own JSON format not gettext .mo + +**The ugly** + +* runtime dependency on nodeJS + +# using Jed + +* patch draft: + +https://gerrit-ring.savoirfairelinux.com/c/ring-client-gnome/+/10615 + +* Retrieve strings: + +``` +$ cd po/chatview +$ xgettext -o ring-client-gnome-chatview.pot -L Javascript --from-code=utf-8 -D ../../ -f POTFILES.in + +``` + +* Translate them: + +``` +$ cp ring-client-gnome-chatview.pot en.po +$ cp ring-client-gnome-chatview.pot fr.po +# translate ... +``` + +* Install po2json: + +``` +$ apt-get install node-po2json + +``` + +* Generate JSON files: + +``` +$ ./scripts/build-chatview-locales.sh web/i18n +``` + +That's it. JSON translations are loaded as gresource. The `webkitchatcontainer` class calls `init_i18n()`, passing appropriate translation resources to the chatview. diff --git a/technical/conference-protocol.md b/technical/conference-protocol.md @@ -0,0 +1,123 @@ +# Conference protocol + +This document aims to describe evolutions we will do for managing conferences (audio/video). The goal is to improve the current implementation which simply merge SIP calls together and provide a grid view, to a view where participants are listed, can be muted independantly, or the video layout changed (to show only one participant) + +## Definitions + ++ Host: Is the user which mix the audio/video streams for the others ++ Participant: Every user in the conference, even the host + +## Disclaimer + +This document only describes the first steps for now. This means the identification of participants and position in the video mixer sent to all participants. + +## Improve on layouts + +Actually, Jami only provides the possibility to show a grid view of the users. We want to be able to only show one member of the conference (the one which shares its screen for example) + +### Possible layouts + ++ GRID: Every member is shown with the same height/width ++ ONE_BIG_WITH_SMALL: One member is zoomed and the other preview are shown ++ ONE_BIG: One member take the full screen rendered + +### New API + +Two new methods are available to manage the conference Layout in CallManager: + +```cpp +/** + * Change the conference layout + * @param confId + * @param layout 0 = matrix, 1 = one big, others in small, 2 = one in big + */ +void setConferenceLayout(const std::string& confId, int layout); + +/** + * Change the active participant (used in layout != matrix) + * @param confId + * @param participantId If participantId not found, the local video will be shown + */ +void setActiveParticipant(const std::string& confId, const std::string& participantId); +``` + +### Implementation + +The implementation is pretty straight forward. Everything is managed by `conference.cpp` (to link participant to sources) and `video_mixer.cpp` (to render the wanted layout). + + +## Syncing Conferences Informations + +Note: Actually, the word participant is used for callId mixed in a conference. This can lead at first to some problems for the API and must be fixed in the furture + +The goal is to notify all participants the metadatas of the rendered video. This means what participant is in the conference and where the video is located. + +### Layout Info + +The Layout is stored as a VectorMapStringString for clients and internally with a vector<ParticipantInfo> with the following format: + +``` +Layout = { + { + "uri": "participant", "x":"0", "y":"0", "w": "0", "h": "0", "isModerator": "true" + }, + { + "uri": "participant1", "x":"0", "y":"0", "w": "0", "h": "0", "isModerator": "false" + } + (...) +} +``` + +### New API + +A new method (in CallManager) and a new signal to respectively get current conference infos and updates are available: + +```cpp +VectorMapStringString getConferenceInfos(const std::string& confId); + +void onConferenceInfosUpdated(const std::string& confId, const VectorMapStringString& infos); +``` + +### Implementation + +The `Conference` Object (which only exists if we mix calls, this means that we are the master) manages the informations for the whole conference, based on the LayoutInfos of each `Call` objects. The getConferenceInfos will retrieve infos directly from this object. + +So, every `Call` object now have a LayoutInfo and if updated, ask the `Conference` object to updates its infos. + +The master of a conference sends its infos via the SIP channel as a message with the following MIME type: +`application/confInfo+json` + +So, if a call receives some confInfo, we know that this call is a member of a conference. + +To summarize, `Call` manages received layouts, `Conference` managed sent layouts. + +## Moderators + +A conference can be controlled (for now, only the layout an be controlled). +A moderator is added if on the same device of the host for now. + +### Implementation + +To change a layout, the moderator can send a payload with "application/confOrder+json" as type: + +``` +{ + "layout": int +} +``` + +To set a participant as active, the moderator can send a payload with "application/confOrder+json" as type: + +``` +{ + "activeParticipant": "uri" +} +``` + +## Future + ++ Control moderators on a conference ++ Moderators can hangup ++ Moderators can mute/unmute ++ Multiple master management ++ Separate streams to allow more controls?+ \ No newline at end of file diff --git a/technical/connection-manager.md b/technical/connection-manager.md @@ -0,0 +1,66 @@ +# Introduction + +The connection manager is the first piece of the group chat features. This class manages connections to peers and offer to the user multiplexed sockets to devices they want to connect. For example, if Alice wants to be connected to one of Bob's device to transfer 2 files, she will ask the ConnectionManager to open 2 channels (one per file) to Bob. This will give: + +```cpp + aliceAccount->connectionManager().connectDevice(bobDeviceId, "file://file1", + [](std::shared_ptr<ChannelSocket> socket) { + if (socket) { + // transfer first file + } + }); + + aliceAccount->connectionManager().connectDevice(bobDeviceId, "file://file2", + [](std::shared_ptr<ChannelSocket> socket) { + if (socket) { + // transfer second file + } + }); +``` + +Behind that, the ConnectionManager will first connect to Bob's device via the DHT (via ICE) and setup a TLS Socket. Then, it will ask for a channel, and when the channel is ready, inform Alice via a callback. For the second file, it will use the first socket and will just open a new channel (only needs 2 TLS packet, so it's fast) + +# DHT side + +It's the same as [calls](https://git.jami.net/savoirfairelinux/ring-project/wikis/technical/2.4.%20Let's%20do%20a%20call), see **Exchange ICE candidates**, **ICE negotiation**, **Encrypt the control socket** but only in TCP. + +However, when a side receives a new ICE request, the callback set by ` void onICERequest(onICERequestCallback&& cb);` is triggered. + +# Negotiating a new channel + +A channel is defined by an id (unique) and a uri (not unique). For example (1, 'git://*') + +When ready, the ConnectionManager considers that the channel 0 exists. This channel is called the *CONTROL* channel and is used to ask for new channels. + +The protocol used is pretty simple and looks like the RTP protocol: + +1. 16 bits are used to store the length of the body. +2. 16 bits for the channel id (destination) +3. body + +So all packets have a 32 bits len header. + +To ask for a new channel, the ConnectionManager will send a `ChannelRequest` object (msgpack is used to serialize the struct) in the channel 0 to send the id and the name of the new channel to the peer (with `isAnswer = false`). The peer will call the callback given with ̀ void onChannelRequest(ChannelRequestCallBack&& cb);` and will refuse or accept the request. If accepted, the peer will answer with a ChannelRequest with the same data (and ̀`isAnswer = true`) and then both peers callbacks will be triggered to inform that the ChannelSock is usable. + +# Closing a channel + +A *EOF* is transmitted for a channel if the length of the content is 0. + +# Structure of the connectionManager + +## Ownership + +1. A JamiAccount owns the ConnectionManager and have access to the ChannelSocket objects (shared_ptr owned with the MultiplexedSocket. +2. The ConnectionManager owns MultiplexedSockets and ICE objects +3. MultiplexedSockets owns the TLS transport and the ChannelSocket objects +4. ChannelSocket owns the data buffers + +## Roles + +1. ConnectionManager is used to manage connections to peers. +2. MultiplexedSockets are used to send data over the TLSSocket, read the incoming packets and manage channels. +3. ChannelSockets are used by the client to interact with the other peer. + +# Usage + +Scenarios are described in the corresponding unit tests (`test/unitTest/connectionManager/connectionManager.cpp`)+ \ No newline at end of file diff --git a/technical/guidelines/Banned-Contact.md b/technical/guidelines/Banned-Contact.md @@ -0,0 +1,48 @@ +Following information are here for development purposes and may not +reflect the current state of any Ring client. + +Introducing scenario +-------------------- + +Let's explain banned contacts with a simple scenario: + +Alice and Jessica are friends, and like all good friends do, they use +Ring to communicate. They are both Ring contact of each other, so Alice +is a contact of Jessica and Jessica is a contact of Alice. Some day +however, Jessica does something really bad to Alice and Alice doesn't +want to hear from her anymore. Instead of removing Jessica from her Ring +contacts -- which would still allow Jessica to send her contact +requests, Alice *bans* Jessica. + +**So, what does it mean ?** + +In the daemon +------------- + +Jessica *won't be notified that she was banned by Alice*. + +As a *banned contact* of Alice, Jessica *won't be allowed to contact her +anymore*, in any way. Text messages, voice or video calls won't be +acknowledged or answered in any way. Alice won't even be aware that +Jessica tried to contact her. + +Banned contacts are synched across linked devices like other contacts. + +In Jami clients (recommended implementation) +-------------------------------------------- + +As long as Jessica is a banned contact, the conversation with Jessica +doesn't appears in the conversations list. The conversation history, +however, is not deleted. Jessica appears in Alice' account banned +contact list. Alice might also find/open the conversation by performing +an exact search for Jessica' username. + +Alice can un-ban Jessica from the conversation or the banned contact +list. + +The search result and the conversation indicates Jessica' banned status. +Alice must unban Jessica to send her a message, call her or otherwise +interract with her. + +Alice can still delete the conversation history using a *Delete History* +button.+ \ No newline at end of file diff --git a/technical/guidelines/Coding-rules.md b/technical/guidelines/Coding-rules.md @@ -0,0 +1,8 @@ +**This page gives rules and/or guidances to all +developers that want to integrate some code to Jami ** + +C++ format rules are defined by this clang-format file: +https://git.jami.net/savoirfairelinux/ring-daemon/blob/master/.clang-format + +All developers are recommended to format their code using the script in `jami-project/scripts/format.sh`. +This is done automatically (as a pre-commit hook) when using `./make-ring.py --init`+ \ No newline at end of file diff --git a/technical/guidelines/Kde-guidelines.md b/technical/guidelines/Kde-guidelines.md @@ -0,0 +1,99 @@ +# Coding Style + +- Indentation using 3 spaces +- 0 Indentation for preprocessor +- Class declaration intentation = 0 space for + public/private/protected/Q\_SIGNALS, 3 space for the methods, + friends and attributes +- Every function/method need a oneliner doxygen description using + "///" +- Function need to be ordered by 1) usage (history, contact, slots) 2) + alphabetical order +- Lines related to each other (like many setter on the same object + need to be aligned down to the ";" +- Oneliner if-elseif/switch/connect/tables need to have an header + using /\* HEADER \*/ where \*/ close of the end of the table, also + align the lines +- Header file function need to be classified and aligned +- License header is necessary for all source files +- Attribute are named with prefix "m\_", then "s" for static, then "p" + for pointer, "l" for list, "h" for hash, "f" for function/lambdas + then the first letter is uppercase +- Includes are located in the .cpp and only "class" and "namespace" + are declared in the .h (where applicable) +- Includes are sorted by libraries (Qt -> QtCore/QtGui/QtDbus, KDE + -> Akonadi/KABC, Dring) +- Setters take const references as parameter +- Destructor are always virtual +- C++11 is allowed +- Every class need a oneliner "///@class <Classname> <description>" + description +- Use QDebug +- Align everything as much as possible +- one line "if" block and "()?:;" are allowed and recommended +- Minimize the number of lines, but add as many white lines are + necessary +- Header ifnedef need to be the class upper name with upper replaced + with \_ and \_H as suffix +- functions/methods need to end with } //function name (more than 10 + lines only) +- "using" (namespace keyword) usage is prohibited +- Every \#endif need to make explicit what it is closing "\#endif + MY\_CLASS\_H" +- Code block have the "{" at the end of line while mothods use "\\n{" +- Use enum class where possible +- Always use the Full::Path for enums +- Expose library objects as "real" qobjects (with Q\_PROPERTY, + Q\_ENUM, Q\_FLAGS, Q\_INTERFACE and Q\_DECLARE\_NAME) +- Follow Qt getter/setter naming conventions +- Follow Krazy2 advices +- string should be stored in a static struct or class as "constexpr + static const char\*". C++14 will add +- daemon constants should always be stored in the .h in pure static + structs (with embedded structs to emulate namespace) +- avoid keeping maps and list around when they represent an object. +- transform string into enum class literal or object as soon as + possible in the code +- use const and static variables in methods where applicable +- Classes should not expose multiple mutually exclusive boolean + properties and use enum classes instead. (Example: isAudioOnly() + and hasVideo()) +- Friendship ("friend" keyword) is accepted between the model and + their data object to avoid exposing factory interfaces +- Private classes declaration need to be at the top of the CPP file +- Always an empty line before and after if/else, for, while, do, + lambda + +# Necessary Constraints + +- D-Pointer (public class pointer to private structure) need to be + called d\_ptr <http://qt-project.org/wiki/Dpointer> +- Q-Pointer (not to be confused with QPointer) need to be called + q\_ptr <http://qt-project.org/wiki/Dpointer> +- If a private header is necessary, then add it to the private folder + with a classname\_p.h name. Do not install those, never \#include + them in other .h, only .cpp + +# Design Guideline + +- No hack +- No logic that belong to the daemon +- Use the state machine, don't add useless functions +- Coherent code is not a very high priority, but avoid making things + worst +- Avoid using dbus call outside of the library, if you have to, your + doing it wrong + +# Current Design + +``` +=========================QML plasmoid====== <- Abstact frontend using the dataengine model +=====KDE frontend=========Dateengine======= <- Library frontend +==============KDE library================== <- Common GUI code and contact implementation +=============libringclient================= <- Dbus to C++ object + state machine + frontend model, -GUI LESS- +==================DBUS===================== +==================DRing==================== +==================PJSIP==================== +===================UDP===================== +=================SERVER==================== +```+ \ No newline at end of file diff --git a/technical/guidelines/Libringclient-Coding-Rules.md b/technical/guidelines/Libringclient-Coding-Rules.md @@ -0,0 +1,102 @@ +**This page gives rules and/or guidances to all +developers that want to integrate some code to the LibRingClient (a +Daemon Middleware for Desktop Clients).** + +- Following rules apply to this sub-project only. Others may re-use + this one in parts or whole, have their own, etc. Refer to their + pages for that. + +<!-- --> + +- Some rules are <u>strict</u> and are **strongly marked** or using + modals like SHALL or MUST. + + +**!!!Not respecting these rules results in code-review rejection!!!** + +- Some are guidance or recommendation (using modals like SHOULD or + RECOMMEND), it's up to the developer to balance them with his own + way to write code. + +<!-- --> + +- Rules are based on following well known ones, we've modified them + for this project: + - [Google C++ + CodingStyle](https://google.github.io/styleguide/cppguide.html) + for the general C++ domain + - [Qt Coding Conventions](https://wiki.qt.io/Coding_Conventions) + and [Qt Coding Style](https://wiki.qt.io/Qt_Coding_Style) as + this project use QtCore as framework. + +<!-- --> + +- This wiki is a permanent Work-In-Progress system. **So not all rules + are written yet.** That doesn't mean we don't have them yet (we = + core developer members). + +<!-- --> + +- We've changed some rules during the project. We update the code + iteratively, so tt's extremly possible to find whole files or parts + of code that not using these rules. + +In such situation, if you need to change only few lines in a "logic" +block (i.e. a class declaration, a function body, ...) keep the local +rules to make easier the review: we can see what's the important change +without being disturbed by coding rules refactoring. If you need such +action, use a dedicated patch with a clear mention in the commit +message. + +--- + +## Rules + +### **Language Standard** + +We SHALL use the **C++14 standard** syntax only. + +For gnu-gcc and clang, use compiler command line option `-std=c++14` to +enforce this rule. + +### **Maximum Line Length** + +You SHOULD keep your line length under the **120 characters limit**. + +This limitation is mainly due that integrators used splited view to +compare files, and keep it below 120 cols make easier to compare them, +even on wide screen. Configure your editing tools to this limit. + +If keeping this rule make the code worst to read or non-valid than +exceed the limit, the rule can be transgressed. These why we've +**exceptions** to this rule: + +- If a comment line contains an example command or a literal URL + longer than 120 characters, that line may be longer than 120 + characters for ease of cut and paste. +- A raw-string literal may have content that exceeds 120 characters. + Except for test code, such literals should appear near top of + a file. +- An \#include statement with a long path may exceed 120 columns. + +### **Non-ASCII Characters** + +Non-ASCII characters should be rare, and MUST use **UTF-8** formatting. + +You SHALL NOT use the C++11 char16\_t and char32\_t character types, +since they're for non-UTF-8 text. For similar reasons you also SHALL NOT +use wchar\_t (unless you're writing code that interacts with the Windows +API, which uses wchar\_t extensively). + +### **Spaces vs. Tabs** + +In code files (i.e. .cpp or .h), you SHALL use **spaces to indent**. + +One indent level is 4 (four) consecutive spaces. You should set your +editor to emit spaces when you hit the tab key and when you save your +file. + +### **Testing** + +Each patch for LRC should be accompanied by a test. CppUnit is used to +write tests.+ \ No newline at end of file diff --git a/technical/guidelines/Qt-and-QML-coding-guidelines.md b/technical/guidelines/Qt-and-QML-coding-guidelines.md @@ -0,0 +1,111 @@ +# Qt/c++ +## Signal and slot naming +Both signals and slots should use camelCase. +A signal should use the simple past tense or past participle of some verb, likely prefixed by a short subject. A corresponding slot should be the signal prefixed with the word "on" and not the word "slot". Here are some examples: +```c +// correct +signal: + void shutdownScheduled(); + void navigationRequested(); +... +slot: + void onShutdownScheduled(); + void onNavigationRequested(); + +// incorrect +signal: + void shutdown(); + void navigateToHomeView(); +... +slot: + void onShutdown(); + void slotNavigateToHomeView(); +``` + +--- +# QML +# Code formatting +The Qt 5.15.0 version of qmlformat has some issues dealing with comment sections and currently does not discriminate against max columns, so we will continue to format using these guidelines for now. +The following is a comprehensive sample component, adapted from https://doc.qt.io/qt-5/qml-codingconventions.html, that attempts to illustrate the ideally formatted component. +```javascript +/* + * authored by/copyright ... + */ + +// Id always on the first line, and always present and called root +// for root component. Only present if referenced for children. +// Multi-line comments use '//'. +// There should always be a brief description of a component. +// Multi-line comments start with a capital letter and end with +// a period. +Rectangle { + id: root + + // property declarations + property bool thumbnail: false + property alias image: photoImage.source + + // signal declarations + signal clicked + + // javascript functions + function doSomething(x) { + return x + photoImage.width + } + + // object properties + color: "gray" + // try to group related properties together + x: 20 + y: 20 + height: 150 + // large bindings + width: { + // Always space all operators, and add a single space + // before opening brackets. + if (photoImage.width > 200) { + photoImage.width + } else { + height + } + } + + // child objects (contentItem, defaults, etc. first) + Rectangle { + id: border + anchors.centerIn: parent + color: "white" + + Image { + id: photoImage + anchors.centerIn: parent + } + } + + Text { + anchors.centerIn: photoImage + // Use the ellipsis character (…) for manual ellision within + // UI text and don't translate untranslatables. + text: qsTr("an emoji: %1 and a number: %2 …").arg("💡").arg(42) + } + + // states + states: State { + name: "selected" + PropertyChanges { + target: border + color: "red" + } + } + + // transitions + transitions: Transition { + from: "" + to: "selected" + ColorAnimation { + target: border + duration: 200 + } + } +} +```+ \ No newline at end of file diff --git a/technical/guidelines/Qt-and-QML-testing-tools.md b/technical/guidelines/Qt-and-QML-testing-tools.md @@ -0,0 +1,58 @@ +# C++ + +## Google Test +Google's c++ test framework. + +### Installation +- Ubuntu / Debian: +`apt install googletest libgtest-dev` + +## Example main.cpp +``` +#include <gtest/gtest.h> + +TEST(Test, Test1) +{ + EXPECT_EQ(0, 0); // OK + EXPECT_EQ(1, 0); // ERROR and continues + ASSERT_EQ(0, 0); // OK + ASSERT_EQ(1, 0); // ERROR and stops execution +} + +int main(int argc, char *argv[]) +{ + ::testing::InitGoogleTest(&argc, argv); + return RUN_ALL_TESTS(); // Runs Test1 and any other included test +} +``` + +# QML + +## QtQuickTest + +### Installation +- Ubuntu / Debian: `apt install qml-module-qqtest libqt5quicktest5` + +- Example main.cpp +``` +#include <QtQuickTest/quicktest.h> +#include <QQmlEngine> + +class Setup : public QObject +{ + Q_OBJECT + +public: + Setup() {} + +public slots: + + void qmlEngineAvailable(QQmlEngine *engine) + { + // Code to be run before the tests + } +}; + +QUICK_TEST_MAIN_WITH_SETUP(testqml, Setup) +#include "main.moc" +```+ \ No newline at end of file diff --git a/technical/guidelines/UI-and-UX-development.md b/technical/guidelines/UI-and-UX-development.md @@ -0,0 +1,23 @@ +The purpose of this page is to act as a reference during UI/UX development and to help synchronize the various Ring clients. + +As a start, we should post screenshots from all important Ring views. + + +# Screenshots + +**Disclaimer: This page is not up-to-date!** + +Part | GNOME | OSX | WIN32 | UWP | Android +-----|-------|-----|-------|-----|-------- +Main |  | TODO |  | TODO | TODO +New account creation (initial) |  | TODO |  | TODO | TODO +New account creation (main) |  | TODO |  | TODO | TODO +new account (clicked next) | Text with spinner + confirmation/error message | TODO |  | TODO | TODO +Link device (initial) |  | TODO |  | TODO | TODO +Link device (explanatory) |  | TODO |  | TODO | TODO +Link device (pin + password) |  | TODO |  | TODO | TODO +Chat |  | TODO | TODO | TODO | TODO + +# Future + +[Ring_GNOME_client_mock-ups.pdf](/uploads/698997f2126004a2e82b3d6e32bcdae4/Ring_GNOME_client_mock-ups.pdf)+ \ No newline at end of file diff --git a/technical/identifiers.md b/technical/identifiers.md @@ -0,0 +1,60 @@ +Jami identifiers +---------------- + +There are many identifiers in Jami. We need to unify the naming of these +identifiers between all implementations. This page reference various kind of +identifiers used in Jami with relevant examples. + +- **Jami Infohash** or **Jami public key fingerprint** : a public key + fingerprint such as `3d1112ab2bb089370c0744a44bbbb0786418d40b` +- **Registered name** : a username associated to a Jami Infohash on + the blockchain such as `jeandupont` +- **URI** : a Jami or SIP URI such as + `ring:3d1112ab2bb089370c0744a44bbbb0786418d40b` or `ring:jeandupont` + or `<sip:nnnnn@host:5060>`. Must be compliant with [rfc3986](https://tools.ietf.org/html/rfc3986). If it's a SIP URI, it must be compliant with [rfc3261#19.1](https://tools.ietf.org/html/rfc3261#section-19.1). +- **Canonical URI** : `ring:3d1112ab2bb089370c0744a44bbbb0786418d40b` or `sip:nnnnn@host:5060`. The most simplified form of the URI. Registered name must be resolved, doesn't include <> brackets or display name. Prefixed with the scheme (`ring:` or `sip:` or `sips:`). +- **User ID**: registered name (preferred) or public key fingerprint. User-facing identifier for an account public key. +- **Display name** or **Profile name** : an editable user-defined + profile name such as `Jean Dupont`. + +When displaying a contact: + +``` + _____ +|photo| Display name or User ID +|_____| User ID +``` + +- If Display name is empty, User ID is shown instead +- If both lines have the same content, only the first line is + displayed +- If no photo is available and a registered name (ring) or display name (sip) is available, the first letter of this name can be used to generate a placeholder. Otherwise a generic placeholder is used. +- If no photo is available, a placeholder with an Canonical URI-specific background color can be used: + +```java +final int[] contactColors = { + color.red_500, color.pink_500, + color.purple_500, color.deep_purple_500, + color.indigo_500, color.blue_500, + color.cyan_500, color.teal_500, + color.green_500, color.light_green_500, + color.grey_500, color.lime_500, + color.amber_500, color.deep_orange_500, + color.brown_500, color.blue_grey_500 +}; + +int generateAvatarColor(String canonicalUri) { + if (isEmpty(canonicalUri)) + return R.color.grey_500; + String h = md5(canonicalUri); + if (h == null) + return R.color.grey_500; + int colorIndex = Integer.parseInt(h.charAt(0) + "", 16); + return contactColors[colorIndex % contactColors.length]; +} +``` + +Color values are from the material palette: https://material.io/tools/color + + ++ \ No newline at end of file diff --git a/technical/important-RFCs.md b/technical/important-RFCs.md @@ -0,0 +1,42 @@ +# SIP + +Reference: <http://tools.ietf.org/html/rfc3261> + +Valuable updates and extensions: + ++ SIP Re-INVITE: <http://tools.ietf.org/html/rfc6141> + +# ICE + +Reference: <http://tools.ietf.org/html/rfc5245> + +# SDP + +Reference: <http://tools.ietf.org/html/rfc4566> + +How to use SDP: <http://tools.ietf.org/html/rfc3264> +Valuable updates and extensions: + ++ SDP and IPv6: <http://tools.ietf.org/html/rfc6157> ++ SDP for SRTP: <http://tools.ietf.org/html/rfc4568> + + +# RTP + +Reference: <http://tools.ietf.org/html/rfc3550> + +Valuable updates and extensions: + ++ RTP and RTCP on same port: <http://tools.ietf.org/html/rfc5761> + + +# SRTP + +Reference: <http://tools.ietf.org/html/rfc3711> + +Valuable updates and extensions: + ++ DTLS for SRTP: <https://tools.ietf.org/html/rfc5763> et <http://tools.ietf.org/html/rfc5764> ++ AES-192 and AES-256: <https://tools.ietf.org/html/rfc6188> ++ AES-GCM: <https://tools.ietf.org/html/rfc7714> + + \ No newline at end of file diff --git a/technical/improving-jami's-quality.md b/technical/improving-jami's-quality.md @@ -0,0 +1,84 @@ +## Unit-tests + +* It is harder to make unit-test on Jami project because of the race conditions on multi-level dependance. + +* There is about 30 unit-tests and 26% coverage. Due to Jami high demand to give new functionalities to user quickly, they are not maintained by the developers or by a QA dept. + +* We use lcov for the coverage, you can find the lcov’s configuration in the daemon’s Makefile.am. Also, the coverage can be found at https://docs.jami.net/coverage/ + +* A system needs to be implemented to start convincing the team to make a unit-test for new code before merging + +* You can launch them by doing ‘make check’ in the daemon folder or separately in the unit-test folder with gdb: ‘gdb ut_media_encoder’ + +* The environment needs to be set with ‘--disable-shared’ during the ’./configure’ command + +## Framework Tests + +* You can find framework tests in the daemon’s Makefile.am and lunch it with ‘make integration’. This calls jami_test.py script in the tools/dringctrl folder. It uses dringctrl.py and controller.py which let you control Jami through bash. + +* This makes a series of calls to assure jami’s opendht network is stable. + +* Other framework tests need to be implemented in the future to tests Jami’s functionalities as a whole. + +## Integration tests + +* Each commit goes through integration tests in dockers on the build machines you can find the details at: jenkins.jami.net + +* Code-review is made by a fellow developer, sometimes the code is reviewed by the same developer, this should be avoided to emphasize Linus’ law. The ‘Jenkins verified’ label is sometimes discarded and replaced by +1 from a developer, this should also be avoided. + +* Sonarqube lets Jenkins build Jami and verify linting. You can find filters and results at: sonar- jami.savoirfairelinux.net Sonar uses clang-tidy as a preprocessor linting compilator, you can find clang’s filters in .clang-tidy file in the daemon folder. + +* On sflvault sonarqube can be found at service m#2637 and admin logins at service s#7169 + +## Doc and feedback: + +* You can find all the documentation on docs.jami.net + +* Issues are made by developers or users on git.jami.net + +## Monitoring + +* A script is called every 30 minutes on a virtual machine jami-monitorpeervm-01. You can find it on sflvault service s#7209 and is calling an other client viratual jami- monitorpeer-02 (service s#7224). A series of calls is being made and it returns the failure rate. You can find all the details at https://wiki.savoirfairelinux.com/wiki/Jami-monitorpeervm-01.mtl.sfl. + +* If needed, the manual command is ./script.sh –peer 031acbb73f2a3385b2babc7161f13325be103431 + +* It traces a real time point by point graph on https://monitoring.savoirfairelinux.com/grafana/dashboard/script/dyndash.js?host=jami-monitorpeervm-01.mtl.sfl&service=Check%20JamiCall&panelId=1&fullscreen&orgId=1 + +## Smoke tests + +Before each releases every clients MUST past a list of scenarios. + +Scenarios are described here: +[SmokeTestsJami.ods](uploads/264048655e2b25e1c1c207b84febec95/SmokeTestsJami.ods) + +They are reviewed by QA dpt. before sending it to the developers if needed. + +If a release contains a network commit that has been merged, the QA dept. Should be able to automate the different connectivity tests (as descibed below in Calls configurations) + +### Calls configurations. + +This is the list of network configurations that need to be tested: + +(IPv4 | IPv6) + (TURN | !TURN) + (STUN | !STUN) + (UPnP | !UPnP) for both sides. + +If both sides are IPv4 only without TURN/STUN/UPnP, the call should be only local. + +## Special note: FDroid + +The script to generate MR is in the client-android repo (fdroidMergeRequest.sh) + +## What needs to be done + +* Push coverage closer to 60% + +* Establish a system within the team to assure maintenance and creation of unit-tests. + +* Each major functionality should be tested as whole by adding a framework test (i.e. making sure a message was received, the call was ended well on both side, etc...) + +* Each new functionality should be tested on each platform before merging it to reduce regression + +* Integrate sonarqube on each client + +* Automate the testing of Jami’s behaviour on network compatibility + +* Make a make_ring.py script adaptable to windows also+ \ No newline at end of file diff --git a/technical/index.rst b/technical/index.rst @@ -0,0 +1,32 @@ +########## +Technical Reference +########## + +This chapter includes in-depth explanations of how Jami is +designed. It also serves as a reference for contributors. + +.. toctree:: + :maxdepth: 2 + + identifiers + APIs + basic-features/index + advanced-features/index + certificates + name-server-protocol + +If you're reading this, you probably want to either contribute to one +of the projects or to implement your own client. There are three main +layers in this project: + +1. `OpenDHT <https://opendht.net>`_ for p2p communication. You can + interact with this library like any Cpp library or with the python + wrapper or via the `REST + API <https://github.com/savoirfairelinux/opendht/wiki/REST-API>`_. +2. The Jami daemon, which is the main part. It's the part which does + all the logic for Ring and interacts with OpenDHT, pjsip, ffmpeg + and all libraries used and implement the whole protocol. If you + want to do a client, we recommend that you implement your client on + top of this daemon and use one of the many APIs + (REST/dbus/libwrap/JNI). +3. The client part, which is basically the frontend. diff --git a/technical/name-server-protocol.md b/technical/name-server-protocol.md @@ -0,0 +1,209 @@ +The protocol used by Jami to query and register a name is based on an +HTTP +[REST](https://en.wikipedia.org/wiki/Representational_state_transfer) +API answering requests with JSON documents and regular HTTP status +codes. + +The public nameserver is hosted at `ns.jami.net` and uses a blockchain as +its backend. Another implementation could use any other database or +directory service making the nameserver protocol reusable. + +Rules on name formatting +------------------------ + +Usernames are checked by a regex to ensure some rules about their +format: + +- Length must be between 3 and 32 characters +- Those characters must be alphanumerical with underscore `_` being + also accepted. + +Querying a name +--------------- + +This is the main service provided by a name server. It enables to get +the RingID corresponding to a username. + +### Request + +A request for the name `foobar` is a `GET` request with +`/name/`*`foobar`* as the URI. + +### Response (Success) + +If the name is found, a response with status code `200` `OK` must be +sent to the client with a `Content-type` field set as +`application/json`. + +The body is a JSON documents with 2 string attributes : `name` and +`addr`. `name` is equal to the one requested and `addr` is an +hexadecimal representation of the RingID prefixed with `0x`. + +In our example, the JSON answer would be: + +``` {.javascript} +{ + "name":"foobar", + "addr":"0x29347542eb07159f316577e1ae16243d152f6b7b" +} +``` + +### Response (Not found) + +If the name is not found, a response with status code `404` `Not` +`Found` must be sent to the client with a `Content-type` field set as +`application/json`. + +The body is a JSON documents with 1 string attribute : `error`. This +attribute is filled with an error message that explains the error (and +could be displayed in the client in the future). + +On the reference implementation, the returned document is: + +``` {.javascript} +{ + "error":"name not registred" +} +``` + +Querying an address +------------------- + +This service is a reverse lookup. You query for an address and a +username is returned if one is registered on the name server. + +### Request + +A request for the ID `ring:29347542eb07159f316577e1ae16243d152f6b7b` +is a `GET` request with +`/addr/`*`29347542eb07159f316577e1ae16243d152f6b7b`* as the URI. + +### Response (Success) + +If the address corresponds to a username, a response with status code +`200` `OK` must be sent to the client with a `Content-type` field set as +`application/json`. + +The body is a JSON documents with 1 string attribute : `name`. The value +of this field is the name registered to this address + +In our example, the JSON answer would be: + +``` {.javascript} +{ + "name":"foobar" +} +``` + +### Response (Not found) + +If the address is not found, a response with status code `404` `Not` +`Found` must be sent to the client with a `Content-type` field set as +`application/json`. + +The body is a JSON documents with 1 string attribute : `error`. This +attribute is filled with an error message that explains the error (and +could be displayed in the client in the future). + +On the reference implementation, the returned document is: + +``` {.javascript} +{ + "error":"address not registred" +} +``` + +Registering a name +------------------ + +This part of the protocol is used to register a new name/address pair. +It is used on the main public registry but may be optional in a custom +implementation. + +### Request + +A request for registering the name `foobar` is a `POST` request with +`/name/`*`foobar`* as the URI. The header attribute `Content-type` must +be set to `application/json`. + +The body of the request is a JSON document with 2 string attributes: +`addr` and `owner`. `addr` contains the RingID prefixed with `0x` and +`owner` is the name to be registered. + +An example for `foobar` could be: + +``` {.javascript} +{ + "addr":"0x29347542eb07159f316577e1ae16243d152f6b7b", + "owner":"foobar" +} +``` + +### Response (Success) + +If the name/address pair is successfully registered, a response with +status code `200` `OK` must be sent to the client with a `Content-type` +field set as `application/json`. + +The body contain a JSON document with 1 boolean attribute `success` set +to `true`. + +As an example: + +``` {.javascript} +{ + "success":true +} +``` + +Further attempts to query the name or the address should then be +successful. + +### Response (Bad request) + +If the registration cannot be achieved because of an error in the +request (formatting, missing attribute, etc.), a response with status +code `400` `Bad` `Request` must be sent to the client with a +`Content-type` field set as `application/json`. + +The body is a JSON documents with 2 attributes: `success` which is a +boolean and `error` which is a string. `success` is set to `false` and +`error` is filled with an error message that explains the error (and +could be displayed in the client in the future). + +For an invalid formatting of the username, the body could be: + +``` {.javascript} +{ + "success": false, + "error": "invalid name" +} +``` + +### Response (Forbidden) + +If the registration cannot be achieved because the name is already +taken, a response with status code `403` `Forbidden` must be sent to the +client with a `Content-type` field set as `application/json`. + +The body is a JSON documents with 3 attributes: `success` which is a +boolean set to `false`, `name` and `addr` which are both strings +replicated from the original request. + +Registering `foobar`, with it being already registered, would lead to +the following response: + +``` {.javascript} +{ + "success": false, + "name":"foobar", + "addr":"0x29347542eb07159fdeadbeefae16243d152f6b7b" +} +``` + +Some links +---------- + +- [ring-nameservice](https://github.com/savoirfairelinux/ring-nameservice): + reference NodeJS implementation used by `ns.ring.cx` and querying an + Ethereum node.+ \ No newline at end of file diff --git a/technical/protocol.md b/technical/protocol.md @@ -0,0 +1,501 @@ +### Persisting the account + +Persisting a Ring account private key and certificate is implementation +defined. + +Access to a saved Ring account private key must be authenticated and +authorized. Authentication and authorization method to access the +account private key is implementation defined. + +### Adding a device to a Ring account + +*See [RFC 5280](https://tools.ietf.org/html/rfc5280)* + +A **device** is defined by an RSA key pair with a key length of at least +4096 bits. + +A **device certificate** is defined as an x509 certificate whose subject +is a device public key, signed with an account private key. The +certificate MUST be valid. The issuer UID field MUST be the hexadecimal +form of the account public key fingerprint. + +Persisting a device private key and certificate is implementation +defined. Access to a saved device private key should be authenticated. +Authentication method to access the device private key is implementation +defined. + +### Removing a device from a Ring account + +A device can be "removed" from a Ring account through revocation of the +device certificate. Revoked device certificates are added to one or more +standard x509 Certificate Revocation List (CRL). CRLs for revoked device +must be valid and signed with the corresponding CA key, which is the +Ring account private key. + +### Account transmission format + +The **account archive format** defines how to serialize an account +private key for transmission, for instance to sign a new device +certificate. + +The account archive is an encrypted JSON object with the following +structure: + +``` +{ + "ringAccountKey": (PEM-encoded account private key string), + "ringAccountCert": (PEM-encoded account certificate string), + "ringAccountCRL": (PEM-encoded account CRL string) +} +``` + +The JSON object can contain additional implementation-defined key-value +pairs. Implementation-defined key names shouldn't start with "ring". + +The string JSON object is encrypted using a key defined as : + +``` +salt = PIN + timestamp +key = argon2(password, salt) +``` + +Where PIN is a random 32bits number in hexadecimal form, "+" is string +concatenation, timestamp is the current UNIX timestamp divided by 1200 +(20 minutes) and password is a user-chosen password. + +The PIN should be shown to the user to be copied manually on the new +physical device along with the password. + +Contacting another account +-------------------------- + +### ICE descriptor exchange over OpenDHT + +- **Listening for incoming calls** + +A device listens for incoming call by performing a listen OpenDHT +operation on + +`h("callto"+deviceID)` + +where h is SHA1, "+" is the string concatenation and deviceID is the +hexadecimal form of the deviceID. + +Received OpenDHT values that are not encrypted or not properly signed +must be dropped. The value must be encrypted with the called device +public key and signed with the calling device private key according to +OpenDHT specifications. + +- **Sending the Initial Offer** + +*See [RFC 5245](https://tools.ietf.org/html/rfc5245)* + +RFC 5245 defines ICE (Interactive Connectivity Establishment), a +protocol for NAT traversal. + +ICE is used in Ring to establish a peer-to-peer communication between +two devices. + +The calling device gathers candidates and build an Initial Offer +according to the ICE specifications and starts the ICE negotiation +process. + +The calling device puts the encrypted ICE offer (the Initial Offer) on +the DHT at h("callto"+deviceID) where deviceID is the hexadecimal form +of the called deviceID. + +- **ICE serialization format** + +ICE messages exchanged between peers during a call setup use following +format. An ICE message is a chunk of binary data, following +[msgpack](http://msgpack.org/) data format. + +This protocol is a compound of msgpack values, successively packed in +this order: + + ++ an integer giving the version of ICE message format protocol used for the rest of the data. Current defined protocol version is **1**. ++ a 2-elements array of strings of the ICE local session ufrag and the ICE local session password ++ an integer giving the number of components in the ICE session ++ an array of string, of the previous number entries, where each string describe the ICE candidate, formated as an "a=" line (without the "a=" header) described in [rfc5245, section 4.3](https://tools.ietf.org/html/rfc5245#page-26) + +- **Sending the Answer** + +Upon reception of the encrypted and signed Initial ICE Offer (through +the listen operation), a called device should perform authorization +checks of the calling device, identified as the Initial Offer signer. +Authorization rules are implementation defined, but a typical +implementation would authorize known or trusted contacts. + +If the calling device is not authorized or if for any implementation +defined reason the called device refuses the incoming connection +request, the called device must ignore the Initial Offer and may log the +event. + +If the called device authorizes the caller and wish to accept the +connection it must build an ICE answer, start the ICE negotiation +process and send the encrypted and signed ICE answer at the same DHT +key. + +### DTLS negotiation + +Once a peer-to-peer communication channel has been established, the +called device listens on it for incoming DTLS connections (acting as a +DTLS server) while the caller initiates an outgoing DTLS connection +(acting as a DTLS client). + +The DTLS communication must be RFC6347 compliant +([1](https://tools.ietf.org/html/rfc6347)). + +Peers must only support PFS cypher suites. The set of supported cypher +suites is implementation defined but should include at least +ECDHE-AES-GCM (TODO: specify the exact suites recommended to support). + +During the DTLS handshake, both peers must provide their respective +device certificate chain and must authenticate the other peer, checking +that its public key is the same used during the DHT ICE exchange. + +### SIP call + +*See [Important\_RFC](Important_RFC "wikilink")* + +Once an encrypted and authenticated peer-to-peer communication channel +is available, the SIP protocol [2](https://tools.ietf.org/html/rfc3261) +must be used to place a call and send messages. The caller might send a +SIP INVITE as soon as the DTLS channel is established. + +The SIP implementation must support ICE and SRTP. + +Supported codecs are implementation defined, but Ring clients should +support the Opus audio coded and the H264 video codec. + +SRTP must be used when negotiating media with SIP, using a new random +key for each media and each negotiation. ICE should be used when +negotiating media with SIP. + +Cryptographic primitives +------------------------ + +### Password stretching + +*See [Argon2 +specifications](https://github.com/P-H-C/phc-winner-argon2/blob/master/argon2-specs.pdf)* + +Passwords are stretched using argon2i using t\_cost = 16, m\_cost = +2\^16 (64 MiB), mono-threaded, to generate a 512 bits hash. + +The result is then hashed again using SHA{1, 256, 512} depending on the +requested key size. + +### Encryption + +##### Using a provided key (128, 192 or 256 bits) + +Encryption uses standard AES-GCM as implemented by Nettle using a random +IV for each encryption. + +##### Using a text password + +The password is stretched to generate a 256 bits key and a random salt +of 128 bits. + +The input data is encrypted using AES-GCM (see above) and the salt is +appended at the beginning of the resulting cypher-text. + +##### During a call + +Audio/video data are exchanged using encrypted RTP channels between +peers. + +The protocol is a classic SRTP, with following supported crypto suites: + +- Ring account force AES\_CM\_128\_HMAC\_SHA1\_80 +- SIP can use AES\_CM\_128\_HMAC\_SHA1\_80 or + AES\_CM\_128\_HMAC\_SHA1\_32 + +The master key and salt is a random number, different for each call. On +call's master key is constant during the full live of a call. + +The keys are exchanged using SDES method: keys are written into the SIP +SDP messages during the SIP INVITE negotiation. When SDES is used, Ring +forces the underlaying transport to be secure (encrypted) to not +disclose these keys. Ring supports DTLS natively for SIP and Ring +accounts for such. The call cannot be done if this condition is not +fulfilled. + + + + +### Outgoing and Incoming calls + +- Contactable addresses (i.e. IP addresses) of the user are given to + peer only: + - When we call a peer (outgoing call). + - When a **trusted** peer is calling (incoming call). +- All combination forms of how a specific device can be contacted is + summarized by a ICE message: + - *[RFC 5245](https://tools.ietf.org/html/rfc5245)* defines ICE + (Interactive Connectivity Establishment), a protocol for + NAT traversal. + - ICE is used in Jami to establish a peer-to-peer communication + between two devices. + +#### Making an outgoing call + +1. The calling device gathers candidates and build an **Initial Offer** + according to the ICE specifications. +2. The calling device puts the encrypted ICE offer (the *Initial + Offer*) on the DHT at: + `h("`[`callto:"+DeviceID`](callto:%22+DeviceID)`)` where *h* is + SHA1, *+* is the string concatenation, *DeviceID* is in + hexadecimal form. +3. The calling device waits on the peer answer, with its own ICE + candidates lists. +4. At peer answer reception, the calling device starts the + ICE negotiation. +5. If the negotiation succeed, the process continues on a client-side + DTLS session establishment over the created ICE socket (see below). + +#### Listening for incoming calls + +1. A device listens for incoming calls by performing a listen OpenDHT + operation on `h("`[`callto:"+DeviceID`](callto:%22+DeviceID)`)` + where *h* is SHA1, *+* is the string concatenation and *DeviceID* is + in hexadecimal form. +2. At ICE *Initial Offer* reception, the called device **must** do a + security validation of the peer (see below). +3. If the security validation succeed, the called device starts the + ICE negotiation. +4. If the negotiation succeed, the process continues on a server-side + DTLS session establishment over the created ICE socket (see below). + +- *Note: OpenDHT drops values that are not properly encrypted or + signed, as specified by OpenDHT protocol.* + +#### ICE serialization format + +- ICE messages exchanged between peers during a call set up use + following format. +- An ICE message is a chunk of binary data, following + [msgpack](http://msgpack.org/) data format. + +``` +<8-bits unsigned integer> giving the version of ICE message format protocol used for the rest of the data, +<C++ std::pair of string> of the ICE local session ufrag and the ICE local session password, +<8-bits unsigned integer> giving the number of components in the ICE session, +<array of string> of the previous number entries, where each string describe the ICE candidate, formated as an "a=" line (without the "a=" header) described in [rfc5245, section 4.3](https://tools.ietf.org/html/rfc5245#page-26) +``` + +- **Current defined protocol is 1**: + +#### Security Validation of the Peer + +- Upon reception of the encrypted and signed Initial ICE Offer + (through the listen operation), a called device should perform + authorization checks of the calling device, identified as the + Initial Offer signer. +- Authorization rules are implementation defined, but a typical + implementation would authorize known or trusted contacts. +- If the calling device is not authorized or if for any implementation + defined reason the called device refuses the incoming connection + request, the called device must ignore the Initial Offer and may log + the event. +- If the called device authorizes the caller and wish to accept the + connection it must build an ICE answer, start the ICE negotiation + process and send the encrypted and signed ICE answer at the same + DHT key. + +#### DTLS negotiation + +- Once a peer-to-peer communication channel has been established by + ICE protocol, the called device starts a server-side DTLS session on + the ICE socket, while the caller starts a client-side DTLS session + on the other side of the ICE socket. +- The DTLS communication is + [RFC6347](https://tools.ietf.org/html/rfc6347) compliant using + gnutls library. +- To prevent peer certificates to be displayed in plain-text for the + call anonymity, the session handshake is done twice: + +1. A first handshake in **anonymous mode** to create a secure but + anonymous transport. +2. A second handshake in **certificate mode**, over the first one, to + prove the identity of peers. + +- Only PFS cipher suites are supported: + - The set of supported cipher suites is implementation defined but + should include at least ECDHE-AES-GCM. + - The actual cipher suites (in gnutls form) is: + - anonymous step: + `SECURE192:-KX-ALL:+ANON-ECDH:+ANON-DH:+SECURE192:-VERS-TLS-ALL:+VERS-DTLS-ALL:-RSA:%SERVER_PRECEDENCE:%SAFE_RENEGOTIATION` + - certificate step: + `SECURE192:-VERS-TLS-ALL:+VERS-DTLS-ALL:-RSA:%SERVER_PRECEDENCE:%SAFE_RENEGOTIATION` + +#### SIP signaling + +- Used over the DTLS session to signaling the call (vcard, media + negotiation, hangup, instant messaging, ...) +- Once an encrypted and authenticated peer-to-peer communication + channel is available, the [SIP + protocol](https://tools.ietf.org/html/rfc3261) must be used to place + a call and send messages. +- The caller might send a SIP INVITE as soon as the DTLS channel + is established. +- The SIP implementation must support ICE and SRTP. +- Supported codecs are implementation defined, but Jami clients should + support the Opus audio coded and the H264 video codec. +- SRTP must be used when negotiating media with SIP, using a new + random key for each media and each negotiation. ICE should be used + when negotiating media with SIP. + + +## extra + + +To add a new contact, you send them a trust request over the DHT +consisting of <check this> the account certificate. If they accept the +request, they will send you their account certificate + + +who do not know each other's IP address. + + + +So + + + + +## Usages + +- The JamiId: + - It's the DHT key where the list of account devices are published + and where all devices listen to synchronize on account + changes (i.e. adding or revoke a device). +- The Jami certificate RSA keys are used as long-term keys to + sign/encrypt/decrypt messages sent over the DHT: + - private key to sign-off and decrypt incoming messages and + device certificates. + - public key to encrypt messages (this is done by the message + issuer using the receiver public key). +- A device can be "removed" from a Jami account through revocation of + the device certificate: + - Revoked device certificates are added to one or more standard + x509 Certificate Revocation List (CRL). + - CRLs for revoked device must be valid and signed with the + corresponding CA key, which is the Jami account private key. + +### Long-term Storage + +- Why storing data? + +<!-- --> + +- - Jami needs to load certificates and key-pairs each time the + application is started. + - When Jami creates a new device, these information are also + needed, shared from another trusted device in a secure way. + - All platforms doesn't provide secure way to store data, Jami + supports this fact by encrypting data stored outside the + memory (i.e. on a file-system) using a user defined password + during the account creation. + +<!-- --> + +- These files are stored on user device (see below for details): + - a compressed and encrypted archive with private account data. + - the public certificates chain as a CRT file + - the device private key. + +#### Jami archive (export.gz) + +- Contains private account data. +- Currently transmitted over the DHT network when device is created + or revoked. +- It's a JSON compressed and encrypted file. + +<!-- --> + +- The current format is (could change at any time): + +``` +{ + ringCaKey: <base64 serialized PEM-encoded CA private key>, + ringAccountKey: <base64 serialized PEM-encoded Jami private key> + ringAccountCert: <base64 serialized PEM-encoded Jami certificate (public key)>, + ethKey: <base64 serialized of the optional 160-bits Etherium key used to register the JamiId on a public blockchain>, + ringAccountCRL: <base64 serialized of packet list of revoked device certificates PEM-encoded>, + ringAccountContacts: <JSON dictionary to export trusted user contacts> +} +``` + +- The JSON byte-stream is compressed using \*gzip\* algorithm. + +<!-- --> + +- Then the gzip-stream is encrypted using AES-GCM-256 symmetric cipher + with a 256-bits key. + - This key is derived from the user provided password, a PIN and a + timestamp, using + [Argon2](https://github.com/P-H-C/phc-winner-argon2) (a password + stretching and normalizer) as follow: + +``` +salt = PIN + timestamp +key = argon2(password, salt) + argon2 is the argon2i algorithm using t_cost = 16, m_cost = 2^16 (64 MiB), mono-threaded, to generate a 512-bits key and then hashed with SHA-256 to generate a 256-bits key. + PIN is a random 32bits number in hexadecimal form. + + is string concatenation operator. + timestamp is the current UNIX timestamp divided by 1200 (20 minutes). + password is a user-chosen password (given at account creation). +``` + +- The PIN should be shown to the user to be copied manually on the new + physical device along with the password to finish the device + creation process. +- NOTE: when exporting a file on the DHT or anywhere else, the daemon update the archive first, to write latest contacts. This is the reason why the password is needed when exporting (it's not just a copy of the archive somewhere else) + +#### Jami device certificate chain (ring\_device.crt) + +- PEM format +- Includes the Device certificate and parent certificates (Jami device + certificate and parents) + +#### Device private key (ring\_device.key) + +- PEM format +- not encrypted, we let the device file-system protect this file + +#### Exporting data (creating new devices) + +*TBD* + +### The DHT network + +Dedicated [ Jami distributed +network](Ring_distributed_network "wikilink") page. + +### Contact Request + +*TBD* + +### Instant Message + +*TBD* + +### Presence + +*TBD* + + +### Security / Privacy + +Jami provides perfect forward secrecy for calls and in call text +messages with different Eliptic Curve Diffie-Hellman key negociation at +every call. For out of call messaging single RSA-4096 is used. The +cryptography library used is GNUTLS + +More informations: + +- [Technical overview](technical/Technical-overview) of concepts and + protocols inside Jami diff --git a/technical/release-process.md b/technical/release-process.md @@ -0,0 +1,184 @@ +Each Ring sub-project has its own repository, build process, integration +cycle and so on. More over the **Ring architecture is split into two +independent modules**: LibRing *(daemon)* and clients. + +Having a unique revision is not a solution in this situation. The +retained idea is having a global "state" and **various updates per +module**. + +For consistency, **each Ring module has to follow the same process** as +described in following points. But not all modules have to be modified +in same time. + +------------------------------------------------------------------------ + +**PROCESS FLOW:** + +1 | 2 | 3 | 4 | 5 | 6 +:-----:|:-----:|:-----:|:-----:|:-----:|:-----: +Redmine Ticket |Repository Preparation |Testing |Push tags |Packaging | Advertisement + +------------------------------------------------------------------------ + +Redmine Ticket +-------------- + +Create a new Task on redmine attached to the release story, for the +right sub-module. Set the title to "Release Major.Minor.Micro", with the +appropriate version number. + +Repository Preparation +---------------------- + +**This section was outdated and removed** + +Testing +------- + +* Remove any existing Ring installations from your machine. +* Start with clean git tree by running `git clean -d -f -x` from the top +level directory of the project. +* Build and install the daemon and client, see How\\\_to\\\_build +* Run the test suite in daemon and client, on different distributions and +machines. +* Run manual tests + * Try registering and using different accounts. + * Try making calls between Ring and other free softphones (Ekiga, +Linphone), as well as hardware VoIP phones. + * To catch uninitialized values being used, memory leaks, invalid frees, +etc. run `valgrind --track-origins=yes --db-attach=yes ./bin/dring` + +Push tags +-------- + +`git push --tags` + +Packaging +--------- + +```bash +git clone ssh://tcohen@gerrit-ring.savoirfairelinux.com:29420/ring +cd ring +git checkout packaging-releases +``` + +### RPM + +```bash +vim ring-daemon.spec +``` +```bash +%define version 2.2.0 +%define release 1 +... +... +... +%changelog +* Tue Apr 14 2015 Thibault Cohen <thibault.cohen@savoirfairelinux.com> - 2.2.0-1 +- New upstream version +``` + +### DEB + +```bash +vim debian/changelog +``` +```bash +ring-daemon (2.2.0-1) unstable; urgency=medium + + [ Thibault Cohen] + * New upstream version + + -- Thibault Cohen <thibault.cohen@savoirfairelinux.com> Tue, 14 Apr 2015 12:40:24 -0400 +``` + +### Release + +You just have to launch release script. This script launch build, +download and update files and repositories... + +``` +sflvault connect 525 +... +... +cd /root/repos/ring/ +./ring-release-daemon.sh +``` + +## Gnome + +```bash +git clone ssh://tcohen@gerrit-sflphone.savoirfairelinux.com:29420/ring-client-gnome +cd ring-client-gnome +git checkout packaging-releases +``` + +### RPM + +vim ring-daemon.spec + +```bash +%define version 0.2.1 +%define release 1 +%define daemon_tag 2.1.0 +%define lrc_tag 0.2.1 +%define gnome_tag %{version} +... +... +... +%changelog +* Tue Apr 14 2015 Thibault Cohen <thibault.cohen@savoirfairelinux.com> - 0.2.1-1 +- New upstream version +``` + +### DEB + +```bash +debian/changelog +``` + +```bash +ring-gnome (0.2.1-1) unstable; urgency=medium + + [ Thibault Cohen] + * New Upstream version + + -- Thibault Cohen <thibault.cohen@savoirfairelinux.com> Tue, 14 Apr 2015 13:16:38 -0400 +``` + +```bash +debian/rules +``` + +```bash +DAEMON_TAG = 2.1.0 +LRC_TAG = 0.2.1 +GNOME_TAG = $(VER) +``` + +### Release + + +You just have to launch release script. This script launch build, download and update files and repositories... + +```bash +sflvault connect 525 +... +... +cd /root/repos/ring/ +./ring-release-gnome.sh +``` + + +------------------------------------------------------------------------ + +Advertisement +------------- + +When the packaging is finished, test that they are installable. Then +announce the release + +* on the official website <https://ring.cx> +* on Twitter <https://twitter.com/JoinTheRing> +* by email to ring@lists.savoirfairelinux.net with the subject line: "Ring +Major.Minor.Patch released"+ \ No newline at end of file diff --git a/technical/sync-protocol.md b/technical/sync-protocol.md @@ -0,0 +1,93 @@ +The swarm chat provides new possibilities for every device. Now, it's possible to sync history between devices by sharing the related repository. Devices sync needs to be redefined to follow those changes. + +A lot of scenarios are defined in the [RFC for swarms](https://git.jami.net/savoirfairelinux/ring-project/wikis/Group-chat-feature-(design-draft)), however, this doesn't imply for syncing conversations between devices for the same user. Some new scenarios must be written. + +# Old method + +Device sync were done via the DHT. Because every value MUST NOT exceed 64k, conversations were not sent in device sync, nor member profiles, because it's too heavy. This is a problem and MUST be improved. + +In the old method, the daemon is listening on "inbox:DEVICE_ID" for DeviceSync values which contains the contact list to sync (cf `AccountManager::startSync()`); + +**NOTE:** The curretn **DeviceSync** value present on the **DHT** is deprecated with this draft. + +# New method + +Since Jami has the [ConnectionManager](https://git.jami.net/savoirfairelinux/ring-project/wikis/technical/5.2.%20The%20connection%20manager), using p2p socket is possible to perform sync quickly with big values (cause the socket is not limited in data) + +Now, this is the scenario used to sync: + +1. When the device (*A*) goes online, it announces its presence via a DeviceAnnouncement like the OldMethod +2. Other devices (*!A*) will detect that announce and will ask this device through the **ConnectionManager** to open a new channel named "sync://DEVICE_ID_A". (Note: A will get announcement from other devices, so it will asks for sync channels too). +3. As soon as this channel is opened, the device which is asking this channel will send a **DeviceSync** (cf next part) value containing its known conversations and contacts. +4. *A* will check the **DeviceSync** value and: + + Remove contacts if it detects removed contacts + + Add contacts if it detects added contacts + + Remove conversations if it detects removed conversations + + Add conversations if it detects added conversations + + Remove conversation's requests if request is accepted (now in conversations)/declined + + Add conversation's requests if detected + +Note: If *A* detects new conversations, it will asks the device which announced that conversation to clone the repository through a git channel (so like described in [Swarm chat design](https://git.jami.net/savoirfairelinux/ring-project/wikis/Group-chat-feature-(design-draft))) + +# Device Sync + +This value is a JSON containing: +```json +{ + "contacts": [/* Contacts (TODO) */], + "conversation": [ + { "id":"convID", "created":TIMESTAMP, "removed":OPTIONAL_TIMESTAMP }, + { "id":"convID2", "created":TIMESTAMP2, "removed":OPTIONAL_TIMESTAMP2 } /* ... */ + ], + "conversationsRequests": [ + { "id":"convID", "received":TIMESTAMP, "declined":OPTIONAL_TIMESTAMP, + "members":[], "metadatas:[] }, + { "id":"convID2", "received":TIMESTAMP2, "declined":OPTIONAL_TIMESTAMP2 + "members":[], "metadatas:[] } /* ... */ + ], +} +``` + +# User stories + +## Sync when adding device + ++ Alice creates a conversation ++ (Optional) Alice add some messages ++ Alice adds another device ++ The other device should receives and sync the conversation previously created + +## Sync when connect a device + ++ Alice creates a conversation ++ (Optional) Alice add some messages ++ Alice connects another device ++ The other device should receives and sync the conversation previously created + +## Sync between multiple devices + ++ Alice got 2 devices ++ Alice creates a conversation ++ The other device should receives and sync the conversation created on one of the devices + +## Sync for detecting new requests + ++ Alice receives a conversation's request ++ Alice add a new device ++ The other device should retrieve the requests from device A + +## Sync for accepted requests + ++ Alice has 2 devices ++ Alice accepts a conversation's request ++ The other device should detect the accepted request + +## Sync for decline requests + ++ Alice has 2 devices ++ Alice declines a conversation's request ++ The other device should detect the declined request + +# Current implementation + +https://review.jami.net/c/ring-daemon/+/15584 implements this page+ \ No newline at end of file diff --git a/technical/working-with-gerrit.md b/technical/working-with-gerrit.md @@ -0,0 +1,96 @@ +# Working With Gerrit + +## Account Setup + ++ Gerrit server: https://review.jami.net ++ User documentation: https://review.jami.net/Documentation/intro-user.html ++ Jami projects on Gerrit: https://review.jami.net/#/admin/projects/ + +1. Sign-in with your google or github account or git.jami.net account +2. You'll also need to [upload an SSH key](https://review.jami.net/settings/#SSHKeys) to be able to commit changes for review. +3. Don't forget to select a username. +4. Finally, the email address specified in your git config must match one the email address registered with your Gerrit account. + +*Note for Savoir-faire Linux employees: please continue to use your @savoirfairelinux.com email address.* + +## To Know your Git Config + +`git config --list` + +## To Test your SSH Access + +To check that your SSH access is properly setup, run the following command: + +`ssh -p 29420 <username>@review.jami.net` + +<username> is your Gerrit username, that you should have set during the account creation. If not, you can do that here. + +``` +If your access is granted, you should see a message like: + +**** Welcome to Gerrit Code Review **** + +Hi, you have successfully connected over SSH. + + Unfortunately, interactive shells are disabled. + To clone a hosted Git repository, use: + + git clone ssh://<username>@review.jami.net:29420/REPOSITORY_NAME.git + +Connection to review.jami.net closed. +``` + +## Git Configuration + +Gerrit is the official git repository. + +## To update the configuration + +You must update your remote information to use now the Gerrit repository. To do that, update your origin url: + +`git remote set-url origin ssh://<username>@review.jami.net:29420/<project_name>` + +Replace `<project_name>` by the correct project (example: ring-daemon) + +Or clone the existing repository if you want to start fresh. + +## To Push by Default in refs/for/master + +You can configure git to automatically create a review when a change is pushed. + +`git config remote.origin.push HEAD:refs/for/master` + +## To Create the Review + +When pushing to this magic branch, a review will automatically be created on Gerrit. + +`git push origin HEAD:refs/for/master` + +If you configured the default to refs/for/master as described above, simply + +`git push` + +If HEAD currently points to the branch with the commits you'd like to push. Ideally, you should work in a feature/bug branch for the issue at hand. Then you can do: + +`git push origin <bugfix_branchname>:refs/for/master` + +If this is the first time you've pushed, you will be prompted to install a post-commit Hook to insert a Change-ID in your commit message. Gerrit needs this to track patchsets and will reject pushes until you install it. Simply copy paste the command to install the hook as instructed by Gerrit, and amend your commits. + +## To Push a Private patch + +You can push a work in progress (a.k.a draft) by pushing to refs/for/master%private + +For instance, you may want a "private" remote to push to; open <project_dir>/.git/config and add: + +``` +[remote "private"] + + url = ssh://<username>@review.jami.net:29420/ring-daemon + push = HEAD:refs/for/master%private +``` + +Then: + +`git push private` + +Private work the same way as patchsets, except they are not visible to others by default and don't trigger any Jenkins builds. A draft can then be shared or published.
