Merge branch 'master' of github.com:/krombel/matrix-register-bot

- add note to require php to be able to send mails
- explain operationMode

README.md

@@ -1,28 +1,34 @@
 # matrix-register-bot
 
-This bot provides a two-step-registration for matrix.
+This bot provides a two-step-registration for matrix ([synapse](https://github.com/matrix-org/synapse)).
 
 This is done in several steps:
-- potential new user registers on a bot-provided side
+- potential new user registers on a bot-provided site
+- user has to verify its mail address
 - bot sends a message to predefined room with a registration notification.
 - users in that room now can approve or decline the registration.
 - When approved
-  - the bot creates credentials
+  - the bot creates short time credentials
   - sends them to the user
-  - stores them encrypted in own database
+  - stores them encrypted in own databas or uses that as initial password for registration
-  - provides that credentials to [matrix-synapse-rest-auth](https://github.com/kamax-io/matrix-synapse-rest-auth#integrate) which has to be configured to query login.php
 
-2nd step: Implement the other apis to integrade [mxisd](https://github.com/kamax-io/mxisd/blob/master/docs/backends/rest.md)
+To configure synapse so that the users can login that were created via this bot you can either 
+- set `operationMode=synapse` so the bot uses the register api to push the new users to synapse or 
+- integrate it via [matrix-synapse-rest-auth](https://github.com/kamax-io/matrix-synapse-rest-auth#integrate) by configuring your system to point at `internal/login.php`.
+
+When using `operationMode=local` you can have the following benefits (some require [mxisd](https://github.com/kamax-io/mxisd/blob/master/docs/backends/rest.md))
+- Automatically set the display name based on first and last name on first login
+- Use the 3PID lookup for other users (only email)
+- Search for users that you have not seen yet
 
-This bot takes care for user accounts. So it stores the credentials itself and provides ways to access them via matrix-synapse-rest-auth or mxisd.
 ## How to install
 
 - Copy `config.sample.php` to `config.php` and configure the bot as you can find there
-- Configure your webserver to publish the folder `public`.
+- Configure your webserver to have the folder `public` accessible via web.
-  The folder `internal` contains files that can be accessed by mxisd or matrix-synapse-rest-auth or else via a reverse proxy
+  The folder `internal` contains files that only provide API access. They can be accessed by mxisd or matrix-synapse-rest-auth 
 - To integrate with [matrix-synapse-rest-auth](https://github.com/kamax-io/matrix-synapse-rest-auth):
   - `/_matrix-internal/identity/v1/check_credentials` should map to `internal/login.php`
-- To integrate with [mxisd](https://github.com/kamax-io/mxisd): Have a look at [the docs](https://github.com/kamax-io/mxisd/blob/master/docs/backends/rest.md) and apply as follows:
+- To integrate with [mxisd](https://github.com/kamax-io/mxisd): Have a look at [the docs of mxisd](https://github.com/kamax-io/mxisd/blob/master/docs/backends/rest.md) and apply as follows:
 
 
 | Key                            | file which handles that       | Description                                          |
@@ -33,15 +39,25 @@ This bot takes care for user accounts. So it stores the credentials itself and p
 | rest.endpoints.identity.bulk   | internal/identity_bulk.php    | Endpoint to query a list of 3PID                     |
 
 
-## Implement usage of additional features:
+## Further notes:
-### Use the ChangePasswortInterceptor:
 
-You need a reverse proxy which maps `/_matrix/client/r0/account/password` to `internal/intercept_change_password.php`.
+### This bot sends mails
+To allow the bot to verify the email address of the user and to interact with them e.g. in case of approval this bot needs a running mailserver configuration.
+This bot relies on php to be properly configured.
+
+### Use the ChangePasswortInterceptor (if `operationMode=local`)
+
+To allow users to change their pasword you need a reverse proxy which maps `/_matrix/client/r0/account/password` to `internal/intercept_change_password.php`.
 Here is an example for nginx:
-
 ```
         location /_matrix/client/r0/account/password {
                 proxy_pass http://localhost/mxbot/internal/intercept_change_password.php;
                 proxy_set_header X-Forwarded-For $remote_addr;
         }
 ```
+### The bot postpones some actions
+There is a cron.php which implements retries and database cleanups (e.g. to remove a username claim)
+For this run cron.php regularly with your system of choice.
+
+### Chat
+For further questions, comments, feedback and more come and talk in [#matrix-register-bot:msg-net.de](https://matrix.to/#/#matrix-register-bot:msg-net.de)

cron.php

@@ -14,17 +14,16 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-require_once("config.php");
+require_once(__DIR__ . "/config.php");
-require_once("mail_templates.php");
+require_once(__DIR__ . "/language.php");
-require_once("database.php");
+require_once(__DIR__ . "/mail_templates.php");
+require_once(__DIR__ . "/database.php");
 
 $sql = "SELECT id, first_name, last_name, username, email, state, note, verify_token, admin_token FROM registrations "
         . "WHERE state = " . RegisterState::PendingEmailSend
         . " OR state = " . RegisterState::PendingAdminSend
         . " OR state = " . RegisterState::PendingRegistration
-        . " OR state = " . RegisterState::PendingSendRegistrationMail
+        . " OR state = " . RegisterState::PendingSendRegistrationMail . ";";
-        . " OR state = " . RegisterState::RegistrationDeclined
-        . " OR state = " . RegisterState::AllDone . ";";
 foreach ($mx_db->query($sql) as $row) {
     $first_name = $row["first_name"];
     $last_name = $row["last_name"];
@@ -46,16 +45,22 @@ foreach ($mx_db->query($sql) as $row) {
                 }
                 break;
             case RegisterState::PendingAdminSend:
-                require_once("MatrixConnection.php");
+                require_once(__DIR__ . "/MatrixConnection.php");
                 $adminUrl = $config["webroot"] . "/verify_admin.php?t=" . $row["admin_token"];
                 $mxConn = new MatrixConnection($config["homeserver"], $config["access_token"]);
                 $mxMsg = new MatrixMessage();
-                $mxMsg->set_body($first_name . ' ' . $last_name . " möchte sich registrieren und hat folgende Notiz hinterlassen:\r\n"
+                $mxMsg->set_body(strtr($language["MSG_USER_WANTS_REGISTER"], [
-                        . $row["note"] . "\r\n"
+                    "@name" => (strlen($first_name . $last_name) > 0 ? $first_name . " " . $last_name : $username),
-                        . "Zum Bearbeiten hier klicken:\r\n" . $adminUrl);
+                    "@note" => $note,
-                $mxMsg->set_formatted_body($first_name . ' ' . $last_name . " möchte sich registrieren und hat folgende Notiz hinterlassen:<br />"
+                    "@adminUrl" => $adminUrl
-                        . $row["note"] . "<br />"
+                ]));
-                        . "Zum Bearbeiten <a href=\"" . $adminUrl . "\">hier</a> klicken");
+                if (isset($language["MSG_USER_WANTS_REGISTER_FORMATTED"])) {
+                    $mxMsg->set_formatted_body(strtr($language["MSG_USER_WANTS_REGISTER_FORMATTED"], [
+                        "@name" => (strlen($first_name . $last_name) > 0 ? $first_name . " " . $last_name : $username),
+                        "@note" => $note,
+                        "@adminUrl" => $adminUrl
+                    ]));
+                }
                 $mxMsg->set_type("m.text");
                 $response = $mxConn->send($config["register_room"], $mxMsg);
 
@@ -69,11 +74,29 @@ foreach ($mx_db->query($sql) as $row) {
                 break;
             case RegisterState::PendingRegistration:
                 // Registration got accepted but registration failed
-
+                switch ($config["operationMode"]) {
+                    case "synapse":
+                        // register with registration_shared_secret
+                        // generate a password with 10 characters
+                        $password = bin2hex(openssl_random_pseudo_bytes(5));
+                        $res = $mxConn->register($row["username"], $password, $config["registration_shared_secret"]);
+                        if (!$res) {
+                            // something went wrong while registering
+                            $password = NULL;
+                        }
+                        break;
+                    case "local":
+                        // register by adding a user to the local database
                         $password = $mx_db->addUser($row["first_name"], $row["last_name"], $row["username"], $row["email"]);
+                        break;
+                    default:
+                        throw new Exception("Unknown operationMode");
+                }
                 if ($password != NULL) {
                     // send registration_success
-                    $res = send_mail_registration_success($config["homeserver"], $first_name . " " . $last_name, $email, $username, $password, $config["howToURL"]);
+                    $res = send_mail_registration_success(
+                            $config["homeserver"], strlen($first_name . $last_name) > 0 ? $first_name . " " . $last_name : $username, $email, $username, $password, $config["howToURL"]
+                    );
                     if ($res) {
                         $mx_db->setRegistrationStateById(RegisterState::AllDone, $row["id"]);
                     } else {
@@ -83,7 +106,9 @@ foreach ($mx_db->query($sql) as $row) {
                     send_mail_registration_allowed_but_failed($config["homeserver"], $first_name . " " . $last_name, $email);
                     $mxMsg = new MatrixMessage();
                     $mxMsg->set_type("m.text");
-                    $mxMsg->set_body("Fehler beim Registrieren von " . $first_name . " " . $last_name . ".");
+                    $mxMsg->set_body(strtr($language["REGISTRATION_FAILED_FOR"], [
+                        "@name" => strlen($first_name . $last_name) > 0 ? $first_name . " " . $last_name : $username,
+                    ]));
                     $mxConn->send($config["register_room"], $mxMsg);
                     throw new Exception($language["REGISTRATION_FAILED"]);
                 }
@@ -91,14 +116,29 @@ foreach ($mx_db->query($sql) as $row) {
             case RegisterState::PendingSendRegistrationMail:
                 print ("Error: Unhandled state: PendingSendRegistrationMail for " . $first_name . " " . $last_name . " (" . $username . ")\n");
                 break;
-            case RegisterState::RegistrationDeclined:
-            case RegisterState::AllDone:
-                // do reqular cleanup
-                break;
         }
     } catch (Exception $e) {
         print("Error while handling cron for " . $first_name . " " . $last_name . " (" . $username . ")\n");
         print($e->getMessage());
     }
 }
+
+try {
+    //cleanup: all finished entries older than one month
+    $timestamp = date('Y-m-d H:m:s', strtotime("-1 month"));
+    $mx_db->query("DELETE FROM registrations "
+            . "WHERE request_date < '$timestamp'"
+            . " AND (state = " . RegisterState::RegistrationDeclined
+            . " OR state = " . RegisterState::AllDone . " );"
+    );
+    //cleanup: all entries which are pending email registration older than two days
+    $timestamp = date('Y-m-d H:m:s', strtotime("-2 days"));
+    $mx_db->query("DELETE FROM registrations "
+            . "WHERE request_date < '$timestamp'"
+            . " AND state = " . RegisterState::PendingEmailVerify . ";"
+    );
+} catch (Exception $e) {
+    print("Error while database cleanup\n");
+    print($e->getMessage());
+}
 ?>

database.php

@@ -14,7 +14,7 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-require_once("config.php");
+require_once(__DIR__ . "/config.php");
 if (!isset($config["databaseURI"])) {
     throw new Exception("malformed configuration: databaseURI not defined");
 }

internal/directory_search.php

@@ -14,7 +14,7 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-require_once("../database.php");
+require_once(__DIR__ . "/../database.php");
 $response = [
     "limited" => false,
     "result" => [],

internal/identity_bulk.php

@@ -14,7 +14,7 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-require_once("../database.php");
+require_once(__DIR__ . "/../database.php");
 $response = [
     "lookup" => []
 ];

internal/identity_single.php

@@ -14,7 +14,7 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-require_once("../database.php");
+require_once(__DIR__ . "/../database.php");
 $response = new stdClass;
 try {
     $inputJSON = file_get_contents('php://input');

internal/intercept_change_password.php

@@ -44,14 +44,14 @@ try {
         throw new Exception('"new_password" is not defined');
     }
 
-    require_once("../helpers.php");
+    require_once(__DIR__ . "/../helpers.php");
     $localpart = stripLocalpart($input["auth"]["user"]);
 
     if (empty($localpart)) {
         throw new Exception("localpart cannot be identified");
     }
 
-    require_once("../database.php");
+    require_once(__DIR__ . "/../database.php");
     if (!$mx_db->updatePassword(
                     $localpart, $input["auth"]["password"], $input["new_password"]
             )) {

internal/login.php

@@ -20,7 +20,7 @@ $response = [
     ]
 ];
 
-require_once("../database.php");
+require_once(__DIR__ . "/../database.php");
 
 abstract class LoginRequester {
 
@@ -56,7 +56,7 @@ try {
     // prefer the localpart attribute of mxisd. But in case of matrix-synapse-rest-auth
     // we have to parse it on our own
     if (empty($localpart)) {
-        require_once("../helpers.php");
+        require_once(__DIR__ . "/../helpers.php");
         $localpart = stripLocalpart($mxid);
     }
 

lang/lang.en-gb.php

@@ -1,4 +1,5 @@
 <?php
+
 /**
  * Copyright 2018 Matthias Kesler
  * Licensed under the Apache License, Version 2.0 (the "License");

lang/mail.de-de.php

@@ -1,4 +1,5 @@
 <?php
+
 /**
  * Copyright 2018 Matthias Kesler
  * Licensed under the Apache License, Version 2.0 (the "License");
@@ -14,7 +15,7 @@
  * limitations under the License.
  */
 function send_mail($receiver, $subject, $body) {
-	include("../config.php");
+    include(__DIR__ . "/../config.php");
     $headers = "From: " . $config["register_email"] . "\r\n"
             . "Content-Type: text/plain;charset=utf-8";
     return mail($receiver, $subject, $body, $headers);
@@ -68,7 +69,6 @@ Wir melden uns, wenn die Registrierung erfolgreich war.
 
 Das Administratoren-Team von " . $homeserver;
     return send_mail($receiver, $subject, $body);
-
 }
 
 function send_mail_registration_success($homeserver, $user, $receiver, $username, $password, $howToURL) {
@@ -101,8 +101,8 @@ Bei Fragen findest du nach der Anmeldung ein paar Räume in denen du sie stellen
 
 Das Administratoren-Team von " . $homeserver;
     return send_mail($receiver, $subject, $body);
-
 }
+
 function send_mail_registration_decline($homeserver, $user, $receiver, $reason) {
     $subject = "Registrierung auf $homeserver abgelehnt";
     $body = "Guten Tag " . $user . ",
@@ -121,4 +121,5 @@ Wir hoffen, dass du dies akzeptieren kannst.
 Das Administratoren-Team von " . $homeserver;
     return send_mail($receiver, $subject, $body);
 }
+
 ?>

lang/mail.en-gb.php

@@ -1,4 +1,5 @@
 <?php
+
 /**
  * Copyright 2018 Matthias Kesler
  * Licensed under the Apache License, Version 2.0 (the "License");
@@ -14,7 +15,7 @@
  * limitations under the License.
  */
 function send_mail($receiver, $subject, $body) {
-	include("../config.php");
+    include(__DIR__ . "/../config.php");
     $headers = "From: " . $config["register_email"] . "\r\n"
             . "Content-Type: text/plain;charset=utf-8";
     return mail($receiver, $subject, $body, $headers);
@@ -67,7 +68,6 @@ You will get another email with initial credentials once the registration got ha
 
 The admin team of " . $homeserver;
     return send_mail($receiver, $subject, $body);
-
 }
 
 function send_mail_registration_success($homeserver, $user, $receiver, $username, $password, $howToURL) {
@@ -95,8 +95,8 @@ You can ask further questions inside of the chat system.
 
 The admin team of " . $homeserver;
     return send_mail($receiver, $subject, $body);
-
 }
+
 function send_mail_registration_decline($homeserver, $user, $receiver, $reason) {
     $subject = "Registration on $homeserver declined.";
     $body = "Guten Tag " . $user . ",
@@ -115,4 +115,5 @@ We hope that you can understand this reason.
 The admin team of " . $homeserver;
     return send_mail($receiver, $subject, $body);
 }
+
 ?>

language.php

@@ -14,7 +14,7 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-require_once("config.php");
+require_once(__DIR__ . "/config.php");
 $lang = $config["defaultLanguage"];
 
 if (isset($_GET['lang'])) {
@@ -25,7 +25,7 @@ if (!file_exists($lang_file)) {
     error_log("Translation for " . $lang . " not found. Fallback to 'de-de'");
     $lang = "de-de";
 }
-$lang_file = dirname(__FILE__) . "/lang/lang." . $lang . ".php";
+$lang_file = __DIR__ . "/lang/lang." . $lang . ".php";
 require_once($lang_file);
 unset($lang_file);
 ?>

mail_templates.php

@@ -14,17 +14,17 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-require_once("config.php");
+require_once(__DIR__ . "/config.php");
 $lang = $config["defaultLanguage"];
 if (isset($_GET['lang'])) {
     $lang = filter_var($_GET['lang'], FILTER_SANITIZE_STRING);
 }
-$lang_file = dirname(__FILE__) . "/lang/mail." . $lang . ".php";
+$lang_file = __DIR__ . "/lang/mail." . $lang . ".php";
 if (!file_exists($lang_file)) {
     error_log("Mail templates for '" . $lang . "' not found. Fallback to 'de-de'");
     $lang = "de-de";
 }
-$lang_file = dirname(__FILE__) . "/lang/mail." . $lang . ".php";
+$lang_file = __DIR__ . "/lang/mail." . $lang . ".php";
 require_once($lang_file);
 unset($lang_file);
 ?>

public/index.php

@@ -19,12 +19,12 @@ if (!isset($_SERVER['HTTPS'])) {
     exit();
 }
 
-require_once "../language.php";
+require_once(__DIR__ . "/../language.php");
 if (!file_exists("../config.php")) {
     print($language["NO_CONFIGURATION"]);
     exit();
 }
-require_once "../config.php";
+require_once(__DIR__ . "/../config.php");
 
 // this values will not be used when using the register operation type
 $storeFirstLastName = false;
@@ -86,7 +86,7 @@ if ($_SERVER["REQUEST_METHOD"] == "POST") {
         $note = filter_var($_POST["note"], FILTER_SANITIZE_STRING);
         $email = filter_var($_POST["email"], FILTER_VALIDATE_EMAIL);
 
-        require_once("../database.php");
+        require_once(__DIR__ . "/../database.php");
         $res = $mx_db->addRegistration($first_name, $last_name, $username, $note, $email);
 
         if (!isset($res["verify_token"])) {
@@ -96,7 +96,7 @@ if ($_SERVER["REQUEST_METHOD"] == "POST") {
         $verify_token = $res["verify_token"];
 
         $verify_url = $config["webroot"] . "/verify.php?t=" . $verify_token;
-        require_once "../mail_templates.php";
+        require_once(__DIR__ . "/../mail_templates.php");
         $success = send_mail_pending_verification(
                 $config["homeserver"], $storeFirstLastName ? $first_name . " " . $last_name : $username, $email, $verify_url);
 
@@ -146,8 +146,10 @@ if ($_SERVER["REQUEST_METHOD"] == "POST") {
             <div class="col-xs-12 col-sm-8 col-md-4 col-sm-offset-2 col-md-offset-4">
                 <div class="panel panel-default">
                     <div class="panel-heading">
-                        <h3 class="panel-title"><?php echo strtr($language["TOPIC_PLEASE_REGISTER"], [ "@homeserver" => $config["homeserver"] ])
+                        <h3 class="panel-title"><?php
-                                . "<small>" . $language["TOPIC_PLEASE_REGISTER_NOTE"] . "</small>"; ?></h3>
+                            echo strtr($language["TOPIC_PLEASE_REGISTER"], ["@homeserver" => $config["homeserver"]])
+                            . "<small>" . $language["TOPIC_PLEASE_REGISTER_NOTE"] . "</small>";
+                            ?></h3>
                     </div>
                     <div class="panel-body">
                         <form name="regForm" role="form" action="index.php" method="post">
@@ -198,11 +200,13 @@ if ($_SERVER["REQUEST_METHOD"] == "POST") {
                             <input type="submit" value="<?php echo $language["REGISTER"]; ?>" class="btn btn-info btn-block">
 
                         </form>
-                        <?php if (isset($language["NOTE_FOR_REGISTRATION"])) {
+                        <?php
+                        if (isset($language["NOTE_FOR_REGISTRATION"])) {
                             echo "<p>" . $language["NOTE"] . ": <br />";
                             echo strtr($language["NOTE_FOR_REGISTRATION"], ["@homeserver" => $config["homeserver"]]);
                             echo "</p>";
-                        } ?>
+                        }
+                        ?>
                     </div>
                 </div>
             </div>

public/verify.php

@@ -13,13 +13,13 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-require_once "../language.php";
+require_once(__DIR__ . "/../language.php");
 if (!file_exists("../config.php")) {
     print($language["NO_CONFIGURATION"]);
     exit();
 }
-require_once "../config.php";
+require_once(__DIR__ . "/../config.php");
-require_once "../mail_templates.php";
+require_once(__DIR__ . "/../mail_templates.php");
 
 // enforce admin via https
 if (!isset($_SERVER['HTTPS'])) {
@@ -38,7 +38,7 @@ try {
     }
     $token = filter_var($_GET["t"], FILTER_SANITIZE_STRING);
 
-    require_once("../database.php");
+    require_once(__DIR__ . "/../database.php");
 
     $user = $mx_db->getUserForVerify($token);
     if ($user == NULL) {
@@ -51,7 +51,7 @@ try {
     $email = $user["email"];
     $admin_token = $user["admin_token"];
 
-    require_once("../MatrixConnection.php");
+    require_once(__DIR__ . "/../MatrixConnection.php");
     $adminUrl = $config["webroot"] . "/verify_admin.php?t=" . $admin_token;
     $mxConn = new MatrixConnection($config["homeserver"], $config["access_token"]);
     $mxMsg = new MatrixMessage();

public/verify_admin.php

@@ -13,13 +13,13 @@
  * See the License for the specific language governing permissions and
  * limitations under the License.
  */
-require_once "../language.php";
+require_once(__DIR__ . "/../language.php");
 if (!file_exists("../config.php")) {
     print($language["NO_CONFIGURATION"]);
     exit();
 }
-require_once "../config.php";
+require_once(__DIR__ . "/../config.php");
-require_once "../mail_templates.php";
+require_once(__DIR__ . "/../mail_templates.php");
 
 // enforce admin via https
 if (!isset($_SERVER['HTTPS'])) {
@@ -38,7 +38,7 @@ try {
     }
     $token = filter_var($_GET["t"], FILTER_SANITIZE_STRING);
 
-    require_once("../database.php");
+    require_once(__DIR__ . "/../database.php");
 
     $action = NULL;
     if (isset($_GET["allow"])) {
@@ -67,7 +67,7 @@ try {
         $mx_db->setRegistrationStateAdmin(RegisterState::PendingRegistration, $token);
 
         // register user
-        require_once("../MatrixConnection.php");
+        require_once(__DIR__ . "/../MatrixConnection.php");
         $mxConn = new MatrixConnection($config["homeserver"], $config["access_token"]);
 
         $password = NULL;
@@ -154,7 +154,8 @@ try {
                         </div>
                         <div class="panel-body">
                             <form name="appForm" role="form" action="verify_admin.php" method="GET">
-<?php if (isset($config["operationMode"]) && $config["operationMode"] === "local") {
+                                <?php
+                                if (isset($config["operationMode"]) && $config["operationMode"] === "local") {
                                     // this values will not be used when using the register operation type
                                     ?>
                                     <div class="row">