From 02826e2769b30720bc542eead293c0414c602763 Mon Sep 17 00:00:00 2001 From: Skylar Sadlier Date: Mon, 16 Sep 2024 23:27:29 -0600 Subject: [PATCH] - Fix Room Pagination node event data being unaccessible #28 - Add new node `Get Event` that will give you room data for a corresponding roomId and eventId #117 - Add new node `Fetch Event Relations` that can paginate through events related to another event #119 - README updates --- README.md | 83 ++++++----- examples/README.md | 5 +- package.json | 4 +- src/matrix-event-relations.html | 230 ++++++++++++++++++++++++++++++ src/matrix-event-relations.js | 108 +++++++++++++++ src/matrix-get-event.html | 103 ++++++++++++++ src/matrix-get-event.js | 87 ++++++++++++ src/matrix-paginate-room.html | 239 ++++++-------------------------- src/matrix-paginate-room.js | 5 +- 9 files changed, 622 insertions(+), 242 deletions(-) create mode 100644 src/matrix-event-relations.html create mode 100644 src/matrix-event-relations.js create mode 100644 src/matrix-get-event.html create mode 100644 src/matrix-get-event.js diff --git a/README.md b/README.md index 9c3e9d6..2099c5e 100644 --- a/README.md +++ b/README.md @@ -1,75 +1,72 @@ # node-red-contrib-matrix-chat [Matrix](https://matrix.org/) chat server client for [Node-RED](https://nodered.org/) -***Currently we are in beta. We ask that you open any issues you have on our repository to help us reach a stable well tested version. Things may change & break before our first release so check changelog before updating.*** +***Currently in beta. Please report any issues in our repository to help us reach a stable, well-tested release. Breaking changes may occur before our first stable release, so be sure to check the changelog before updating.*** -If you need help with this feel free to join our public matrix room at [#node-red-contrib-matrix-chat:skylar.tech](https://app.element.io/#/room/#node-red-contrib-matrix-chat:skylar.tech) +Join our public Matrix room for help: [#node-red-contrib-matrix-chat:skylar.tech](https://app.element.io/#/room/#node-red-contrib-matrix-chat:skylar.tech) ### Features -The following is supported from this package: +Supported functionality in this package includes: -- End-to-end encryption - - [Currently a WIP](#end-to-end-encryption-notes) - - Can also use [pantalaimon](https://github.com/matrix-org/pantalaimon) as an alternative solution to E2EE (if you need multiple sessions synced up with keys) -- Receive events from a room (messages, reactions, images, audio, locations, and files) whether encrypted or not -- Fetch and modify room state events (for changing room settings) -- Paginate room history -- Send Images/Files (sending files to e2ee room doesn't currently encrypt them yet) -- Edit messages -- Send typing events (Bot is typing ...) -- Delete events (messages, reactions, etc) -- Decrypt files in e2ee rooms -- Send HTML/Plain Text Message/Notice -- React to messages -- Register user's on closed registration Synapse servers using `registration_shared_secret` (Admin Only) -- List out users on a Synapse server (Admin Only) -- Get WhoIs info for a Synapse user (Admin Only) -- Add/Edit Synapse users using the v2 API (requires a pre-existing admin account) -- Deactivate users on Synapse servers (Admin Only) -- Get a user list from a room -- Kick user from room -- Ban user from room -- Join, Create, Invite, and Leave rooms -- Synapse admin API to force add user to room (requires bot to be in same room already) +- **End-to-end encryption (E2EE)** + - [Work in progress](#end-to-end-encryption-notes) + - Alternative: Use [Pantalaimon](https://github.com/matrix-org/pantalaimon) for E2EE key synchronization across sessions +- **Receive events** from rooms: Messages, reactions, images, audio, locations, files, encrypted or unencrypted +- **Fetch/modify room state**: Update room settings +- **Paginate room history** +- **Send files** (encryption support for files coming soon) +- **Send/edit messages** (supports plain text and HTML formats) +- **Send typing notifications** +- **Delete events** (messages, reactions, etc.) +- **Decrypt files** in E2EE rooms +- **React to messages** +- **Admin tools**: + - Register users on closed Synapse servers (`registration_shared_secret`) + - Manage users, including listing, adding, editing, deactivating (Synapse API) + - Force-add users to rooms +- **Room management**: Invite, kick, ban, join, create, and leave rooms - -Therefore, you can easily build a bot, chat relay, or administrate your Matrix server from within [Node-RED](https://nodered.org/). +These features allow you to easily build bots, set up chat relays, or even administrate your Matrix server directly from [Node-RED](https://nodered.org/). ### Installing -You can either install from within Node-RED by searching for `node-red-contrib-matrix-chat` or run this from within your Node-RED directory: +Install through Node-RED's UI by searching for `node-red-contrib-matrix-chat`, or use the following command inside your Node-RED directory: + ```bash npm install node-red-contrib-matrix-chat ``` ### Usage -We have examples! [Check them out](https://github.com/Skylar-Tech/node-red-contrib-matrix-chat/tree/master/examples#readme) -#### Extra functionality -You are not limited by just the nodes we have created. If you turn on global access when setting up your Matrix Client you can access the client directly from any function node to write your own logic. +Explore our [examples](https://github.com/Skylar-Tech/node-red-contrib-matrix-chat/tree/master/examples#readme) to see the module in action. -View an example [here](https://github.com/Skylar-Tech/node-red-contrib-matrix-chat/tree/master/examples#use-function-node-to-run-any-command) +#### Extending functionality + +You're not limited to just the nodes we've created. Enable global access in your Matrix Client to directly interact with the client from function nodes and create custom logic. + +[View an example here](https://github.com/Skylar-Tech/node-red-contrib-matrix-chat/tree/master/examples#use-function-node-to-run-any-command). ### End-to-End Encryption Notes -Currently, this module has no way of getting encryption keys from other devices on the same account. Therefore it is recommended you use the bot exclusively with Node-RED after it's creation. Failure to do so will lead to your bot being unable to receive messages from e2ee rooms it joined from another client. Shared secret registration makes this super easy since it returns a token and device ID. -This module stores a folder in your Node-RED directory called `matrix-client-storage` and is it vital that you periodically back this up if you are using e2ee. This is where the client stores all the keys necessary to decrypt messages and if lost you will lose access to e2e rooms. If you move your client to another NR install make sure to migrate this folder as well (and do not let both the old and new client run at same time). +- This module doesn't handle encryption key synchronization between devices. It’s recommended to use the bot exclusively in Node-RED to prevent issues with E2EE messages. +- **Storage:** Keys for E2EE are saved in a folder called `matrix-client-storage` within your Node-RED directory. Back up this folder regularly! If lost, you won’t be able to decrypt messages from E2EE rooms. +- To move your bot to a different installation, migrate this folder and ensure the old and new clients don't run simultaneously. -Want to contribute? Any help on getting the last pieces of e2ee figured out would be greatly appreciated :) +Interested in helping? Contributions to finalize E2EE support are welcome! -### Generate user -You will need a user to use this module. Luckily this module comes with a node that allows you to register users to a homeserver using the secret registration endpoint. This is perfect because it returns an `access_token` as well as a `device_id` which is exactly what we need. - -[Click here](https://github.com/Skylar-Tech/node-red-contrib-matrix-chat/tree/master/examples#readme) to see how to generate a user using secret registration +### Registering a User +This module includes a node to register users using the Synapse secret registration endpoint. It returns both an `access_token` and a `device_id`, perfect for setting up the bot. +[See how to register a user here](https://github.com/Skylar-Tech/node-red-contrib-matrix-chat/tree/master/examples#readme). ### Other Packages -- [node-red-contrib-gamedig](https://www.npmjs.com/package/node-red-contrib-gamedig) - Query game servers from Node-RED! +- [node-red-contrib-gamedig](https://www.npmjs.com/package/node-red-contrib-gamedig) - Query game servers from Node-RED. ### Contributing -All contributions are welcome! If you do add a feature please do a pull request so that everyone benefits :) -Sharing is caring! +We welcome all contributions! Please submit a pull request if you add a feature so the whole community can benefit. + +**Sharing is caring!** \ No newline at end of file diff --git a/examples/README.md b/examples/README.md index 9385a2e..616d958 100644 --- a/examples/README.md +++ b/examples/README.md @@ -264,4 +264,7 @@ If you say "force_join @test:example.com !320j90mf0394f:example.com" the bot wil Note: This requires the bot to be a server admin. This also only works for rooms on the same server. -![room-kick-ban.png](force-join-room.png) \ No newline at end of file +![room-kick-ban.png](force-join-room.png) + +### Paginate room history + diff --git a/package.json b/package.json index 54afa7c..cf1ba85 100644 --- a/package.json +++ b/package.json @@ -50,7 +50,9 @@ "matrix-synapse-deactivate-user": "src/matrix-synapse-deactivate-user.js", "matrix-synapse-join-room": "src/matrix-synapse-join-room.js", "matrix-whois-user": "src/matrix-whois-user.js", - "matrix-paginate-room": "src/matrix-paginate-room.js" + "matrix-paginate-room": "src/matrix-paginate-room.js", + "matrix-get-event": "src/matrix-get-event.js", + "matrix-event-relations": "src/matrix-event-relations.js" } }, "engines": { diff --git a/src/matrix-event-relations.html b/src/matrix-event-relations.html new file mode 100644 index 0000000..d016596 --- /dev/null +++ b/src/matrix-event-relations.html @@ -0,0 +1,230 @@ + + + + + diff --git a/src/matrix-event-relations.js b/src/matrix-event-relations.js new file mode 100644 index 0000000..9e37030 --- /dev/null +++ b/src/matrix-event-relations.js @@ -0,0 +1,108 @@ +const {RelationType, EventType, Direction} = require("matrix-js-sdk"); + +module.exports = function(RED) { + function MatrixFetchRelations(n) { + RED.nodes.createNode(this, n); + + let node = this; + this.name = n.name; + this.server = RED.nodes.getNode(n.server); + this.roomType = n.roomType; + this.roomValue = n.roomValue; + this.eventIdType = n.eventIdType; + this.eventIdValue = n.eventIdValue; + this.relationTypeType = n.relationTypeType; + this.relationTypeValue = n.relationTypeValue; + this.eventTypeType = n.eventTypeType; + this.eventTypeValue = n.eventTypeValue; + this.directionType = n.directionType; + this.directionValue = n.directionValue; + this.limitType = n.limitType; + this.limitValue = n.limitValue; + this.recurseType = n.recurseType; + this.recurseValue = n.recurseValue; + this.fromType = n.fromType; + this.fromValue = n.fromValue; + this.toType = n.toType; + this.toValue = n.toValue; + + node.status({ fill: "red", shape: "ring", text: "disconnected" }); + + if (!node.server) { + node.error("No configuration node", {}); + return; + } + node.server.register(node); + + node.server.on("disconnected", function() { + node.status({ fill: "red", shape: "ring", text: "disconnected" }); + }); + + node.server.on("connected", function() { + node.status({ fill: "green", shape: "ring", text: "connected" }); + }); + + node.on("input", async function(msg) { + if (!node.server || !node.server.matrixClient) { + node.error("No matrix server selected", msg); + return; + } + + try { + function evaluateNodePropertySafe(value, type, node, msg) { + try { + return RED.util.evaluateNodeProperty(value, type, node, msg); + } catch (e) { + if (e instanceof TypeError) { + return undefined; // Handle TypeError and return undefined + } + throw e; // Re-throw other errors to prevent masking issues + } + } + + let roomId = RED.util.evaluateNodeProperty(node.roomValue, node.roomType, node, msg), + eventId = RED.util.evaluateNodeProperty(node.eventIdValue, node.eventIdType, node, msg), + relationType = RED.util.evaluateNodeProperty(node.relationTypeValue, node.relationTypeType, node, msg), + eventType = RED.util.evaluateNodeProperty(node.eventTypeValue, node.eventTypeType, node, msg), + direction = RED.util.evaluateNodeProperty(node.directionValue, node.directionType, node, msg) || Direction.Backward, + limit = RED.util.evaluateNodeProperty(node.limitValue, node.limitType, node, msg), + recurse = RED.util.evaluateNodeProperty(node.recurseValue, node.recurseType, node, msg), + from = evaluateNodePropertySafe(node.fromValue, node.fromType, node, msg), + to = evaluateNodePropertySafe(node.toValue, node.toType, node, msg); + + let opts = { dir: direction }; + if(limit) { + opts.limit = limit; + } + if(recurse === true || recurse === false) { + opts.recurse = recurse; + } + if(from) { + opts.from = from; + } + if(to) { + opts.to = to; + } + + msg.payload = await node.server.matrixClient.fetchRelations( + roomId, + eventId, + relationType || null, + eventType || null, + opts + ); + node.send([msg, null]); + } catch (e) { + msg.error = `Event relations pagination error: ${e.stack}`; + node.error(msg.error, msg); + node.send([null, msg]); + } + }); + + node.on("close", function() { + node.server.deregister(node); + }); + } + + RED.nodes.registerType("matrix-fetch-relations", MatrixFetchRelations); +}; diff --git a/src/matrix-get-event.html b/src/matrix-get-event.html new file mode 100644 index 0000000..e958e14 --- /dev/null +++ b/src/matrix-get-event.html @@ -0,0 +1,103 @@ + + + + + + diff --git a/src/matrix-get-event.js b/src/matrix-get-event.js new file mode 100644 index 0000000..4220abb --- /dev/null +++ b/src/matrix-get-event.js @@ -0,0 +1,87 @@ +module.exports = function(RED) { + function MatrixGetEvent(n) { + RED.nodes.createNode(this, n); + + let node = this; + + this.name = n.name; + this.server = RED.nodes.getNode(n.server); + this.roomIdType = n.roomIdType; + this.roomIdValue = n.roomIdValue; + this.eventIdType = n.eventIdType; + this.eventIdValue = n.eventIdValue; + + node.status({ fill: "red", shape: "ring", text: "disconnected" }); + + if (!node.server) { + node.error("No configuration node", {}); + return; + } + node.server.register(node); + + node.server.on("disconnected", function(){ + node.status({ fill: "red", shape: "ring", text: "disconnected" }); + }); + + node.server.on("connected", function() { + node.status({ fill: "green", shape: "ring", text: "connected" }); + }); + + node.on('input', async function(msg) { + if (! node.server || ! node.server.matrixClient) { + node.error("No matrix server selected", msg); + node.send([null, msg]); + return; + } + + if (!node.server.isConnected()) { + node.error("Matrix server connection is currently closed", msg); + node.send([null, msg]); + return; + } + + function getToValue(msg, type, property) { + let value = property; + if (type === "msg") { + value = RED.util.getMessageProperty(msg, property); + } else if ((type === 'flow') || (type === 'global')) { + try { + value = RED.util.evaluateNodeProperty(property, type, node, msg); + } catch(e2) { + throw new Error("Invalid value evaluation"); + } + } else if(type === "bool") { + value = (property === 'true'); + } else if(type === "num") { + value = Number(property); + } + return value; + } + + try { + let roomId = getToValue(msg, node.roomIdType, node.roomIdValue), + eventId = getToValue(msg, node.eventIdType, node.eventIdValue); + + if(!roomId) { + node.error('Missing roomId', msg); + return; + } else if(!eventId) { + node.error('Missing eventId', msg); + return; + } + + msg.payload = await node.server.matrixClient.fetchRoomEvent(roomId, eventId); + node.send([msg, null]); + } catch(e) { + node.error("Failed to get event " + msg.topic + ": " + e, msg); + msg.payload = e; + node.send([null, msg]); + } + }); + + node.on("close", function() { + node.server.deregister(node); + }); + } + RED.nodes.registerType("matrix-get-event", MatrixGetEvent); +} diff --git a/src/matrix-paginate-room.html b/src/matrix-paginate-room.html index f5fdbe5..a10b3ab 100644 --- a/src/matrix-paginate-room.html +++ b/src/matrix-paginate-room.html @@ -1,9 +1,9 @@ \ No newline at end of file + +

Dynamic Properties

+

Some inputs like Room, Pagination Key, and Page Size can be dynamically set using message, flow, or global context variables.

+ +

Usage

+

To paginate through a room's timeline, trigger this node with a msg input. The first run will start the timeline, and a unique pagination key will be generated. Future calls can use this key to continue from where you left off. Use the "Paginate Backwards" option to move through older events or set it to false to move forwards through newer events.

+ diff --git a/src/matrix-paginate-room.js b/src/matrix-paginate-room.js index 8798e6c..7fbab8d 100644 --- a/src/matrix-paginate-room.js +++ b/src/matrix-paginate-room.js @@ -101,7 +101,8 @@ module.exports = function(RED) { moreMessages = true; if(!timelineWindow) { let timelineSet = room.getUnfilteredTimelineSet(); - node.debug(JSON.stringify(timelineSet.getFilter())); + // node.debug(JSON.stringify(timelineSet.getFilter())); + // MatrixClient's option initialSyncLimit gets set to the filter we are using // so override that value with our pageSize timelineWindow = new TimelineWindow(node.server.matrixClient, timelineSet); @@ -135,7 +136,7 @@ module.exports = function(RED) { // user : node.matrixClient.getUser(event.getSender()), topic : event.getRoomId(), eventId : event.getId(), - event : event, + event : event.getEffectiveEvent(), }; }); }