LSPS5 implementation #3662

martinsaposnic · 2025-03-11T20:12:16Z

A complete implementation for LSPS5 (spec defined here lightning/blips#55)

Reviewing commit by commit is recommended (~40% of the added lines are tests)

Notes:

Persistence of the state will come on a future PR
Will rebase from lightning-liquidity: Introduce EventQueue notifier and wake BP for message processing #3509 once that PR is merged

ldk-reviews-bot · 2025-03-11T20:12:19Z

👋 Thanks for assigning @tnull as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

martinsaposnic · 2025-03-11T20:16:28Z

This is a huge PR, but it wasn’t obvious to me how to split it in a way that would still make sense (I did split it into small commits to make it easier to review.). I’m open to suggestions if you have ideas on how this could be structured differently.

codecov · 2025-03-11T20:21:35Z

Codecov Report

Attention: Patch coverage is 86.86660% with 254 lines in your changes missing coverage. Please review.

Project coverage is 89.11%. Comparing base (830ffa0) to head (cb9404e).
Report is 21 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning-liquidity/src/lsps5/client.rs	81.70%	80 Missing and 21 partials ⚠️
lightning-liquidity/src/lsps5/service.rs	86.62%	43 Missing and 20 partials ⚠️
lightning-liquidity/src/lsps5/msgs.rs	88.52%	35 Missing and 10 partials ⚠️
lightning-liquidity/src/lsps0/ser.rs	73.78%	9 Missing and 18 partials ⚠️
lightning-liquidity/src/manager.rs	87.05%	6 Missing and 5 partials ⚠️
lightning-liquidity/src/lsps5/url_utils.rs	98.14%	5 Missing and 1 partial ⚠️
lightning-liquidity/src/lsps0/msgs.rs	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3662      +/-   ##
==========================================
- Coverage   89.26%   89.11%   -0.16%     
==========================================
  Files         155      159       +4     
  Lines      119968   122611    +2643     
  Branches   119968   122611    +2643     
==========================================
+ Hits       107092   109263    +2171     
- Misses      10265    10621     +356     
- Partials     2611     2727     +116

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

tnull

Wow, thank you for looking into this! I did a first pass, and it looks pretty amazing already!

Before going too much into further details, here are a few general comments upfront:

I'm generally no fan of introducing additional dependencies here, an in particular not reqwest and tokio. I think following the pattern so far BroadcastNotifications could be a request that the user handles with any HTTP client they want and then could call back into LSPS5ServiceHandler. Alternatively, we could also use a trait similar to the current HTTPClient, but I don't think we want to keep the default implementation. Note that the blocking reqwest variant wraps a tokio runtime internally, and therefore should never (1, 2, ...) be used together. I guess technically we could consider a default async version of the trait that uses async reqwest, but I would prefer to simply have well-documented trait on our end that the user can implement however they choose to. Also note that stacking tokio runtimes is heavily discouraged in general, so assuming our users would themselves use a tokio runtime, we shouldn't wrap one in LSPS5ServiceHandler.
Note that lightning-liquidity is optionally no-std compliant, so please don't rely on std wherever possible, often it's just a matter of using core instead and importing the respective types from crate::prelude. If you really find yourselves needing to use std, make sure it's feature gated behind feature = "std" and we provide an alternative for users that don't support it.
Minor: Regarding formatting we're using tabs, not spaces. Feel free to run ./contrib/run-rustfmt.sh after each commit to run our formatting scripts.
This PR in its current scope is great, just want to note that eventually we need to add persistence for the state. As we haven't fully fleshed out the persistence strategy for lightning-liquidity in general yet, it's actually preferred to defer this to a follow-up, but just wanted to mention it. Also note that some changes to MessageQueue/EventQueue will happen in lightning-liquidity: Introduce EventQueue notifier and wake BP for message processing #3509, but will ping you to rebase once that has been merged. (just sidenotes here).

I hope these initial points make sense, let me know if you have any questions, or once you made corresponding changes and think this is ready for the next round of review!

ldk-reviews-bot · 2025-03-13T12:45:04Z

👋 The first review has been submitted!

Do you think this PR is ready for a second reviewer? If so, click here to assign a second reviewer.

tnull · 2025-03-13T12:48:30Z

This is a huge PR, but it wasn’t obvious to me how to split it in a way that would still make sense (I did split it into small commits to make it easier to review.). I’m open to suggestions if you have ideas on how this could be structured differently.

Thanks for asking! I'm totally fine to keep this (with its current scope) in a single PR, as long as we keep the commit history pretty clean to allow continuing review to happen commit-by-commit. To this end, please make sure add any fixup commits clearly marked (e.g. via a f or fixup prefix in the commit message header) directly under the commit they belong to, so they can cleanly be squashed into the respective feature commits in-between review rounds.

tnull · 2025-03-13T12:54:53Z

Btw, I'm not sure if you're familiar with the previous attempt of implementing LSPS5: #3499

Given this is a clean slate, not sure how much there is to learn, but still might be worth a look. Also not sure if @johncantrell97 would be interested in reviewing this PR, too, as he's familiar with the codebase and LSPS5.

martinsaposnic · 2025-03-14T17:11:07Z

@tnull, ready for the next review round!

Removed reqwest and tokio.
HttpClient is now a generic trait, allowing users to pass any implementation.
Removed std.
Small refactor on client and service to be able to delete some silly / unnecessary code.
Ran formatting on every commit.
All new changes are in commits prefixed with fixup:.

CI is failing because of the usage of the url crate (which I guess does not support rust 1.63?), which I need for validating the webhook URL. A new commit will come addressing that shortly

ldk-reviews-bot · 2025-03-17T00:00:52Z

🔔 1st Reminder

Hey @tnull @valentinewallace! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-03-17T00:00:54Z

🔔 1st Reminder

Hey @tnull @valentinewallace! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

martinsaposnic · 2025-03-24T20:30:30Z

Hey @tnull, thank you so much for reviewing and answering my questions!

I just pushed some updates, addressing all your comments, including:

All new crates added before are now deleted
Correct notification signature
URL validation is now handled by us (not an external crate)
Generic time trait for user-provided implementations (with a default std implementation)
Added tests with spec examples and test vectors (more tests are coming, so we are sure the PR is following the spec byte-for-byte)

This should be ready for the next round of review.

Thanks again!

tnull

Thanks for the changes so far, this is def. quickly moving in the right direction!

Added a bunch of more high-level comments, will see to get more into the details in the next round. Feel free to squash current fixups!

lightning-liquidity/src/lsps5/msgs.rs

lightning-liquidity/src/lsps5/url_utils.rs

lightning-liquidity/src/lsps5/msgs.rs

lightning-liquidity/src/lsps5/client.rs

lightning-liquidity/src/lsps5/mod.rs

martinsaposnic · 2025-03-27T21:46:08Z

Thanks for the changes so far! This is definitely moving in the right direction quickly.

I’ve added more high-level comments and will get into the details in the next round. Feel free to squash the fixups!

Thanks for the review @tnull!

I've addressed all new comments and pushed them as fixup: commits.

Key changes:

http_client has been removed. Instead, there's now a SendWebhookNotifications event that informs the user which URLs to call, leaving the actual POST requests to them.
No more panics or unsafe unwraps.
TimeProvider can now be injected via an alternative constructor.

CI is taking forever, so I may need to push a few fixes if anything breaks, but this should be ready for review again!

A couple of things I know will get comments:

I'm using fromRFC3339 and toRFC3339 from LSPSDateTime, but since timestamps are handled as Duration, I had to introduce a function to convert Duration into LSPSDateTime:

/// Returns the number of seconds since the Unix epoch.
pub fn new_from_duration(duration: Duration) -> Self {
    Self(DateTime::from_timestamp(duration.as_secs() as i64, duration.subsec_nanos()).unwrap())
}

It works, but it’s not the cleanest approach—open to suggestions.

The client receives app_name and webhook_url as plain strings, which are then wrapped in LSPS5AppName and LSPS5WebhookUrl via a new method that validates them.

When the event is triggered, the service picks it up and receives already-validated LSPS5AppName and LSPS5WebhookUrl, but it calls validate again to ensure nothing changed.

This pattern—new calls validate, and then validate runs again later—works, but feels a bit redundant. Curious to hear thoughts on whether this is a good approach.

Looking forward to your feedback!

tnull

Cool, thank you for the quick turnaround! I did a rather quick round on the fixups, they look good so far, feel free to squash them!

I think CI is currently failing due to some use of moved value: _time_provider`` errors. Will see to do a more thorough round on the entire PR once it passes again.

lightning-liquidity/Cargo.toml

lightning-liquidity/src/lsps0/ser.rs

tnull · 2025-03-28T13:44:13Z

lightning-liquidity/src/lsps0/ser.rs

+	}
+
+	/// Returns the number of seconds since the Unix epoch.
+	pub fn new_from_duration(duration: Duration) -> Self {


Can we make this an impl From<Duration> for LSPSDateTime?

Hmm, thinking about this again, I think this was a bad idea as From should be really only implemented if there is a clear 1:1 conversion between the types applicable that doesn't need additional context. Here, it however hides the fact that we implictly assume it's a Duration from the UNIX epoch.

I have to eat my words, please revert this and make it a new_from_duration_since_epoch constructor. Sorry, excuse the back-and-forth!

lightning-liquidity/src/lsps0/ser.rs

lightning-liquidity/src/lsps5/event.rs

- Add 'time' feature flag to allow disabling time-dependent functionality - Include 'time' in default features - Allow users to disable SystemTime::now without disabling all std features - Follows pattern established in other crates (e.g., lightning-transaction-sync) - Improves compatibility with WASM environments

- Implement From<Duration> for LSPSDateTime - Avoid doing ambiguous timestamp types - Add abs_diff function to use on client / service

Adds a new url_utils.rs module that provides: - A lightweight URL parser specialized for LSPS5 webhook validation - A URL trait and Url implementation focusing on scheme and host extraction - RFC-compliant scheme validation - Comprehensive test coverage for various URL scenarios This implementation allows validating webhook URLs without depending on the external url crate

- Define LSPS5Request and LSPS5Response enums for webhook registration, listing, and removal. - Implement WebhookNotification and associated helper constructors for different notification types. - Implement serialization/deserialization support with comprehensive tests. - Improve LSPS5 message types, validation, and testing - Replace generic String types with strongly-typed Lsps5AppName and Lsps5WebhookUrl with built-in length and format validation - Restructure imports to follow one-per-line convention - Add constants for notification method strings - Make WebhookNotificationMethod enum more consistent with LSPS5 prefix - Use explicit serde_json::json and serde_json::Value instead of imports - Improve code documentation with proper ticks and references - Add comprehensive test vectors from the BLIP-0055 specification

- Introduce LSPS5ServiceEvent for LSPS-side webhook events including registration, listing, removal, and notification. - Define LSPS5ClientEvent for handling webhook outcomes on the client (Lightning node) side. - Outline WebhookNotificationParams enum to support notification-specific parameters. - Improve LSPS5 event documentation and field naming - Rename client/lsp fields to counterparty_node_id for consistent terminology - Replace generic String types with more specific Lsps5AppName and Lsps5WebhookUrl - Add comprehensive documentation for all events and fields - Include format specifications (UTF-8, ISO8601) and size constraints - Add request_id field to all relevant events for consistent request tracking - Provide detailed descriptions of error codes and their meanings - Use complete sentences in documentation comments

…kNotifications

Implements the LSPS5 webhook registration service that allows LSPs to notify clients of important events via webhooks. This service handles webhook registration, listing, removal, and notification delivery according to the LSPS5 specification. Some details: - A generic HttpClient trait is defined so users can provide their own HTTP implementation - A generic TimeProvider trait is defined with a DefaultTimeProvider that uses std functionality - Uses URL utils to validate webhook URLs according to LSPS5 requirements - Uses secure message signing logic from the lightning::util::message_signing module - Works with the events and messages defined in earlier commits - Tests will be provided in a future commit

Implements the client-side functionality for LSPS5 webhook registration, allowing Lightning clients to register, list, and remove webhooks with LSPs. This client handler processes responses and verifies webhook notification signatures. Key features: - Full client API for webhook registration operations - Per-peer state tracking for pending requests - Automatic request timeout and cleanup - Security validation for webhook URLs - Notification signature verification - Add store_signature and check_signature to prevent replay attacks - Some tests are provided but more will come in a future commit This implementation pairs with the service-side LSPS5 webhook handler to complete the webhook registration protocol according to the LSPS5 specification.

Fully integrates the LSPS5 webhook components into the lightning-liquidity framework, enabling usage through the LiquidityManager. It includes - Registering LSPS5 events in the event system - Adding LSPS5 module to the main library exports - Updating LSPS0 serialization to handle LSPS5 messages - Adding LSPS5 configuration options to client and service config structures - Implementing message handling for LSPS5 requests and responses - Adding accessor methods for LSPS5 client and service handlers With this change, LSPS5 webhook functionality can now be accessed through the standard LiquidityManager interface, following the same pattern as other LSPS protocols.

…orrectness of the signing logic

martinsaposnic · 2025-03-31T19:34:19Z

Hey @tnull, thanks for the review! Apologies for pinging you earlier when the CI was failing.

I've addressed your last comments in fixup: commits, and the CI is now passing.

ldk-reviews-bot · 2025-04-03T21:34:34Z

🔔 1st Reminder

Hey @tnull! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

tnull

Started with more detailed review, unfortunately still didn't make it through for now.

Some comments/questions, and this also needs a minor rebase by now. Feel free to squash fixups!

tnull · 2025-04-04T08:45:38Z

lightning-liquidity/src/lsps5/event.rs

@@ -71,9 +71,15 @@ pub enum LSPS5ServiceEvent {
 		request_id: LSPSRequestId,
 	},

-	/// A notification needs to be sent to a client's webhook


No, having a one-line description paragraph first is still a good idea, IMO.

tnull · 2025-04-04T08:45:51Z

lightning-liquidity/src/lsps0/ser.rs

+	}
+
+	/// Returns the number of seconds since the Unix epoch.
+	pub fn new_from_duration(duration: Duration) -> Self {


Hmm, thinking about this again, I think this was a bad idea as From should be really only implemented if there is a clear 1:1 conversion between the types applicable that doesn't need additional context. Here, it however hides the fact that we implictly assume it's a Duration from the UNIX epoch.

I have to eat my words, please revert this and make it a new_from_duration_since_epoch constructor. Sorry, excuse the back-and-forth!

tnull · 2025-04-04T08:49:41Z

lightning-liquidity/src/lsps0/ser.rs

Seems this fixup should be an actual prefactor commit as it's changing pre-existing code?

tnull · 2025-04-04T08:50:57Z

lightning-liquidity/src/lsps5/url_utils.rs

Now that we don't use the trait, the commit message needs an update.

tnull · 2025-04-04T08:54:32Z

lightning-liquidity/src/lsps5/url_utils.rs

+/// According to RFC 1738, a scheme must:
+/// 1. Start with a letter (a-z, A-Z)
+/// 2. Contain only letters, digits, plus (+), period (.), or hyphen (-)
+fn validate_scheme(scheme: &str) -> bool {


nit: Given this returns a bool naming this is_valid_scheme would be more idiomatic.

tnull · 2025-04-04T09:07:11Z

lightning-liquidity/src/lsps5/url_utils.rs

+
+		let mut final_host = clean_host;
+		if let Some(port_idx) = clean_host.rfind(':') {
+			final_host = &clean_host[0..port_idx];


Isn't the port part of the host though? If we strip it, the resulting Url wouldn't be usable if it doesn't use default ports, no?

tnull · 2025-04-04T09:14:49Z

lightning-liquidity/src/lsps5/msgs.rs

+
+use super::url_utils::LSPSUrl;
+
+/// Maximum allowed length for an app_name (in bytes)


nit: Tick app_name.

tnull · 2025-04-04T09:17:27Z

lightning-liquidity/src/lsps5/msgs.rs

+pub(crate) const LSPS5_LIST_WEBHOOKS_METHOD_NAME: &str = "lsps5.list_webhooks";
+pub(crate) const LSPS5_REMOVE_WEBHOOK_METHOD_NAME: &str = "lsps5.remove_webhook";
+
+pub(crate) const LSPS5_WEBHOOK_REGISTERED: &str = "lsps5.webhook_registered";


Maybe suffix these consts with _NOTIFICATION or similar?

tnull · 2025-04-04T09:27:01Z

lightning-liquidity/src/lsps5/msgs.rs

+
+impl WebhookNotificationMethod {
+	/// Extract parameters for JSON serialization
+	pub fn parameters_json_value(&self) -> serde_json::Value {


Hmm, kinda unfortunate we need this. Maybe we should make this private as it's specific to the way WebhookNotification's serialization works?

tnull · 2025-04-04T09:29:46Z

lightning-liquidity/src/lsps5/msgs.rs

+			LSPS5_EXPIRY_SOON => {
+				// Default timeout when deserializing without params
+				// The actual timeout will be set from the params field later
+				Ok(Self::LSPS5ExpirySoon { timeout: 0 })


Ugh, this is kind of hacky and seems it's inviting bugs down the line. Do we have an idea how to avoid it? Maybe it means we need to serialize/deserialize WebhookNotification in one go rather than also implementing Serialize/Deserialize for just WebhookNotificationMethod?

martinsaposnic mentioned this pull request Mar 11, 2025

Add bLIP 55: Webhook Registration (LSPS5) lightning/blips#55

Open

ldk-reviews-bot requested a review from arik-so March 11, 2025 20:22

tnull self-requested a review March 11, 2025 20:37

wpaulino removed the request for review from arik-so March 11, 2025 20:39

tnull reviewed Mar 13, 2025

View reviewed changes

martinsaposnic force-pushed the lsps5 branch from 3aca4fc to 4dbe74d Compare March 14, 2025 14:48

martinsaposnic marked this pull request as draft March 14, 2025 14:49

martinsaposnic force-pushed the lsps5 branch 2 times, most recently from 1d4b47c to edf5346 Compare March 14, 2025 16:31

martinsaposnic marked this pull request as ready for review March 14, 2025 16:31

ldk-reviews-bot requested a review from valentinewallace March 14, 2025 16:32

martinsaposnic requested a review from tnull March 14, 2025 17:12

martinsaposnic force-pushed the lsps5 branch 7 times, most recently from 9809682 to af5929e Compare March 16, 2025 14:24

tnull removed the request for review from valentinewallace March 17, 2025 08:33

martinsaposnic force-pushed the lsps5 branch from af5929e to e018300 Compare March 17, 2025 16:10

martinsaposnic force-pushed the lsps5 branch 3 times, most recently from 2c51275 to b0b7bad Compare March 24, 2025 20:06

martinsaposnic requested a review from tnull March 25, 2025 13:08

tnull reviewed Mar 25, 2025

View reviewed changes

martinsaposnic force-pushed the lsps5 branch 3 times, most recently from f057bda to 979abf1 Compare March 27, 2025 21:27

martinsaposnic requested a review from tnull March 27, 2025 21:46

martinsaposnic force-pushed the lsps5 branch from 979abf1 to b70e9fb Compare March 28, 2025 15:05

tnull reviewed Mar 28, 2025

View reviewed changes

martinsaposnic added 12 commits March 31, 2025 13:06

fixup: time is a feature, not a cfg flag, so delete it from check-cfg

6e43a5f

fixup: Improvements on LSPSDateTime

afe320b

- Implement From<Duration> for LSPSDateTime - Avoid doing ambiguous timestamp types - Add abs_diff function to use on client / service

fixup: Provide concrete guidance on event.rs for new event SendWebhoo…

62d336e

…kNotifications

Add LSPS5 module structure

9eb7681

Add tests for LSPS5 client and service. Also tests for checking the c…

cb9404e

…orrectness of the signing logic

martinsaposnic force-pushed the lsps5 branch from 9a250cd to cb9404e Compare March 31, 2025 16:19

martinsaposnic requested a review from tnull April 1, 2025 21:34

tnull reviewed Apr 4, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LSPS5 implementation #3662

LSPS5 implementation #3662

martinsaposnic commented Mar 11, 2025 •

edited

Loading

ldk-reviews-bot commented Mar 11, 2025 •

edited

Loading

martinsaposnic commented Mar 11, 2025 •

edited

Loading

codecov bot commented Mar 11, 2025 •

edited

Loading

tnull left a comment •

edited

Loading

ldk-reviews-bot commented Mar 13, 2025

tnull commented Mar 13, 2025 •

edited

Loading

tnull commented Mar 13, 2025

martinsaposnic commented Mar 14, 2025 •

edited

Loading

ldk-reviews-bot commented Mar 17, 2025

ldk-reviews-bot commented Mar 17, 2025

martinsaposnic commented Mar 24, 2025

tnull left a comment

martinsaposnic commented Mar 27, 2025

tnull left a comment

tnull Mar 28, 2025

tnull Apr 4, 2025

martinsaposnic commented Mar 31, 2025

ldk-reviews-bot commented Apr 3, 2025

tnull left a comment •

edited

Loading

tnull Apr 4, 2025

tnull Apr 4, 2025

tnull Apr 4, 2025

tnull Apr 4, 2025

tnull Apr 4, 2025

tnull Apr 4, 2025

tnull Apr 4, 2025

tnull Apr 4, 2025

tnull Apr 4, 2025

tnull Apr 4, 2025


		use super::url_utils::LSPSUrl;

		/// Maximum allowed length for an app_name (in bytes)

LSPS5 implementation #3662

Are you sure you want to change the base?

LSPS5 implementation #3662

Conversation

martinsaposnic commented Mar 11, 2025 • edited Loading

ldk-reviews-bot commented Mar 11, 2025 • edited Loading

martinsaposnic commented Mar 11, 2025 • edited Loading

codecov bot commented Mar 11, 2025 • edited Loading

Codecov Report

tnull left a comment • edited Loading

Choose a reason for hiding this comment

ldk-reviews-bot commented Mar 13, 2025

tnull commented Mar 13, 2025 • edited Loading

tnull commented Mar 13, 2025

martinsaposnic commented Mar 14, 2025 • edited Loading

ldk-reviews-bot commented Mar 17, 2025

ldk-reviews-bot commented Mar 17, 2025

martinsaposnic commented Mar 24, 2025

tnull left a comment

Choose a reason for hiding this comment

martinsaposnic commented Mar 27, 2025

tnull left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

martinsaposnic commented Mar 31, 2025

ldk-reviews-bot commented Apr 3, 2025

tnull left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

martinsaposnic commented Mar 11, 2025 •

edited

Loading

ldk-reviews-bot commented Mar 11, 2025 •

edited

Loading

martinsaposnic commented Mar 11, 2025 •

edited

Loading

codecov bot commented Mar 11, 2025 •

edited

Loading

tnull left a comment •

edited

Loading

tnull commented Mar 13, 2025 •

edited

Loading

martinsaposnic commented Mar 14, 2025 •

edited

Loading

tnull left a comment •

edited

Loading