mastodon/spec/models
ThibG ca56527140
Add follower synchronization mechanism (#14510)
* Add support for followers synchronization on the receiving end

Check the `collectionSynchronization` attribute on `Create` and `Announce`
activities and synchronize followers from provided collection if possible.

* Add tests for followers synchronization on the receiving end

* Add support for follower synchronization on the sender's end

* Add tests for the sending end

* Switch from AS attributes to HTTP header

Replace the custom `collectionSynchronization` ActivityStreams attribute by
an HTTP header (`X-AS-Collection-Synchronization`) with the same syntax as
the `Signature` header and the following fields:
- `collectionId` to specify which collection to synchronize
- `digest` for the SHA256 hex-digest of the list of followers known on the
   receiving instance (where “receiving instance” is determined by accounts
   sharing the same host name for their ActivityPub actor `id`)
- `url` of a collection that should be fetched by the instance actor

Internally, move away from the webfinger-based `domain` attribute and use
account `uri` prefix to group accounts.

* Add environment variable to disable followers synchronization

Since the whole mechanism relies on some new preconditions that, in some
extremely rare cases, might not be met, add an environment variable
(DISABLE_FOLLOWERS_SYNCHRONIZATION) to disable the mechanism altogether and
avoid followers being incorrectly removed.

The current conditions are:
1. all managed accounts' actor `id` and inbox URL have the same URI scheme and
   netloc.
2. all accounts whose actor `id` or inbox URL share the same URI scheme and
   netloc as a managed account must be managed by the same Mastodon instance
   as well.

As far as Mastodon is concerned, breaking those preconditions require extensive
configuration changes in the reverse proxy and might also cause other issues.

Therefore, this environment variable provides a way out for people with highly
unusual configurations, and can be safely ignored for the overwhelming majority
of Mastodon administrators.

* Only set follower synchronization header on non-public statuses

This is to avoid unnecessary computations and allow Follow-related
activities to be handled by the usual codepath instead of going through
the synchronization mechanism (otherwise, any Follow/Undo/Accept activity
would trigger the synchronization mechanism even if processing the activity
itself would be enough to re-introduce synchronization)

* Change how ActivityPub::SynchronizeFollowersService handles follow requests

If the remote lists a local follower which we only know has sent a follow
request, consider the follow request as accepted instead of sending an Undo.

* Integrate review feeback

- rename X-AS-Collection-Synchronization to Collection-Synchronization
- various minor refactoring and code style changes

* Only select required fields when computing followers_hash

* Use actor URI rather than webfinger domain in synchronization endpoint

* Change hash computation to be a XOR of individual hashes

Makes it much easier to be memory-efficient, and avoid sorting discrepancy issues.

* Marginally improve followers_hash computation speed

* Further improve hash computation performances by using pluck_each
2020-10-21 18:04:09 +02:00
..
admin Add option to include reported statuses in warning e-mail (#11639) 2019-08-23 22:37:23 +02:00
concerns Add follower synchronization mechanism (#14510) 2020-10-21 18:04:09 +02:00
form Change deletes to preserve soft-deleted statuses in unresolved reports (#11805) 2019-09-11 16:32:44 +02:00
web Lint pass (#8876) 2018-10-04 12:36:53 +02:00
account_alias_spec.rb Add account migration UI (#11846) 2019-09-19 20:58:19 +02:00
account_conversation_spec.rb Add conversations API (#8832) 2018-10-07 23:44:58 +02:00
account_deletion_request_spec.rb Change account suspensions to be reversible by default (#14726) 2020-09-15 14:37:58 +02:00
account_domain_block_spec.rb Spec AccountDomainBlock (#3816) 2017-06-18 02:59:49 +02:00
account_filter_spec.rb Redesign admin accounts index (#9340) 2018-11-26 15:53:27 +01:00
account_migration_spec.rb Add account migration UI (#11846) 2019-09-19 20:58:19 +02:00
account_moderation_note_spec.rb Lint pass (#8876) 2018-10-04 12:36:53 +02:00
account_spec.rb Search account domain in lowercase (#13016) 2020-02-01 15:42:24 +01:00
account_stat_spec.rb Fix account counters being overwritten by parallel writes (#12045) 2019-10-02 04:59:37 +02:00
account_tag_stat_spec.rb Add specs for AccountTagStat model (#9452) 2018-12-07 16:37:56 +01:00
announcement_mute_spec.rb Add announcements (#12662) 2020-01-23 22:00:13 +01:00
announcement_reaction_spec.rb Add announcements (#12662) 2020-01-23 22:00:13 +01:00
announcement_spec.rb Add announcements (#12662) 2020-01-23 22:00:13 +01:00
backup_spec.rb Lint pass (#8876) 2018-10-04 12:36:53 +02:00
block_spec.rb Cover Block more (#3837) 2017-06-19 11:31:37 +02:00
conversation_mute_spec.rb Lint pass (#8876) 2018-10-04 12:36:53 +02:00
conversation_spec.rb Revert "Remove conversation URI (#11423)" (#11424) 2019-07-28 17:47:37 +02:00
custom_emoji_category_spec.rb Add categories for custom emojis (#11196) 2019-06-28 15:54:10 +02:00
custom_emoji_filter_spec.rb Add specs for CustomEmojiFilter (#9599) 2018-12-21 18:52:57 +01:00
custom_emoji_spec.rb Make custom emoji domains case insensitive #9351 (#9474) 2018-12-11 05:30:57 +01:00
custom_filter_spec.rb Lint pass (#8876) 2018-10-04 12:36:53 +02:00
device_spec.rb Add E2EE API (#13820) 2020-06-02 19:24:53 +02:00
domain_allow_spec.rb Add whitelist mode (#11291) 2019-07-30 11:10:46 +02:00
domain_block_spec.rb Allow blocking TLDs, and fix TLD blocks not being editable (#12805) 2020-01-08 22:42:05 +01:00
email_domain_block_spec.rb Add uniqueness to block email domains (#5692) 2017-11-14 20:37:17 +01:00
encrypted_message_spec.rb Add E2EE API (#13820) 2020-06-02 19:24:53 +02:00
export_spec.rb Export and import show_reblogs together with following list (#10495) 2019-04-08 07:28:27 +02:00
favourite_spec.rb Cover Favourite more (#3841) 2017-06-19 01:38:50 +02:00
featured_tag_spec.rb Add featured hashtags to profiles (#9755) 2019-02-04 04:25:59 +01:00
follow_request_spec.rb Add option to be notified when a followed user posts (#13546) 2020-09-18 17:26:45 +02:00
follow_spec.rb Limit the number of people that can be followed from one account (#8807) 2018-10-04 17:36:11 +02:00
home_feed_spec.rb Fix performance of home feed regeneration (#12084) 2019-10-06 22:11:17 +02:00
identity_spec.rb Add spec for Identity.find_for_oauth (#9441) 2018-12-06 17:38:49 +09:00
import_spec.rb Validate data of Imports (#4782) 2017-09-02 20:45:42 +02:00
invite_spec.rb Change account suspensions to be reversible by default (#14726) 2020-09-15 14:37:58 +02:00
ip_block_spec.rb Add IP-based rules (#14963) 2020-10-12 16:33:49 +02:00
list_account_spec.rb Lint pass (#8876) 2018-10-04 12:36:53 +02:00
list_spec.rb Lint pass (#8876) 2018-10-04 12:36:53 +02:00
marker_spec.rb Add timeline read markers API (#11762) 2019-09-06 13:55:51 +02:00
media_attachment_spec.rb Fix media attachments without file being uploadable (#12562) 2020-01-23 21:40:03 +01:00
mention_spec.rb add more tests to models 2017-04-05 00:29:56 +02:00
mute_spec.rb Lint pass (#8876) 2018-10-04 12:36:53 +02:00
notification_spec.rb Add follow_request notification type (#12198) 2019-12-01 17:25:29 +01:00
one_time_key_spec.rb Add E2EE API (#13820) 2020-06-02 19:24:53 +02:00
poll_spec.rb Add polls (#10111) 2019-03-03 22:18:23 +01:00
poll_vote_spec.rb Implement pending tests (#11415) 2019-07-27 10:24:26 +02:00
preview_card_spec.rb Lint pass (#8876) 2018-10-04 12:36:53 +02:00
public_feed_spec.rb Refactor how public and tag timelines are queried (#14728) 2020-09-07 11:02:04 +02:00
relationship_filter_spec.rb Fix followings list order | Issue #13538 (#13676) 2020-05-08 20:17:16 +02:00
relay_spec.rb Add federation relay support (#7998) 2018-07-13 02:16:06 +02:00
remote_follow_spec.rb Fix uncaught domain normalization error in remote follow (#11703) 2019-08-30 02:19:17 +02:00
report_filter_spec.rb Add coverage for ReportFilter and AccountFilter (#3236) 2017-05-22 15:50:58 -04:00
report_spec.rb Bump faker from 1.9.6 to 2.1.0 and update faker api (#11489) 2019-08-06 15:33:03 +02:00
scheduled_status_spec.rb Add scheduled statuses (#9706) 2019-01-05 12:43:28 +01:00
session_activation_spec.rb Add tests for SessionActivation (#5668) 2017-11-13 09:54:48 +09:00
setting_spec.rb Isolate each specs for cache store (#6450) 2018-02-17 22:35:05 +01:00
site_upload_spec.rb Add a test for SiteUpload#cache_key (#5685) 2017-11-14 20:44:11 +01:00
status_pin_spec.rb Append '.test' to hostname in stub data (#7260) 2018-04-25 14:12:28 +02:00
status_spec.rb Refactor how public and tag timelines are queried (#14728) 2020-09-07 11:02:04 +02:00
status_stat_spec.rb Extract counters from accounts table to account_stats table (#9295) 2018-11-19 00:43:52 +01:00
system_key_spec.rb Add E2EE API (#13820) 2020-06-02 19:24:53 +02:00
tag_feed_spec.rb Refactor how public and tag timelines are queried (#14728) 2020-09-07 11:02:04 +02:00
tag_spec.rb Fix hashtags being split by ZWNJ character (#11821) 2019-09-13 16:01:26 +02:00
trending_tags_spec.rb Change trending hashtags to not disappear instantly after midnight (#11712) 2019-09-02 18:11:13 +02:00
unavailable_domain_spec.rb Change delivery failure tracking to work with hostnames instead of URLs (#13437) 2020-04-15 20:33:24 +02:00
user_invite_request_spec.rb Add "why do you want to join" field to invite requests (#10524) 2019-04-09 23:06:30 +09:00
user_spec.rb Add WebAuthn as an alternative 2FA method (#14466) 2020-08-24 16:46:27 +02:00
webauthn_credentials_spec.rb Fix validates :sign_count of WebauthnCredential (#14806) 2020-09-16 20:16:46 +02:00