* Fix PostgreSQL load when linking in announcements
Fixes#13245 by caching status lookups
Since statuses are supposed to be known already and we only
need their URLs and a few other things, caching them should
be fine.
Since it's only used by announcements so far, there won't
be much statuses to cache.
* Perform status lookup when saving announcements, not when rendering them
* Change EntityCache#status to fetch URLs instead of looking into the database
* Move announcement link lookup to publishing worker
* Address issues pointed out during review
Also:
- Fix locks not being removed when jobs go to the dead job queue
- Add UI for managing locks to the Sidekiq dashboard
- Remove unused Sidekiq workers
Fix#13349
Change `all_day` to be a visual client-side cue only
Publish immediately if `scheduled_at` is in the past
Add `published_at` and `updated_at` to announcements JSON
* Add announcements
Fix#11006
* Add reactions to announcements
* Add admin UI for announcements
* Add unit tests
* Fix issues
- Add `with_dismissed` param to announcements API
- Fix end date not being formatted when time range is given
- Fix announcement delete causing reactions to send streaming updates
- Fix announcements container growing too wide and mascot too small
- Fix `all_day` being settable when no time range is given
- Change text "Update" to "Announcement"
* Fix scheduler unpublishing announcements before they are due
* Fix filter params not being passed to announcements filter
* Fix being able to follow oneself by moving to an account that was following the old one
* Add specs
* Add spec to catch MoveWorker issue with local followers following both accounts
* Fix move worker breaking when a local account follows both source and target accounts
* Fix migration from remote to local account not sending Undo Follow
* Fix show_reblogs not being preserved for moved account's followers
* Add soft delete for statuses to allow them to appear instant
* Allow reporting soft-deleted statuses and show them in the admin UI
* Change index for getting an account's statuses
The reason for unattaching media instead of removing it is to support
delete & redraft functionality, but remote or staff-removed statuses
will never be redrafted, so the media should be deleted immediately
* Add database columns for adding notes to domain blocks/restrctions
* Add admin UI to set private and public comments when blocking a domain
* Add text for private and public comments on domain blocks
* Show domain block comments in admin UI
* Add comments to the domain block undo page
* Make UnblockDomainService more robust regarding upgraded domain blocks
* Allow editing domain blocks
* Rename button from “undo domain block” to “view domain block” in account admin UI
* Change test to unsilence silenced users from upgraded blocks
* Add `tootctl preview_cards remove`
* fix code style
* Remove `Scheduler::PreviewCardsCleanupScheduler` file
* fix code style again
Add exclude case where image_file_name is blank
* Added a function to output confirmation if the specified number of days is less than 2 weeks
- Change the maximum count of retry for web push notification (Default -> 5).
- In case of high load of subscribe server, the retries will be repeated many times.
- Because the retries occupy the default queue, maximum retry count should be reduced.
* Remove Salmon and PubSubHubbub endpoints
* Add error when trying to follow OStatus accounts
* Fix new accounts not being created in ResolveAccountService
* Add request pool to improve delivery performance
Fix#7909
* Ensure connection is closed when exception interrupts execution
* Remove Timeout#timeout from socket connection
* Fix infinite retrial loop on HTTP::ConnectionError
* Close sockets on failure, reduce idle time to 90 seconds
* Add MAX_REQUEST_POOL_SIZE option to limit concurrent connections to the same server
* Use a shared pool size, 512 by default, to stay below open file limit
* Add some tests
* Add more tests
* Reduce MAX_IDLE_TIME from 90 to 30 seconds, reap every 30 seconds
* Use a shared pool that returns preferred connection but re-purposes other ones when needed
* Fix wrong connection being returned on subsequent calls within the same thread
* Reduce mutex calls on flushes from 2 to 1 and add test for reaping
HTTP 401 responses returned by Mastodon's inbox controller may
be temporary if, for instance, the requesting user's actor/key json
could not be retrieved in a timely fashion. This changes allow retries
instead of dropping the message entirely.
Also added HTTP 408 as that error is by nature temporary.
* Do not retry processing ActivityPub jobs raising validation errors
Jobs yielding validation errors most probably won't ever be accepted,
so it makes sense not to clutter the queues with retries.
* Lower RecordInvalid error reporting to debug log level
* Remove trailing whitespace
* Refactor imports
* Export show_reblogs when exporting list of followed users
* Add support for importing show_reblogs with following collection
* Fix tests
* Fix poll update handler calling method was that was not available
Fix regression from #10209
* Refactor VoteService
* Refactor ActivityPub::DistributePollUpdateWorker and optimize it
* Fix typo
* Fix typo
* Process incoming poll tallies update
* Send Update on poll vote
* Do not send Updates for a poll more often than once every 3 minutes
* Include voters in people to notify of results update
* Schedule closing poll worker on poll creation
* Add new notification type for ending polls
* Add front-end support for ended poll notifications
* Fix UpdatePollSerializer
* Fix Updates not being triggered by local votes
* Fix tests failure
* Fix web push notifications for closing polls
* Minor cleanup
* Notify voters of both remote and local polls when those close
* Fix delivery of poll updates to mentioned accounts and voters
* Fetch up to 5 replies when discovering a new remote status
This is used for resolving threads downwards. The originating
server must add a “replies” attributes with such replies for it to
be useful.
* Add some tests for ActivityPub::FetchRepliesWorker
* Add specs for ActivityPub::FetchRepliesService
* Serialize up to 5 public self-replies for ActivityPub notes
* Add specs for ActivityPub::NoteSerializer
* Move exponential backoff logic to a worker concern
* Fetch first page of paginated collections when fetching thread replies
* Add specs for paginated collections in replies
* Move Note replies serialization to a first CollectionPage
The collection isn't actually paginable yet as it has no id nor
a `next` field. This may come in another PR.
* Use pluck(:uri) instead of map(&:uri) to improve performances
* Fix fetching replies when they are in a CollectionPage
Reject those from accounts with no local followers, from relays
that are not enabled, which do not address local accounts and are
not replies to accounts that do have local followers
- Reduce time-to-digest from 20 to 7 days
- Fetch mentions starting from +1 day since last login
- Fix case when last login is more recent than last e-mail
- Do not render all mentions, only 40, but show number in subject
- Do not send digest to moved accounts
- Do send digest to silenced accounts
* Do not LDS-sign Follow, Accept, Reject, Undo, Block
* Do not use LDS for Create activities of private toots
* Minor cleanup
* Ignore unsigned activities instead of misattributing them
* Use status.distributable? instead of querying visibility directly
- Some associations were missing from the clean-up
- Some attributes were not reset on suspension
- Skip federation and streaming deletes when purging a dead domain
- Move account association definitions to concern
* Eliminate extra accounts select query from FollowService
* Optimistically update follow state in web UI and hide loading bar
Fix#6205
* Asynchronize NotifyService in FollowService
And fix failing test
* Skip Webfinger resolve routine when called from FollowService if possible
If an account is ActivityPub, then webfinger re-resolving is not necessary
when called from FollowService. Improve options of ResolveAccountService
* Add silent column to mentions
* Save silent mentions in ActivityPub Create handler and optimize it
Move networking calls out of the database transaction
* Add "limited" visibility level masked as "private" in the API
Unlike DMs, limited statuses are pushed into home feeds. The access
control rules between direct and limited statuses is almost the same,
except for counter and conversation logic
* Ensure silent column is non-null, add spec
* Ensure filters don't check silent mentions for blocks/mutes
As those are "this person is also allowed to see" rather than "this
person is involved", therefore does not warrant filtering
* Clean up code
* Use Status#active_mentions to limit returned mentions
* Fix code style issues
* Use Status#active_mentions in Notification
And remove stream_entry eager-loading from Notification
* Add conversations API
* Add web UI for conversations
* Add test for conversations API
* Add tests for ConversationAccount
* Improve web UI
* Rename ConversationAccount to AccountConversation
* Remove conversations on block and mute
* Change last_status_id to be a denormalization of status_ids
* Add optimistic locking
* Verify link ownership with rel="me"
* Add explanation about verification to UI
* Perform link verifications
* Add click-to-copy widget for verification HTML
* Redesign edit profile page
* Redesign forms
* Improve responsive design of settings pages
* Restore landing page sign-up form
* Fix typo
* Support <link> tags, add spec
* Fix links not being verified on first discovery and passive updates
* If an Update is signed with known key, skip re-following procedure
Because it means the remote actor did *not* lose their database
* Add CLI method for rotating keys
bin/tootctl accounts rotate [USERNAME]
Generates a new RSA key per account and sends out an Update activity
signed with the old key.
* Key rotation: Space out Update fan-outs every 5 minutes per 1000 accounts
* Skip suspended accounts in key rotation
* Fix uncaching worker
* Revert to using Paperclip's filesystem backend instead of fog-local
fog-local has lots of concurrency issues, causing failure to delete files,
dangling file records, and spurious errors UncacheMediaWorker
* Send rejections to followers when user hides domain they're on
* Use account domain blocks for "authorized followers" action
Replace soft-blocking (block & unblock) behaviour with follow rejection
* Split sync and async work of account domain blocking
Do not create domain block when removing followers by domain, that
is probably unexpected from the user's perspective.
* Adjust confirmation message for domain block
* yarn manage:translations
* Speed up some rake tasks by moving execution to Sidekiq
mastodon:media:remove_silenced
mastodon:media:remove_remote
mastodon:media:redownload_avatars
mastodon:feeds:build
* Fix code style issue
* Do not raise delivery failure on 4xx errors, increase stoplight threshold
Stoplight failure threshold from 3 to 10
Status code 429 will raise a failure/get retried
* Oops
- POST /api/v1/push/subscription
- PUT /api/v1/push/subscription
- DELETE /api/v1/push/subscription
- New OAuth scope: "push" (required for the above methods)
* Revert "Fixes/do not override timestamps (#7331)"
This reverts commit 581a5c9d29.
* Document Snowflake ID corner-case a bit more
Snowflake IDs are used for two purposes: making object identifiers harder to
guess and ensuring they are in chronological order. For this reason, they
are based on the `created_at` attribute of the object.
Unfortunately, inserting items with older snowflakes IDs will break the
assumption of consumers of the paging APIs that new items will always have
a greater identifier than the last seen one.
* Add `override_timestamps` virtual attribute to not correlate snowflake ID with created_at
* Do not override timestamps for incoming toots
* Remove every reference to override_timestamps
Statuses are now created with the announced publishing date
and are only pushed to timelines if that date is at most
6 hours earlier than the time at which it is processed.
* Revert "Weblate translations 20180503 (#7325)"
This reverts commit dfa6bccb64.
* Revert "Prevent timeline from moving when cursor is hovering over it (fixes#7278) (#7327)"
This reverts commit 58852695c8.
* Revert "Add pry-byebug (#7307)"
This reverts commit ab773e4d5f.
* Revert "Do not override timestamps for incoming toots (#7326)"
This reverts commit bd36791832.
Offload creation of local notifications to a worker. Remove two
redundant SQL queries from ProcessMentionsService, remove n+1
XML/JSON serialization via memoization
* No need to re-require sidekiq plugins, they are required via Gemfile
* Add derailed_benchmarks tool, no need to require TTY gems in Gemfile
* Replace ruby-oembed with FetchOEmbedService
Reduce startup by 45382 allocated objects
* Remove preloaded JSON-LD in favour of caching HTTP responses
Reduce boot RAM by about 6 MiB
* Fix tests
* Fix test suite by stubbing out JSON-LD contexts
* Adjust privacy policy to be more specific to Mastodon
Fix#6613
* Change data retention of IP addresses from 5 years to 1 year
* Add even more information
* Remove all (now invalid) translations of the privacy policy
* Add information about archive takeout, remove pointless consent section
* Emphasis on DM privacy
* Improve wording
* Add line about data use for moderation purposes
to_s method of HTTP::Response keeps blocking while it receives the whole
content, no matter how it is big. This means it may waste time to receive
unacceptably large files. It may also consume memory and disk in the
process. This solves the inefficency by checking response length while
receiving.
