Commit graph

9984 commits

Author SHA1 Message Date
Claire ad06423e71 Fix some old migration scripts (#17394)
* Fix some old migration scripts

* Fix edge case in two-step migration from older releases
2022-01-30 23:00:21 +01:00
Claire 82a490ac7f Fix filtering DMs from non-followed users (#17042) 2022-01-28 22:53:15 +01:00
Claire dbe5e29e38 Fix upload of remote media with OpenStack Swift sometimes failing (#16998)
Under certain conditions, files fetched from remotes trigger an error when
being uploaded using OpenStack Swift. This is because in some cases, the
remote server will not return a content-length, so our ResponseWithLimitAdapter
will hold a `nil` value for `#size`, which will lead to an invalid value
for the Content-Length header of the Swift API call.

This commit fixes that by taking the size from the actually-downloaded file
size rather than the upstream-provided Content-Length header value.
2022-01-28 22:53:15 +01:00
Claire ff19501e50 Fix confusing error when webfinger request returns empty document (#16986)
For some reason, some misconfigured servers return an empty document when
queried over webfinger. Since an empty document does not lead to a parse
error, the error is not caught properly and triggers uncaught exceptions
later on.

This PR fixes that by immediately erroring out with `Webfinger::Error` on
getting an empty response.
2022-01-28 22:53:15 +01:00
Claire e02a7cfeb2 Fix AccountNote not having a maximum length (#16942) 2022-01-28 22:53:15 +01:00
Eugen Rochko 5d72e6c4d0 Fix login being broken due to inaccurately applied backport fix in 3.4.2
See #16943
2022-01-28 22:53:15 +01:00
Claire b6b19419e2 Fix reviving revoked sessions and invalidating login (#16943)
Up until now, we have used Devise's Rememberable mechanism to re-log users
after the end of their browser sessions. This mechanism relies on a signed
cookie containing a token. That token was stored on the user's record,
meaning it was shared across all logged in browsers, meaning truly revoking
a browser's ability to auto-log-in involves revoking the token itself, and
revoking access from *all* logged-in browsers.

We had a session mechanism that dynamically checks whether a user's session
has been disabled, and would log out the user if so. However, this would only
clear a session being actively used, and a new one could be respawned with
the `remember_user_token` cookie.

In practice, this caused two issues:
- sessions could be revived after being closed from /auth/edit (security issue)
- auto-log-in would be disabled for *all* browsers after logging out from one
  of them

This PR removes the `remember_token` mechanism and treats the `_session_id`
cookie/token as a browser-specific `remember_token`, fixing both issues.
2022-01-28 22:53:15 +01:00
Claire a19aec0f48 Fix newlines in accout notes added by the Move handler (#16415)
* Fix newlines in account notes added by the move handler

* Make MoveWorker more robust
2022-01-28 22:53:15 +01:00
Takeshi Umeda 1ddbefb787 Fix when MoveWorker cannot get locale from remote account (#16576) 2022-01-28 22:53:15 +01:00
Takeshi Umeda 85c845c001 Fix invalid blurhash handling in Create activity (#16583) 2022-01-28 22:53:15 +01:00
Claire 80ca4fdb3c Fix crash when encountering invalid account fields (#16598)
* Add test

* Fix crash when encountering invalid account fields
2022-01-28 22:53:15 +01:00
Claire 168272fe61 Fix remotely-suspended accounts' toots being merged back into timelines (#16628)
* Fix remotely-suspended accounts' toots being merged back into timelines

* Mark remotely-deleted accounts as remotely suspended
2022-01-28 22:53:15 +01:00
Claire 2c02cb59ef Fix webauthn secure key authentication (#16792)
* Add tests

* Fix webauthn secure key authentication

Fixes #16769
2022-01-28 22:53:15 +01:00
Claire edc55002cf Fix authentication failures after going halfway through a sign-in attempt (#16607)
* Add tests

* Add security-related tests

My first (unpublished) attempt at fixing the issues introduced (extremely
hard-to-exploit) security vulnerabilities, addressing them in a test.

* Fix authentication failures after going halfway through a sign-in attempt

* Refactor `authenticate_with_sign_in_token` and `authenticate_with_two_factor` to make the two authentication steps more obvious
2022-01-28 22:53:15 +01:00
Claire e10920eb20 Fix processing mentions to domains with non-ascii TLDs (#16689)
Fixes #16602
2022-01-28 22:53:14 +01:00
Claire d33b361000 Fix addressing of remote groups' followers (#16700)
Fixes #16699
2022-01-28 22:52:42 +01:00
Claire 5b07f4e90e Fix some link previews being incorrectly generated from other prior links (#16885)
* Add tests

* Fix some link previews being incorrectly generated from different prior links

PR #12403 added a cache to avoid redundant queries when the OEmbed endpoint can
be guessed from the URL. This caching mechanism is not perfectly correct as
there is no guarantee that all pages from a given domain share the same
OEmbed provider endpoint.

This PR prevents the FetchOEmbedService from caching OEmbed endpoint that
cannot be generalized by replacing a fully-qualified URL from the endpoint's
parameters, greatly reducing the number of incorrect cached generalizations.
2022-01-28 22:52:42 +01:00
Claire 0994c4b11a Fix "bundle exec rails mastodon:setup" crashing in some circumstances (#16976)
Fix regression from #16896
2022-01-28 22:52:42 +01:00
Claire d2cdfe92ed Fix mastodon:setup to take dotenv/docker-compose differences into account (#16896)
In order to work around https://github.com/mastodon/mastodon/issues/16895,
add a warning to .env.production.sample, and change the mastodon:setup rake
task to:
- output a warning if a variable will be interpreted differently by dotenv
  and docker-compose
- ensure the printed config is compatible with docker-compose
2022-01-28 22:52:42 +01:00
Claire 6c344d90c7 Fix tootctl self-destruct not sending Delete activities for recently-suspended accounts (#16688)
* Do not block existing users' emails on self-destruct

That is wasteful and unintuitive

* Do not close registrations when running tootctl self-destruct with --dry-run

* Close registrations on self-destruct regardless of known remote accounts

* Fix tootctl self-destruct not sending Deletes for recently-suspended accounts

* Suspend local users even if no remote account is known

* Do not show scary confirmation text if ran with --dry-run
2022-01-28 22:52:42 +01:00
Claire c0b2c2c166 Fix serialization of followers/following counts when user hides their network (#16418)
* Add tests

* Fix serialization of followers/following counts when user hides their network

Fixes #16382

Signed-off-by: Claire <claire.github-309c@sitedethib.com>
2022-01-28 22:52:42 +01:00
Claire 19edb7a3f4 Fix followers synchronization mechanism not working when URI has empty path (#16744)
Follow-up to #16510, forgot the controller exposing the actual followers…
2022-01-28 22:52:42 +01:00
Claire 53e4efd07d Fix followers synchronization mechanism not working when URI has empty path (#16510)
* Fix followers synchronization mechanism not working when URI has empty path

To my knowledge, there is no current implementation on the fediverse
that can use bare domains (e.g., actor is at https://example.org instead of
something like https://example.org/actor) that also plans to support the
followers synchronization mechanism. However, Mastodon's current implementation
would exclude such accounts from followers list.

Also adds tests and rename them to reflect the proper method names.

* Move url prefix regexp to its own constant
2022-01-28 22:52:41 +01:00
Claire 3f882c2c17 Fix scheduled statuses decreasing statuses counts (#16791)
* Add tests

* Fix scheduled statuses decreasing statuses counts

Fixes #16774
2022-01-28 22:39:48 +01:00
Holger 1cfa2bdb03 use relative path for scope (#16714)
Use relative path for `scope` in web manifest to allow users use PWA correctly via alternate domains.
2022-01-28 22:39:48 +01:00
Claire 1b32c001bc Fix migration script not being able to run if it fails midway (#16312)
* Fix migration script not being able to run if it fails midway

* Fix old migration script

* Fix old migration script

* Refactor CorruptionError
2022-01-28 22:39:48 +01:00
Claire 31d9aa8ed0 Fix media proxy RedisLocks auto-releasing too fast (#16291)
Follow-up to #16276
2022-01-28 22:39:48 +01:00
Claire 4d41c91335 Fix some RedisLocks auto-releasing too fast (#16276)
* Fix Delete and Create-related locks expiring too fast

Fixes #16238

By default, RedisLock expires after 10 seconds, which may not be enough to
process statuses, especially when those have attached media files.

This commit extends those 10 seconds to 15 minutes, which should be plenty
enough to handle any status, while being short enough to not waste many
sidekiq job retries in the exceedingly rare case in which a sidekiq process
would crash when processing a `Create` or `Delete`.

* Fix other RedisLock autorelease durations

Fixes #15645

- things that only perform a few simple database queries (e.g. finding and
  saving a record) have been left unchanged, so they'll still use the default
  10s duration
- things that perform significantly more complex database queries have been
  changed to a 5 minutes timeout
- things that perform multiple HTTP queries have been changed to a 15 minutes
  timeout
2022-01-28 22:39:48 +01:00
Jeong Arm 678e0ad419 Remove set-cookie header on custom.css (#16314)
* Remove set-cookie header on custom.css

* Additional fix for set-cookie
2022-01-28 22:39:48 +01:00
Claire c89809afc5 Fix some IDs in instance actor outbox (#16343) 2022-01-28 22:39:48 +01:00
Eugen Rochko a319fd3cc4 Fix app name, website and redirect URIs not having a maximum length (#16042)
Fix app scopes not being validated
2022-01-28 22:39:48 +01:00
Claire 24dee67d32 Create instance actor if it hasn't been properly seeded (#15693)
An uncommon but somewhat difficult to digagnose issue is dealing with
improperly-seeded databases. In such cases, instance-signed fetches will
fail with a ActiveRecord::RecordNotFound error, usually caught and handled
as generic 404, leading people to think the remote resource itself has not
been found, while it's the local instance actor that does not exist.

This commit changes the code so that failure to find the instance actor
automatically creates a new one, so that improperly-seeded databases do
not cause any issue.
2022-01-28 22:39:48 +01:00
kaiyou 07042a0913 Support clock drift in Omniauth SAML provider (#15511)
The setting is not well documented by the provider, but allows for
clock skew between SP and IDP, see:
https://github.com/omniauth/omniauth-saml/blob/master/spec/omniauth/strategies/saml_spec.rb

Co-authored-by: kaiyou <dev@kaiyou.fr>
2022-01-28 22:39:48 +01:00
Eugen Rochko 4978d387ee WIP (#15222) 2022-01-28 22:39:48 +01:00
Stanislas 0951c691ff tootctl emoji import: case insensitive duplicate check (#15738) 2022-01-28 22:39:48 +01:00
Sophie Parker fc4b9856f8 Improve Emoji import (fix #15429) (#15430)
* Improve Emoji import

Skip macOS '._' shadow files in tar archive to speed up import

* Fix codeclimate format issue with whitespace

* Update lib/mastodon/emoji_cli.rb

suggestions from Gargron to improve comment

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>

* Update emoji_cli.rb

Remove extraneous comment (macOS-specific comment now with correct line)

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
2022-01-28 22:39:48 +01:00
ThibG add7b9f82e Fix “tootctl accounts unfollow” (#15639)
Fixes #15635

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-01-28 22:39:48 +01:00
Claire 9437e9f0b9 Fix custom CSS when CDN_HOST is set (#15927) 2022-01-28 22:39:48 +01:00
Levi Bard 0fe57a9140 Fix muting users with duration via the REST api (#15516) 2022-01-28 22:39:48 +01:00
Claire 3a4d9f1f2d Fix not being able to change world filter expiration back to “Never” (#15858)
Fixes #15849
2022-01-28 22:39:48 +01:00
ThibG 78d5bda973 Fix race conditions on account migration creation (#15597)
* Atomically check for processing lock in Move handler

* Prevent race condition when creating account migrations

Fixes #15595

* Add tests

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-01-28 22:39:48 +01:00
ThibG 4b025cf7e6 Fix sign-up restrictions based on IP addresses not being enforced (#15607)
Fixes #15606

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-01-28 22:39:48 +01:00
Eugen Rochko 4bd8dc09d8 Fix reports of already suspended accounts being recorded (#16047) 2022-01-28 22:39:48 +01:00
Claire 3799fd17ba Fix edge case where accepted follow cannot be processed because of follow limit (#16098) 2022-01-28 22:39:48 +01:00
Claire 53814b2b31 Fix blocking someone not clearing up list feeds (#16205) 2022-01-28 22:39:48 +01:00
ThibG 2012c5ae50 Fix maintenance script not re-indexing some indexes on textual values (#15515)
* Fix maintenance script not re-indexing some indexes on textual values

Fixes #15475

* Refresh instance view at the end of the maintenance script run

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-01-28 22:39:48 +01:00
Claire 9aa7286c8f Change deduplication order of tootctl maintenance fix-duplicates (#15923)
Hopefully fixes #15922

Also update support up to latest database schema version
2022-01-28 22:39:48 +01:00
Eugen Rochko 4b9a0cfe5e Fix media processing getting stuck on too much stdin/stderr (#16136)
* Fix media processing getting stuck on too much stdin/stderr

See thoughtbot/terrapin#5

* Remove dependency on paperclip-av-transcoder gem

* Remove dependency on streamio-ffmpeg gem

* Disable stdin on ffmpeg process
2022-01-28 22:39:48 +01:00
Eugen Rochko b593a7da8c Fix database serialization failure returning HTTP 500 (#16101)
Database serialization failure occurs when a read-replica is used
and a query takes long enough that rows on the primary database
become unavailable. It should return HTTP 503 as it is temporary.

Re-order rescue definitions according to their status codes
2022-01-28 22:39:48 +01:00
ThibG 48b25e457d Fix /activity endpoint not require signature in authorized fetch mode (#15592)
Fixes #15589

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-01-28 22:39:48 +01:00