Commit graph

9987 commits

Author SHA1 Message Date
Yusuke Nakamura
c2bd6e90b4 Build container image by GitHub Actions (#16973)
* Build container image by GitHub Actions

* Trigger docker build only pushed to main branch

* Tweak tagging imgae

- "edge" is the main branch
- "latest" is the tagged latest release
2022-01-31 00:03:05 +01:00
Claire
7619689fcb Add more advanced migration tests
- populate the database with some data when testing migrations
- try both one-step and two-step migrations (`SKIP_POST_DEPLOYMENT_MIGRATIONS`)
2022-01-30 23:23:00 +01:00
Claire
5ec943f85d Fix edge case in migration helpers that caused crash because of PostgreSQL quirks (#17398) 2022-01-30 23:00:21 +01:00
Claire
ad06423e71 Fix some old migration scripts (#17394)
* Fix some old migration scripts

* Fix edge case in two-step migration from older releases
2022-01-30 23:00:21 +01:00
Claire
82a490ac7f Fix filtering DMs from non-followed users (#17042) 2022-01-28 22:53:15 +01:00
Claire
dbe5e29e38 Fix upload of remote media with OpenStack Swift sometimes failing (#16998)
Under certain conditions, files fetched from remotes trigger an error when
being uploaded using OpenStack Swift. This is because in some cases, the
remote server will not return a content-length, so our ResponseWithLimitAdapter
will hold a `nil` value for `#size`, which will lead to an invalid value
for the Content-Length header of the Swift API call.

This commit fixes that by taking the size from the actually-downloaded file
size rather than the upstream-provided Content-Length header value.
2022-01-28 22:53:15 +01:00
Claire
ff19501e50 Fix confusing error when webfinger request returns empty document (#16986)
For some reason, some misconfigured servers return an empty document when
queried over webfinger. Since an empty document does not lead to a parse
error, the error is not caught properly and triggers uncaught exceptions
later on.

This PR fixes that by immediately erroring out with `Webfinger::Error` on
getting an empty response.
2022-01-28 22:53:15 +01:00
Claire
e02a7cfeb2 Fix AccountNote not having a maximum length (#16942) 2022-01-28 22:53:15 +01:00
Eugen Rochko
5d72e6c4d0 Fix login being broken due to inaccurately applied backport fix in 3.4.2
See #16943
2022-01-28 22:53:15 +01:00
Claire
b6b19419e2 Fix reviving revoked sessions and invalidating login (#16943)
Up until now, we have used Devise's Rememberable mechanism to re-log users
after the end of their browser sessions. This mechanism relies on a signed
cookie containing a token. That token was stored on the user's record,
meaning it was shared across all logged in browsers, meaning truly revoking
a browser's ability to auto-log-in involves revoking the token itself, and
revoking access from *all* logged-in browsers.

We had a session mechanism that dynamically checks whether a user's session
has been disabled, and would log out the user if so. However, this would only
clear a session being actively used, and a new one could be respawned with
the `remember_user_token` cookie.

In practice, this caused two issues:
- sessions could be revived after being closed from /auth/edit (security issue)
- auto-log-in would be disabled for *all* browsers after logging out from one
  of them

This PR removes the `remember_token` mechanism and treats the `_session_id`
cookie/token as a browser-specific `remember_token`, fixing both issues.
2022-01-28 22:53:15 +01:00
Claire
a19aec0f48 Fix newlines in accout notes added by the Move handler (#16415)
* Fix newlines in account notes added by the move handler

* Make MoveWorker more robust
2022-01-28 22:53:15 +01:00
Takeshi Umeda
1ddbefb787 Fix when MoveWorker cannot get locale from remote account (#16576) 2022-01-28 22:53:15 +01:00
Takeshi Umeda
85c845c001 Fix invalid blurhash handling in Create activity (#16583) 2022-01-28 22:53:15 +01:00
Claire
80ca4fdb3c Fix crash when encountering invalid account fields (#16598)
* Add test

* Fix crash when encountering invalid account fields
2022-01-28 22:53:15 +01:00
Claire
168272fe61 Fix remotely-suspended accounts' toots being merged back into timelines (#16628)
* Fix remotely-suspended accounts' toots being merged back into timelines

* Mark remotely-deleted accounts as remotely suspended
2022-01-28 22:53:15 +01:00
Claire
2c02cb59ef Fix webauthn secure key authentication (#16792)
* Add tests

* Fix webauthn secure key authentication

Fixes #16769
2022-01-28 22:53:15 +01:00
Claire
edc55002cf Fix authentication failures after going halfway through a sign-in attempt (#16607)
* Add tests

* Add security-related tests

My first (unpublished) attempt at fixing the issues introduced (extremely
hard-to-exploit) security vulnerabilities, addressing them in a test.

* Fix authentication failures after going halfway through a sign-in attempt

* Refactor `authenticate_with_sign_in_token` and `authenticate_with_two_factor` to make the two authentication steps more obvious
2022-01-28 22:53:15 +01:00
Claire
e10920eb20 Fix processing mentions to domains with non-ascii TLDs (#16689)
Fixes #16602
2022-01-28 22:53:14 +01:00
Claire
d33b361000 Fix addressing of remote groups' followers (#16700)
Fixes #16699
2022-01-28 22:52:42 +01:00
Claire
5b07f4e90e Fix some link previews being incorrectly generated from other prior links (#16885)
* Add tests

* Fix some link previews being incorrectly generated from different prior links

PR #12403 added a cache to avoid redundant queries when the OEmbed endpoint can
be guessed from the URL. This caching mechanism is not perfectly correct as
there is no guarantee that all pages from a given domain share the same
OEmbed provider endpoint.

This PR prevents the FetchOEmbedService from caching OEmbed endpoint that
cannot be generalized by replacing a fully-qualified URL from the endpoint's
parameters, greatly reducing the number of incorrect cached generalizations.
2022-01-28 22:52:42 +01:00
Claire
0994c4b11a Fix "bundle exec rails mastodon:setup" crashing in some circumstances (#16976)
Fix regression from #16896
2022-01-28 22:52:42 +01:00
Claire
d2cdfe92ed Fix mastodon:setup to take dotenv/docker-compose differences into account (#16896)
In order to work around https://github.com/mastodon/mastodon/issues/16895,
add a warning to .env.production.sample, and change the mastodon:setup rake
task to:
- output a warning if a variable will be interpreted differently by dotenv
  and docker-compose
- ensure the printed config is compatible with docker-compose
2022-01-28 22:52:42 +01:00
Claire
6c344d90c7 Fix tootctl self-destruct not sending Delete activities for recently-suspended accounts (#16688)
* Do not block existing users' emails on self-destruct

That is wasteful and unintuitive

* Do not close registrations when running tootctl self-destruct with --dry-run

* Close registrations on self-destruct regardless of known remote accounts

* Fix tootctl self-destruct not sending Deletes for recently-suspended accounts

* Suspend local users even if no remote account is known

* Do not show scary confirmation text if ran with --dry-run
2022-01-28 22:52:42 +01:00
Claire
c0b2c2c166 Fix serialization of followers/following counts when user hides their network (#16418)
* Add tests

* Fix serialization of followers/following counts when user hides their network

Fixes #16382

Signed-off-by: Claire <claire.github-309c@sitedethib.com>
2022-01-28 22:52:42 +01:00
Claire
19edb7a3f4 Fix followers synchronization mechanism not working when URI has empty path (#16744)
Follow-up to #16510, forgot the controller exposing the actual followers…
2022-01-28 22:52:42 +01:00
Claire
53e4efd07d Fix followers synchronization mechanism not working when URI has empty path (#16510)
* Fix followers synchronization mechanism not working when URI has empty path

To my knowledge, there is no current implementation on the fediverse
that can use bare domains (e.g., actor is at https://example.org instead of
something like https://example.org/actor) that also plans to support the
followers synchronization mechanism. However, Mastodon's current implementation
would exclude such accounts from followers list.

Also adds tests and rename them to reflect the proper method names.

* Move url prefix regexp to its own constant
2022-01-28 22:52:41 +01:00
Claire
3f882c2c17 Fix scheduled statuses decreasing statuses counts (#16791)
* Add tests

* Fix scheduled statuses decreasing statuses counts

Fixes #16774
2022-01-28 22:39:48 +01:00
Holger
1cfa2bdb03 use relative path for scope (#16714)
Use relative path for `scope` in web manifest to allow users use PWA correctly via alternate domains.
2022-01-28 22:39:48 +01:00
Claire
1b32c001bc Fix migration script not being able to run if it fails midway (#16312)
* Fix migration script not being able to run if it fails midway

* Fix old migration script

* Fix old migration script

* Refactor CorruptionError
2022-01-28 22:39:48 +01:00
Claire
31d9aa8ed0 Fix media proxy RedisLocks auto-releasing too fast (#16291)
Follow-up to #16276
2022-01-28 22:39:48 +01:00
Claire
4d41c91335 Fix some RedisLocks auto-releasing too fast (#16276)
* Fix Delete and Create-related locks expiring too fast

Fixes #16238

By default, RedisLock expires after 10 seconds, which may not be enough to
process statuses, especially when those have attached media files.

This commit extends those 10 seconds to 15 minutes, which should be plenty
enough to handle any status, while being short enough to not waste many
sidekiq job retries in the exceedingly rare case in which a sidekiq process
would crash when processing a `Create` or `Delete`.

* Fix other RedisLock autorelease durations

Fixes #15645

- things that only perform a few simple database queries (e.g. finding and
  saving a record) have been left unchanged, so they'll still use the default
  10s duration
- things that perform significantly more complex database queries have been
  changed to a 5 minutes timeout
- things that perform multiple HTTP queries have been changed to a 15 minutes
  timeout
2022-01-28 22:39:48 +01:00
Jeong Arm
678e0ad419 Remove set-cookie header on custom.css (#16314)
* Remove set-cookie header on custom.css

* Additional fix for set-cookie
2022-01-28 22:39:48 +01:00
Claire
c89809afc5 Fix some IDs in instance actor outbox (#16343) 2022-01-28 22:39:48 +01:00
Eugen Rochko
a319fd3cc4 Fix app name, website and redirect URIs not having a maximum length (#16042)
Fix app scopes not being validated
2022-01-28 22:39:48 +01:00
Claire
24dee67d32 Create instance actor if it hasn't been properly seeded (#15693)
An uncommon but somewhat difficult to digagnose issue is dealing with
improperly-seeded databases. In such cases, instance-signed fetches will
fail with a ActiveRecord::RecordNotFound error, usually caught and handled
as generic 404, leading people to think the remote resource itself has not
been found, while it's the local instance actor that does not exist.

This commit changes the code so that failure to find the instance actor
automatically creates a new one, so that improperly-seeded databases do
not cause any issue.
2022-01-28 22:39:48 +01:00
kaiyou
07042a0913 Support clock drift in Omniauth SAML provider (#15511)
The setting is not well documented by the provider, but allows for
clock skew between SP and IDP, see:
https://github.com/omniauth/omniauth-saml/blob/master/spec/omniauth/strategies/saml_spec.rb

Co-authored-by: kaiyou <dev@kaiyou.fr>
2022-01-28 22:39:48 +01:00
Eugen Rochko
4978d387ee WIP (#15222) 2022-01-28 22:39:48 +01:00
Stanislas
0951c691ff tootctl emoji import: case insensitive duplicate check (#15738) 2022-01-28 22:39:48 +01:00
Sophie Parker
fc4b9856f8 Improve Emoji import (fix #15429) (#15430)
* Improve Emoji import

Skip macOS '._' shadow files in tar archive to speed up import

* Fix codeclimate format issue with whitespace

* Update lib/mastodon/emoji_cli.rb

suggestions from Gargron to improve comment

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>

* Update emoji_cli.rb

Remove extraneous comment (macOS-specific comment now with correct line)

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
2022-01-28 22:39:48 +01:00
ThibG
add7b9f82e Fix “tootctl accounts unfollow” (#15639)
Fixes #15635

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-01-28 22:39:48 +01:00
Claire
9437e9f0b9 Fix custom CSS when CDN_HOST is set (#15927) 2022-01-28 22:39:48 +01:00
Levi Bard
0fe57a9140 Fix muting users with duration via the REST api (#15516) 2022-01-28 22:39:48 +01:00
Claire
3a4d9f1f2d Fix not being able to change world filter expiration back to “Never” (#15858)
Fixes #15849
2022-01-28 22:39:48 +01:00
ThibG
78d5bda973 Fix race conditions on account migration creation (#15597)
* Atomically check for processing lock in Move handler

* Prevent race condition when creating account migrations

Fixes #15595

* Add tests

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-01-28 22:39:48 +01:00
ThibG
4b025cf7e6 Fix sign-up restrictions based on IP addresses not being enforced (#15607)
Fixes #15606

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-01-28 22:39:48 +01:00
Eugen Rochko
4bd8dc09d8 Fix reports of already suspended accounts being recorded (#16047) 2022-01-28 22:39:48 +01:00
Claire
3799fd17ba Fix edge case where accepted follow cannot be processed because of follow limit (#16098) 2022-01-28 22:39:48 +01:00
Claire
53814b2b31 Fix blocking someone not clearing up list feeds (#16205) 2022-01-28 22:39:48 +01:00
ThibG
2012c5ae50 Fix maintenance script not re-indexing some indexes on textual values (#15515)
* Fix maintenance script not re-indexing some indexes on textual values

Fixes #15475

* Refresh instance view at the end of the maintenance script run

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-01-28 22:39:48 +01:00
Claire
9aa7286c8f Change deduplication order of tootctl maintenance fix-duplicates (#15923)
Hopefully fixes #15922

Also update support up to latest database schema version
2022-01-28 22:39:48 +01:00