Commit graph

456 commits

Author SHA1 Message Date
Mike Barnes aebfb3c148 Bump version, -cw1 borken 2021-03-06 20:19:34 +11:00
Mike Barnes 7cf79a6d90 Version 3.3.0-cw1 2021-03-02 13:57:35 +11:00
Mike Barnes 14917cdb73 Chinwag 3.3.0 merge 2021-03-02 13:55:46 +11:00
Eugen Rochko 633d175146
Bump version to 3.3.0 (#15433) 2020-12-27 06:00:55 +01:00
ThibG f1f96ebf02
Fix being able to import more than allowed number of follows (#15384)
* Fix being able to import more than allowed number of follows

Without this commit, if someone tries importing a second list of accounts to
follow before the first one has been processed, this will queue imports for
the two whole lists, even if they exceed the account's allowed number of
outgoing follows.

This commit changes it so the individual queued imports aren't exempt from
the follow limit check (they remain exempt from the rate-limiting check
though).

* Catch validation errors to not re-queue failed follows

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-26 23:52:46 +01:00
Eugen Rochko 444b21b55f
Bump version to 3.3.0rc3 (#15411) 2020-12-23 01:36:13 +01:00
ThibG 1cf2c3a810
Fix external user creation failing when invite request text is required (#15405)
* Fix external user creation failing when invite request text is required

Also fixes tootctl-based user creation.

* Add test about invites when invite request text is otherwise required

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-22 17:14:32 +01:00
Eugen Rochko 9915d11c0d
Fix unnecessary queries when batch-removing statuses, 100x faster (#15387) 2020-12-22 17:13:55 +01:00
Mike Barnes 2ce2ba7438 Merge tag 'v3.2.2' 2020-12-22 17:40:29 +11:00
ThibG c4e860277d
Fix "tootctl accounts fix-duplicates" (#15373)
- `pluck_each` cannot be used this way with `group`
- typo

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-19 13:34:16 +01:00
Eugen Rochko 2d5a5bac67 Bump version to 3.2.2 2020-12-19 00:52:04 +01:00
Eugen Rochko c6598b17d9
Bump version to 3.3.0rc2 (#15365) 2020-12-19 00:28:38 +01:00
ThibG 406adfca27
Backport fixes to 3.2 (#15360)
* Fix 2FA/sign-in token sessions being valid after password change (#14802)

If someone tries logging in to an account and is prompted for a 2FA
code or sign-in token, even if the account's password or e-mail is
updated in the meantime, the session will show the prompt and allow
the login process to complete with a valid 2FA code or sign-in token

* Fix Move handler not being triggered when failing to fetch target (#15107)

When failing to fetch the target account, the ProcessingWorker fails
as expected, but since it hasn't cleared the `move_in_progress` flag,
the next attempt at processing skips the `Move` activity altogether.

This commit changes it to clear the flag when encountering any
unexpected error on fetching the target account. This is likely to
occur because, of, e.g., a timeout, when many instances query the
same actor at the same time.

* Fix slow distinct queries where grouped queries are faster (#15287)

About 2x speed-up on inboxes query

* Fix possible inconsistencies in tag search (#14906)

Do not downcase the queried tag before passing it to postgres when searching:
- tags are not downcased on creation
- `arel_table[:name].lower.matches(pattern)` generates an ILIKE anyway
- if Postgres and Rails happen to use different case-folding rules,
  downcasing before query but not before insertion may mean that some
  tags with some casings are not searchable

* Fix updating account counters when account_stat is not yet created (#15108)

* Fix account processing failing because of large collections (#15027)

Fixes #15025

* Fix downloading remote media files when server returns empty filename (#14867)

Fixes #14817

* Fix webfinger redirect handling in ResolveAccountService (#15187)

* Fix webfinger redirect handling in ResolveAccountService

ResolveAccountService#process_webfinger! handled a one-step webfinger
redirection, but only accepting the result if it matched the exact URI passed
as input, defeating the point of a redirection check.

Instead, use the same logic as in `ActivityPub::FetchRemoteAccountService`,
updating the resulting `acct:` URI with the result of the first webfinger
query.

* Add tests

* Remove dependency on unused and unmaintained http_parser.rb gem (#14574)

It seems that years ago, the “http” gem dependend on the “http_parser.rb” gem
(it now depends on the “http-parser” gem), and, still years ago, we pulled
it from git in order to benefit from a bugfix that wasn't released yet (#7467).

* Add tootctl maintenance fix-duplicates (#14860, #15201, #15264, #15349, #15359)

* Fix old migration script not being able to run if it fails midway (#15361)

* Fix old migration script not being able to run if it fails midway

Improve the robustness of a migration script likely to fail because of database
corruption so it can run again once database corruptions are fixed.

* Display a specific error message in case of index corruption

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-18 23:31:14 +01:00
ThibG a60d9335d8
Fix resolving accounts sometimes creating duplicate records for a given AP id (#15364)
* Fix ResolveAccountService accepting mismatching acct: URI

* Set attributes that should be updated regardless of suspension

* Fix key fetching

* Automatically merge remote accounts with duplicate `uri`

* Add tests

* Add "tootctl accounts fix-duplicates"

Finds duplicate accounts sharing a same ActivityPub `id`, re-fetch them and
merge them under the canonical `acct:` URI.

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-18 23:26:26 +01:00
ThibG 75944a2f2d
Bump max supported schema version in maintenance script (#15359)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-18 09:46:59 +01:00
ThibG 8485c436d5
Bump max supported schema version in maintenance script (#15349)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-15 12:57:24 +01:00
Eugen Rochko 1045549f85
Add stoplight for object storage failures, return HTTP 503 (#13043) 2020-12-15 12:55:29 +01:00
Eugen Rochko 216b85b053
Fix performance on instances list in admin UI (#15282)
- Reduce duplicate queries
- Remove n+1 queries
- Add accounts count to detailed view
- Add separate action log entry for updating existing domain blocks
2020-12-14 09:06:34 +01:00
ThibG fbe7519243
Fix account merging in maintenance script (#15264)
Also include AccountNote and AccountDeletionRequest to the list of classes
needing to be reassigned the merged account.
2020-12-04 02:23:51 +01:00
ThibG fbff89be03
Fix tootctl ip_blocks add failing if not given a comment (#15263)
Fixes #15261
2020-12-02 21:19:41 +01:00
Eugen Rochko 8e0b98556f
Bump version to 3.3.0rc1 (#15230) 2020-12-01 06:38:05 +01:00
Eugen Rochko e7f1439b97
Bump version to 3.2.1 (#15227) 2020-11-27 18:29:22 +01:00
Eugen Rochko ec0edf74d6
Fix post-migration warning about index corruption being too scary (#15223) 2020-11-27 15:41:43 +01:00
Eugen Rochko f844386809
Add tootctl accounts merge (#15201)
* Add `tootctl accounts merge`

* Update lib/mastodon/accounts_cli.rb

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2020-11-23 17:50:16 +01:00
ThibG 1242e57c27
Deal with collation-related index corruption (#14860)
* Add tootctl maintenance fix-duplicates

This tool goes through the database to detect and fix duplicates.
This operation is very slow and may cause data loss (of data that would be
inaccessible without intervention because of the existing index corruptions).
It tries its best to make sensible decisions, and asks the user in some cases.

* Add warning message in db:migrate hook

* Clear Rails cache after being done with database deduplication

Avoids followers hash cache being incorrect, among other things
2020-11-19 17:37:49 +01:00
Mike Barnes 0c115ff63f Merge tag 'v3.2.1' 2020-11-15 22:32:36 +11:00
Eugen Rochko 3134691948
Add support for reversible suspensions through ActivityPub (#14989) 2020-11-08 00:28:39 +01:00
Eugen Rochko 9b1f2a4b61
Add subresource integrity for JS and CSS assets (#15096)
Fix #2744
2020-11-06 11:56:31 +01:00
Eugen Rochko a583e54023 Bump version to 3.2.1 2020-10-19 16:07:06 +02:00
ThibG 8b448aecef Fix tootctl media commands not handling snowflake ids for media_attachments (#14536) 2020-10-19 15:40:34 +02:00
ThibG 3b699f1732 Fix thumbnail color extraction (#14464)
* Fix contrast calculation for thumbnail color extraction

Luminance calculation was using 0-255 RGB values instead of 0-1 sRGB values,
leading to incorrectly-computed contrast values.

Since we use ColorDiff already, just use its XYZ colorspace conversion code
to get the value.

* Require at least 3:1 contrast for both accent and foreground colors

* Lower required contrast for the accent color
2020-10-19 15:40:14 +02:00
ThibG 399c5f0900 Change content-type to be always computed from file data (#14452)
* Change content-type to be always computed from file data

Restore previous behavior, detecting the content-type isn't very
expensive, and some instances may serve files as application/octet-stream
regardless of their true type, making fetching media from them fail, while
it used to work pre-3.2.0.

* Add test
2020-10-19 15:39:28 +02:00
Takeshi Umeda 58c59af573 Fix an error when file_file_size is nil in tootctl media remove (#14657) 2020-10-19 14:49:31 +02:00
Eugen Rochko 5e1364c448
Add IP-based rules (#14963) 2020-10-12 16:33:49 +02:00
ThibG 5bbc9a4f78
Fix downloading remote media files when server returns empty filename (#14867)
Fixes #14817
2020-09-27 22:00:43 +02:00
Eugen Rochko ed099d8bdc
Change account suspensions to be reversible by default (#14726) 2020-09-15 14:37:58 +02:00
Eugen Rochko e6b272e5c9
Change REST API to return empty data for suspended accounts (#14765) 2020-09-11 15:16:29 +02:00
ThibG 91eecd1b3c
Add border around 🕺 emoji (#14769)
Fixes #14768
2020-09-10 19:08:03 +02:00
Y.Yamashiro abc91e5fa6
Change S3 ACL in "tootctl media remove-orphans --fix-permissions" from fixed value to environment file's value. (#14715) 2020-09-02 00:17:58 +02:00
Eugen Rochko 71085987e8
Add --fix-permissions option to tootctl media remove-orphans (#14383) 2020-09-01 03:33:21 +02:00
dependabot[bot] 8972e5f7f6
Bump rubocop from 0.86.0 to 0.88.0 (#14412)
* Bump rubocop from 0.86.0 to 0.88.0

Bumps [rubocop](https://github.com/rubocop-hq/rubocop) from 0.86.0 to 0.88.0.
- [Release notes](https://github.com/rubocop-hq/rubocop/releases)
- [Changelog](https://github.com/rubocop-hq/rubocop/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rubocop-hq/rubocop/compare/v0.86.0...v0.88.0)

Signed-off-by: dependabot[bot] <support@github.com>

* Fix for latest RuboCop

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2020-09-01 03:04:00 +02:00
ThibG f0b6ddd979
Add border around 🐞 emoji (#14712) 2020-08-31 23:30:27 +02:00
Takeshi Umeda 175cd4f8ef
Fix an error when file_file_size is nil in tootctl media remove (#14657) 2020-08-24 18:09:46 +02:00
ThibG 8d217d7231
Improve email address validation (#14565)
* Increase DNS timeout from 1 second to 5 seconds for MX check

1 seconds is rather short when using a recursive DNS resolver which
hasn't got a cached result already available. Use 5 seconds instead,
which is the timeout value we use for outgoing HTTP queries.

* Add more precise error messages for invalid e-mail addresses
2020-08-12 12:40:25 +02:00
ThibG decc5b9a78
Fix tootctl media commands not handling snowflake ids for media_attachments (#14536) 2020-08-10 01:51:06 +02:00
ThibG 0ebc8d8626
Fix thumbnail color extraction (#14464)
* Fix contrast calculation for thumbnail color extraction

Luminance calculation was using 0-255 RGB values instead of 0-1 sRGB values,
leading to incorrectly-computed contrast values.

Since we use ColorDiff already, just use its XYZ colorspace conversion code
to get the value.

* Require at least 3:1 contrast for both accent and foreground colors

* Lower required contrast for the accent color
2020-08-02 18:47:44 +02:00
ThibG a1412491b7
Change content-type to be always computed from file data (#14452)
* Change content-type to be always computed from file data

Restore previous behavior, detecting the content-type isn't very
expensive, and some instances may serve files as application/octet-stream
regardless of their true type, making fetching media from them fail, while
it used to work pre-3.2.0.

* Add test
2020-08-02 11:21:10 +02:00
Eugen Rochko 93dd413a47
Bump version to 3.2.0 (#14395) 2020-07-27 00:07:26 +02:00
Eugen Rochko d1a6f8d5a8
Bump version to 3.2.0rc2 (#14382) 2020-07-24 18:47:25 +02:00
Takeshi Umeda 101485a41f
Fix mimetype returning nil (#14356) 2020-07-19 13:09:47 +02:00