Commit graph

502 commits

Author SHA1 Message Date
Jeong Arm
0e8f8a1a1c
Implement tootctl accounts prune (#18397)
* Implement tootctl accounts prune

* Optimise query

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-01-13 22:34:16 +01:00
Claire
745bdb11a0
Add tootctl accounts migrate (#22330)
* Add tootctl accounts replay-migration

Fixes #22281

* Change `tootctl accounts replay-migration` to `tootctl accounts migrate`
2023-01-13 17:00:23 +01:00
Claire
f239d31f23
Add --email and --dry-run options to tootctl accounts delete (#22328) 2022-12-15 14:52:50 +01:00
Evan
78ef635980
Add command to remove avatar and header images of inactive remote accounts from the local database (#22149)
* Add tootctl subcommand media remove-profile-media

* Trigger workflows

* Correcting external linting

* External linting error

* External linting fix

* Merging with remove command

* Linting

* Correct long option names

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Correct long option names

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Correct long option names

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Remove saving a list of purged accounts

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-12-14 19:50:07 +01:00
Claire
55b210b3e5
Fix crash and incorrect behavior in tootctl domains crawl (#19004) 2022-12-13 20:02:32 +01:00
Claire
c52263f6f8
Fix deprecation warning in tootctl accounts rotate (#22120) 2022-12-07 14:13:10 +01:00
Claire
d587a268fd
Add logging for Rails cache timeouts (#21667)
* Reduce redis cache store connect timeout from default 20 seconds to 5 seconds

* Log cache store errors
2022-11-27 20:37:37 +01:00
Eugen Rochko
03b0f3ac83
Bump version to 4.0.2 (#20725) 2022-11-15 03:57:18 +01:00
Eugen Rochko
4415dd6036
Bump version to 4.0.1 (#20696) 2022-11-14 22:21:14 +01:00
Eugen Rochko
fb389bd73c
Bump version to 4.0.0 (#20636) 2022-11-14 20:27:12 +01:00
Eugen Rochko
75299a042c
Bump version to 4.0.0rc4 (#20634) 2022-11-14 08:50:14 +01:00
Claire
457c37e47a
Fix index name in fix-duplicates task (#20632) 2022-11-14 08:33:48 +01:00
Claire
bd806a3090
Update fix-duplicates (#20502)
Fixes #19133
2022-11-13 21:01:38 +01:00
Eugen Rochko
53028af10e
Bump version to 4.0.0rc3 (#20378) 2022-11-11 08:39:38 +01:00
Eugen Rochko
5187e4e758
Bump version to 4.0.0rc2 (#19831) 2022-11-06 06:59:56 +01:00
Eugen Rochko
8ae0936ddd
Bump version to 4.0.0rc1 (#19473) 2022-10-28 00:26:02 +02:00
Eugen Rochko
d7595adbf4
Add --remove-role option to tootctl accounts modify (#19477)
Fix #19152
2022-10-27 14:31:10 +02:00
Jeong Arm
882e54c786
Fix Ambiguous SQL error on tootctl media refresh (#19206) 2022-09-20 23:50:19 +02:00
luzpaz
4aa3b9bd01
Fix typos (#18604)
* Fix typos

Found via `codespell -q 3 -S ./CHANGELOG.md,./AUTHORS.md,./config/locales,./app/javascript/mastodon/locales -L ba,keypair,medias,pixelx,ro`

* Follow-up typo fix
2022-08-28 17:44:34 +02:00
Eugen Rochko
c556c3a0d1
Add admin API for managing canonical e-mail blocks (#19067) 2022-08-28 03:31:54 +02:00
Jeong Arm
e682975afd
Add '--days' option to tootctl media refresh (#18425)
* Add '--days' option to tootctl media refresh

* Fix undefined scope
2022-08-25 04:40:17 +02:00
Brayd
fc46fa8f99
Minimal adjustments to the short description (#18001)
Minimal adjustments have been made to the short description so that it logically follows the long description
2022-08-13 15:41:12 +02:00
Eugen Rochko
44b2ee3485
Add customizable user roles (#18641)
* Add customizable user roles

* Various fixes and improvements

* Add migration for old settings and fix tootctl role management
2022-07-05 02:41:40 +02:00
Eugen Rochko
fbcbf7898f
Bump version to 3.5.3 (#18530) 2022-05-26 23:26:15 +02:00
Eugen Rochko
a9b64b24d6
Change algorithm of tootctl search deploy to improve performance (#18463) 2022-05-22 22:16:43 +02:00
Claire
014065913c
Bump version to 3.5.2 (#18295)
* Bump version to 3.5.2

* Change some entries to be more clear

* Add some extra notes

* Fix line wrap

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
2022-05-04 00:57:42 +02:00
Claire
bc19c083ce
Add ability to set approval-based registration through tootctl (#18248)
Fixes #18235

Add `tootctl settings registrations approved` with
optional `--require-reason` switch.
2022-05-02 17:41:34 +02:00
Eugen Rochko
7b0fe4aef9
Fix opening and closing Redis connections instead of using a pool (#18171)
* Fix opening and closing Redis connections instead of using a pool

* Fix Redis connections not being returned to the pool in CLI commands
2022-04-29 22:43:07 +02:00
Eugen Rochko
3917353645
Fix single Redis connection being used across all threads (#18135)
* Fix single Redis connection being used across all Sidekiq threads

* Fix tests
2022-04-28 17:47:34 +02:00
Eugen Rochko
ed5491e5de
Bump version to 3.5.1 (#18000) 2022-04-08 21:57:24 +02:00
Claire
cb45c04d26
Fix migration error handling (#17991) 2022-04-07 20:46:30 +02:00
Claire
5f0fc639da
Fix error re-running some migrations if they get interrupted at the wrong moment (#17989) 2022-04-07 20:17:49 +02:00
Eugen Rochko
6221b36b27
Remove sign-in token authentication, instead send e-mail about new sign-in (#17970) 2022-04-06 20:58:12 +02:00
Eugen Rochko
8c7223f4ea
Bump version to 3.5.0 (#17911) 2022-03-30 14:52:37 +02:00
Eugen Rochko
d7d049aab7
Bump version to 3.5.0rc3 (#17876) 2022-03-26 04:29:36 +01:00
Eugen Rochko
07f8b4d1b1
Bump version to 3.5.0rc2 (#17855) 2022-03-26 02:54:11 +01:00
Claire
3afd59df0f
Fix tootctl email_domain_blocks add (#17842)
Fixes #17831
2022-03-21 19:10:09 +01:00
Claire
b07906bdb0
Fix wrong language code for Kurdish languages (#17812) 2022-03-17 01:37:03 +01:00
Eugen Rochko
4bdce2c513
Bump version to 3.5.0rc1 (#17618)
* Bump version to 3.5.0rc1

* Various fixes and improvements

* Update AUTHORS.md

* Various fixes and improvements

* Update README.md
2022-03-15 08:16:45 +01:00
Claire
642528f455
Update fix-duplicates maintenance task (#17731)
* Update fix-duplicates task to 2022_02_10_153119

Also add support for Appeal to AccountMerging#merge_with!

* Update fix-duplicates task to 2022_03_07_094650

* Update fix-duplicates task to 2022_03_09_213005

* Update fix-duplicates task to 2022_03_07_083603

* Update fix-duplicates task to 2022_03_10_060626

* Update fix-duplicates script to 2022_03_07_083603

* Update fix-duplicates task to 2022_03_10_060706

* Update fix-duplicates task to 2022_03_10_060959

* Silence CodeClimate
2022-03-12 08:33:11 +01:00
Eugen Rochko
75e33fd08f
Fix null values being included in some indexes (#17711)
* Fix null values being included in some indexes

* Update lib/mastodon/migration_helpers.rb

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

* Add documentation link to corruption error message

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-03-12 08:12:57 +01:00
Rens Groothuijsen
c439e13e12
Enable importing GIF emojis in CLI (#17706) 2022-03-06 23:41:44 +01:00
Josh Soref
b5329e0035
Spelling (#17705)
* spelling: account

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: affiliated

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: appearance

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: autosuggest

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: cacheable

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: component

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: conversations

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: domain.example

Clarify what's distinct and use RFC friendly domain space.

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: environment

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: exceeds

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: functional

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: inefficiency

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: not

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: notifications

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: occurring

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: position

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: progress

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: promotable

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: reblogging

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: repetitive

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: resolve

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: saturated

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: similar

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: strategies

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: success

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: targeting

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: thumbnails

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: unauthorized

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: unsensitizes

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: validations

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: various

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

Co-authored-by: Josh Soref <jsoref@users.noreply.github.com>
2022-03-06 22:51:40 +01:00
Jeong Arm
3e12abc1fe
Calculate max_id without random vector (#17623) 2022-02-23 16:44:59 +01:00
Eugen Rochko
3413f1c44b
Forward-port version bump to 3.4.6 (#17434) 2022-02-03 14:21:38 +01:00
Claire
54581d43e7
Bump version to 3.4.5 (#17402) 2022-01-31 21:27:40 +01:00
Claire
a99adeaad3
Fix edge case in migration helpers that caused crash because of PostgreSQL quirks (#17398) 2022-01-30 22:34:54 +01:00
Claire
8a07ecd377
Remove leftover database columns from Devise::Models::Rememberable (#17191)
* Remove leftover database columns from Devise::Models::Rememberable

* Update fix-duplication maintenance script

* Improve errors/warnings in the fix-duplicates maintenance script
2022-01-23 15:46:30 +01:00
Claire
76761d5fc0
Add ability for admins to delete canonical email blocks (#16644)
* Add admin option to remove canonical email blocks from a deleted account

* Add tootctl canonical_email_blocks to inspect and remove canonical email blocks
2021-12-17 23:02:14 +01:00
Takeshi Umeda
6c8c031bcd
Add remove orphans to tootctl statuses remove (#17067)
* Add remove orphans to tootctl statuses remove

* Add REINDEX and change option from vacuum to compression-database

* Changed to extract the deletion target of conversations to a temporary table

* Support progress bar and exceptions when media remove

* Add continue option

* Fix compression to compress

* Remove skip_remove_orphans
2021-12-12 06:09:14 +01:00
Takeshi Umeda
0ac7efdc50
Fix performance of tootctl statuses remove (#17052)
* Fix performance of tootctl statuses remove

* Fix model class
2021-11-26 22:08:47 +01:00
Takeshi Umeda
06631fdc53
Fix ElasticSearch to Elasticsearch (#17050) 2021-11-26 08:30:02 +01:00
OSAMU SATO
53aca8aecf
Add batch_size option to bin/tootctl search deploy (#17049) 2021-11-26 08:29:53 +01:00
Eugen Rochko
6e50134a42
Add trending links (#16917)
* Add trending links

* Add overriding specific links trendability

* Add link type to preview cards and only trend articles

Change trends review notifications from being sent every 5 minutes to being sent every 2 hours

Change threshold from 5 unique accounts to 15 unique accounts

* Fix tests
2021-11-25 13:07:38 +01:00
Takeshi Umeda
3419d3ec84
Bump chewy from 5.2.0 to 7.2.3 (supports Elasticsearch 7.x) (#16915)
* Bump chewy from 5.2.0 to 7.2.2

* fix style (codeclimate)

* fix style

* fix style

* Bump chewy from 7.2.2 to 7.2.3
2021-11-18 22:02:08 +01:00
Eugen Rochko
2251db42ec
Forward port version bumps to 3.4.2 and 3.4.3 (#16945)
* Bump version to 3.4.2

* Bump version to 3.4.3
2021-11-06 05:32:14 +01:00
Jeong Arm
884c60002e
Skip blocked domains media on tootctl media refresh (#16914) 2021-10-28 19:30:44 +02:00
Jeong Arm
3f5f4273b3
Add optional domain restrict to tootctl accounts cull (#16511)
* Add optional domain restrict to accounts cull

* Use "unless" - codeclimate
2021-10-14 21:09:56 +02:00
Claire
a8ef6d24d8
Fix tootctl accounts cull not excluding domains on timeouts and certificate issues (#16433)
Fixes #16410
2021-10-14 21:08:37 +02:00
Claire
4ac78e2a06
Add feature to automatically delete old toots (#16529)
* Add account statuses cleanup policy model

* Record last inspected toot to delete to speed up successive calls to statuses_to_delete

* Add service to cleanup a given account's statuses within a budget

* Add worker to go through account policies and delete old toots

* Fix last inspected status id logic

All existing statuses older or equal to last inspected status id must be
kept by the current policy. This is an invariant that must be kept so that
resuming deletion from the last inspected status remains sound.

* Add tests

* Refactor scheduler and add tests

* Add user interface

* Add support for discriminating based on boosts/favs

* Add UI support for min_reblogs and min_favs, rework UI

* Address first round of review comments

* Replace Snowflake#id_at_start with with_random parameter

* Add tests

* Add tests for StatusesCleanupController

* Rework settings page

* Adjust load-avoiding mechanisms

* Please CodeClimate
2021-08-09 23:11:50 +02:00
Claire
5a1e072517
Change references to tootsuite/mastodon to mastodon/mastodon (#16491)
* Change references to tootsuite/mastodon to mastodon/mastodon

* Remove obsolete test fixture

* Replace occurrences of tootsuite/mastodon with mastodon/mastodon in CHANGELOG

And a few other places
2021-07-13 15:46:20 +02:00
Eugen Rochko
771c9d4ba8
Add ability to skip sign-in token authentication for specific users (#16427)
Remove "active within last two weeks" exception for sign in token requirement

Change admin reset password to lock access until the password is reset
2021-07-08 05:31:28 +02:00
Claire
2e0eac71dd
Add --by-uri option to tootctl domains purge (#16434)
Fixes #16410
2021-07-07 21:17:00 +02:00
Eugen Rochko
d6486c969f
Bump version to 3.4.1 (#16350) 2021-06-03 04:26:02 +02:00
Claire
11d3c065a5
Fix migration script not being able to run if it fails midway (#16312)
* Fix migration script not being able to run if it fails midway

* Fix old migration script

* Fix old migration script

* Refactor CorruptionError
2021-06-02 19:15:17 +02:00
Claire
526332c545
Fix account deletion sometimes failing because of optimistic locks (#16317)
* Fix account deletion sometimes failing because of optimistic locks

In some rare occasions[1], deleting accounts would fail with a
`StaleObjectError` exception.

Indeed, account deletion manually sets the `AccountStat` values without
handling cases where the optimistic locking on `AccountStat` would fail.

To my knowledge, with the rewrite of account counters in #15913, the
`DeleteAccountService` is now the only place that changes the counters in
a way that is not atomic.

Since in this specific case, we do not care about the previous values of the
account counters, it appears we don't need locking at all for this table
anymore.

[1]: https://discourse.joinmastodon.org/t/account-cant-be-deleted/3602

* Bump MAX_SUPPORTED_VERSION in maintenance script
2021-06-02 17:41:25 +02:00
Claire
abf4c2ab21
Fix tootctl search deploy on Ruby 3 (#16346)
Fixes #16344
2021-06-01 17:31:49 +02:00
Eugen Rochko
4c7efdba40
Bump version to 3.4.0 (#16239) 2021-05-16 23:55:07 +02:00
Claire
76064e6608
Update fix-duplicates maintenance script to support latest migrations (#16231)
* Update maintenance script to support latest database migrations

* Update Account#merge_with!
2021-05-12 23:19:44 +02:00
Eugen Rochko
7bd2b54a46
Bump version to 3.4.0rc2 (#16206) 2021-05-10 22:23:48 +02:00
Takeshi Umeda
9b18914c35
Add a Redis environment variable for sidekiq (#16188) 2021-05-09 10:40:17 +02:00
Eugen Rochko
d1442a06c3
Bump version to 3.4.0rc1 (#16053) 2021-05-08 17:15:06 +02:00
Claire
a5f91a11d0
Fix older migrations on Ruby 3 (#16174) 2021-05-07 15:56:45 +02:00
Takeshi Umeda
25345c90ff
Fix how to change connection pool for rails 6 (#16158)
* Fix how to change connection pool for rails 6

* Fix to match the code elsewhere
2021-05-05 04:41:10 +02:00
Takeshi Umeda
422df9d670
Fix cache redis not being used (#16131) 2021-04-29 15:43:49 +02:00
abcang
ddabbbf5a6
Fix DB connection pool settings in CLI (#15983) 2021-03-31 17:46:17 +02:00
Stanislas
d33351af3c
tootctl emoji import: case insensitive duplicate check (#15738) 2021-03-24 10:55:16 +01:00
Claire
c31c95ffe4
Remove MySQL-specific code from Mastodon::MigrationHelpers (#15924)
Mastodon::MigrationHelpers has been forked from Gitlab a long time ago, but
Mastodon has never supported using a MySQL database.

Removing MySQL support from Mastodon::MigrationHelpers makes it a little easier
to maintain. In particular, it removes code that would need updating with
Rails 6.
2021-03-19 13:14:40 +01:00
Claire
82caed594c
Change deduplication order of tootctl maintenance fix-duplicates (#15923)
Hopefully fixes #15922

Also update support up to latest database schema version
2021-03-19 11:07:56 +01:00
Claire
b358229834
Further preparation for Rails 6 (#15916)
* Use ActiveRecord::Result#to_ary instead of deprecated to_hash

They do the same thing, and to_hash has been removed from Rails 6.1

* Explicitly name polymorphic indexes to workaround a bug in Rails 6.1

cf. https://github.com/rails/rails/issues/41693

* Fix incorrect usage of “foreign_key” in migration script

* Use `ActiveModel::Errors#delete` instead of deprecated clear method

* Fix link headers tests on Rails 6.1

Rails 6.1 adds values to the Link header by default, thus it is not a
LinkHeader object anymore. Fix the test to parse the Link header instead
of assuming it is a LinkHeader.
2021-03-19 02:45:34 +01:00
Claire
43eff898a0
Prepare Mastodon for Rails 6 (#15911)
* Fix misuse of foreign_type

* Fix use of removed "add_template_helper"

* Use response.media_type instead of response.content_type in tests

* Fix CSV export controller test on Rails 6

Rails 6 sets a "filename*" field in the Content-Disposition header to
explicitly encode the filename as UTF-8.

This changes checks the first part of the Content-Disposition header so
it matches in both Rails 5 and Rails 6.

* Fix emoji formatting with Rails 6

* Make emoji output more idiomatic and robust

* Switch from redis-rails gem to built-in Rails redis cache storage
2021-03-17 10:09:55 +01:00
Shubhendra Singh Chauhan
c8d11b8bdb
Fixed code quality issues (#15541)
* Added .deepsource.toml

* Removed bad use of `alias`

* Fixed operand order in the binary expression

* Prefixed unused method arguments with an underscore

* Replaced the old OpenSSL algorithmic constants with the newer strings initializers.

* Removed unnecessary UTF-8 encoding comment
2021-01-31 21:26:09 +01:00
ThibG
13d5b81579
Fix “tootctl accounts unfollow” (#15639)
Fixes #15635

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2021-01-29 18:38:56 +01:00
luigi
eb51e43fb4
Optimize some regex matching (#15528)
* Use Regex#match?

* Replace =~ too

* Avoid to call match? from Nil

* Keep value of Regexp.last_match
2021-01-22 10:09:08 +01:00
ThibG
54d4e5252b
Use Rails' index_by where it makes sense (#15542)
* Use Rails' index_by where it makes sense

* Fix tests

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2021-01-12 09:27:38 +01:00
ThibG
9395143126
Fix maintenance script not re-indexing some indexes on textual values (#15515)
* Fix maintenance script not re-indexing some indexes on textual values

Fixes #15475

* Refresh instance view at the end of the maintenance script run

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2021-01-09 03:51:32 +01:00
Sophie Parker
c7262d12b2
Improve Emoji import (fix #15429) (#15430)
* Improve Emoji import

Skip macOS '._' shadow files in tar archive to speed up import

* Fix codeclimate format issue with whitespace

* Update lib/mastodon/emoji_cli.rb

suggestions from Gargron to improve comment

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>

* Update emoji_cli.rb

Remove extraneous comment (macOS-specific comment now with correct line)

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
2020-12-30 23:19:55 +01:00
Eugen Rochko
633d175146
Bump version to 3.3.0 (#15433) 2020-12-27 06:00:55 +01:00
ThibG
f1f96ebf02
Fix being able to import more than allowed number of follows (#15384)
* Fix being able to import more than allowed number of follows

Without this commit, if someone tries importing a second list of accounts to
follow before the first one has been processed, this will queue imports for
the two whole lists, even if they exceed the account's allowed number of
outgoing follows.

This commit changes it so the individual queued imports aren't exempt from
the follow limit check (they remain exempt from the rate-limiting check
though).

* Catch validation errors to not re-queue failed follows

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-26 23:52:46 +01:00
Eugen Rochko
444b21b55f
Bump version to 3.3.0rc3 (#15411) 2020-12-23 01:36:13 +01:00
ThibG
1cf2c3a810
Fix external user creation failing when invite request text is required (#15405)
* Fix external user creation failing when invite request text is required

Also fixes tootctl-based user creation.

* Add test about invites when invite request text is otherwise required

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-22 17:14:32 +01:00
ThibG
c4e860277d
Fix "tootctl accounts fix-duplicates" (#15373)
- `pluck_each` cannot be used this way with `group`
- typo

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-19 13:34:16 +01:00
Eugen Rochko
c6598b17d9
Bump version to 3.3.0rc2 (#15365) 2020-12-19 00:28:38 +01:00
ThibG
a60d9335d8
Fix resolving accounts sometimes creating duplicate records for a given AP id (#15364)
* Fix ResolveAccountService accepting mismatching acct: URI

* Set attributes that should be updated regardless of suspension

* Fix key fetching

* Automatically merge remote accounts with duplicate `uri`

* Add tests

* Add "tootctl accounts fix-duplicates"

Finds duplicate accounts sharing a same ActivityPub `id`, re-fetch them and
merge them under the canonical `acct:` URI.

Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-18 23:26:26 +01:00
ThibG
75944a2f2d
Bump max supported schema version in maintenance script (#15359)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-18 09:46:59 +01:00
ThibG
8485c436d5
Bump max supported schema version in maintenance script (#15349)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2020-12-15 12:57:24 +01:00
Eugen Rochko
216b85b053
Fix performance on instances list in admin UI (#15282)
- Reduce duplicate queries
- Remove n+1 queries
- Add accounts count to detailed view
- Add separate action log entry for updating existing domain blocks
2020-12-14 09:06:34 +01:00
ThibG
fbe7519243
Fix account merging in maintenance script (#15264)
Also include AccountNote and AccountDeletionRequest to the list of classes
needing to be reassigned the merged account.
2020-12-04 02:23:51 +01:00
ThibG
fbff89be03
Fix tootctl ip_blocks add failing if not given a comment (#15263)
Fixes #15261
2020-12-02 21:19:41 +01:00
Eugen Rochko
8e0b98556f
Bump version to 3.3.0rc1 (#15230) 2020-12-01 06:38:05 +01:00
Eugen Rochko
e7f1439b97
Bump version to 3.2.1 (#15227) 2020-11-27 18:29:22 +01:00
Eugen Rochko
f844386809
Add tootctl accounts merge (#15201)
* Add `tootctl accounts merge`

* Update lib/mastodon/accounts_cli.rb

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>

Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2020-11-23 17:50:16 +01:00
ThibG
1242e57c27
Deal with collation-related index corruption (#14860)
* Add tootctl maintenance fix-duplicates

This tool goes through the database to detect and fix duplicates.
This operation is very slow and may cause data loss (of data that would be
inaccessible without intervention because of the existing index corruptions).
It tries its best to make sensible decisions, and asks the user in some cases.

* Add warning message in db:migrate hook

* Clear Rails cache after being done with database deduplication

Avoids followers hash cache being incorrect, among other things
2020-11-19 17:37:49 +01:00
Eugen Rochko
3134691948
Add support for reversible suspensions through ActivityPub (#14989) 2020-11-08 00:28:39 +01:00
Eugen Rochko
5e1364c448
Add IP-based rules (#14963) 2020-10-12 16:33:49 +02:00
Eugen Rochko
ed099d8bdc
Change account suspensions to be reversible by default (#14726) 2020-09-15 14:37:58 +02:00
Y.Yamashiro
abc91e5fa6
Change S3 ACL in "tootctl media remove-orphans --fix-permissions" from fixed value to environment file's value. (#14715) 2020-09-02 00:17:58 +02:00
Eugen Rochko
71085987e8
Add --fix-permissions option to tootctl media remove-orphans (#14383) 2020-09-01 03:33:21 +02:00
dependabot[bot]
8972e5f7f6
Bump rubocop from 0.86.0 to 0.88.0 (#14412)
* Bump rubocop from 0.86.0 to 0.88.0

Bumps [rubocop](https://github.com/rubocop-hq/rubocop) from 0.86.0 to 0.88.0.
- [Release notes](https://github.com/rubocop-hq/rubocop/releases)
- [Changelog](https://github.com/rubocop-hq/rubocop/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rubocop-hq/rubocop/compare/v0.86.0...v0.88.0)

Signed-off-by: dependabot[bot] <support@github.com>

* Fix for latest RuboCop

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yamagishi Kazutoshi <ykzts@desire.sh>
2020-09-01 03:04:00 +02:00
Takeshi Umeda
175cd4f8ef
Fix an error when file_file_size is nil in tootctl media remove (#14657) 2020-08-24 18:09:46 +02:00
ThibG
8d217d7231
Improve email address validation (#14565)
* Increase DNS timeout from 1 second to 5 seconds for MX check

1 seconds is rather short when using a recursive DNS resolver which
hasn't got a cached result already available. Use 5 seconds instead,
which is the timeout value we use for outgoing HTTP queries.

* Add more precise error messages for invalid e-mail addresses
2020-08-12 12:40:25 +02:00
ThibG
decc5b9a78
Fix tootctl media commands not handling snowflake ids for media_attachments (#14536) 2020-08-10 01:51:06 +02:00
Eugen Rochko
93dd413a47
Bump version to 3.2.0 (#14395) 2020-07-27 00:07:26 +02:00
Eugen Rochko
d1a6f8d5a8
Bump version to 3.2.0rc2 (#14382) 2020-07-24 18:47:25 +02:00
Eugen Rochko
6f798eb574
Bump version to 3.2.0rc1 (#14312) 2020-07-15 21:07:27 +02:00
Eugen Rochko
4abe3be321
Change tootctl search deploy algorithm (#14300) 2020-07-14 18:10:35 +02:00
Eugen Rochko
96e89d1ef4
Bump version to 3.1.5 (#14284) 2020-07-10 21:07:30 +02:00
Eugen Rochko
64aac30733
Add customizable thumbnails for audio and video attachments (#14145)
- Change audio files to not be stripped of metadata
- Automatically extract cover art from audio if it exists
- Add `thumbnail` parameter to `POST /api/v1/media`, `POST /api/v2/media` and `PUT /api/v1/media/:id`
- Add `icon` to represent it in attachments in ActivityPub
- Fix `preview_url` containing URL of missing missing image when there is no thumbnail instead of null
- Fix duration of audio not being displayed on public pages until the file is loaded
2020-06-29 13:56:55 +02:00
Eugen Rochko
8c04e37b03
Remove the terms blacklist and whitelist from UX (#14149)
Localization strings:

- "Whitelist mode" -> "Limited federation mode"
- "Blacklist e-mail domain" -> "Block e-mail domain"
- "Whitelist domain" -> "Allow domain for federation"

...And so on

Environment variables (backwards-compatible):

- `WHITELIST_MODE` -> `LIMITED_FEDERATION_MODE`
- `EMAIL_DOMAIN_BLACKLIST` -> `EMAIL_DOMAIN_DENYLIST`
- `EMAIL_DOMAIN_WHITELIST` -> `EMAIL_DOMAIN_ALLOWLIST`

tootctl:

- `tootctl domains purge --whitelist-mode` -> `tootctl domains purge --limited-federation-mode`

Removed badly maintained and no longer relevant .env.production.sample file
2020-06-27 20:20:11 +02:00
Eugen Rochko
4662afe075
Fix help text around tootctl email_domain_blocks (#14147) 2020-06-26 21:28:40 +02:00
tateisu
370e38ee15
Add tootctl email-domain-blocks (#13589)
* Add tootctl email_domains (block|unblock)

* fix codeclimate issues.

* fix codeclimate issues.

* fix codeclimate issues.

* add list subcommand, remove log_action.

* fix codeclimate issues.

* filter duplicate hostnames,ips before block

* rebase from currnet master branch.
rename email_domains_cli.rb to email_domain_blocks_cli.rb .
rename Mastodon::EmailDomainsCLI to Mastodon::EmailDomainBlocksCLI .
rename command email_domains to email-domain-blocks . (Thor recognizes both of - and _ )
rename subcommand block to add .
rename subcommand unblock to remove .
change the color in list subcommand to while for domain or cyan for childlen.
don't use include() in list subcommand.
suppress console output about succeeded entry.
add console output about count of processed/skipped.
remove capitalization in subcommand description.
remove long_desc in subcommand 'remove'.
remove duplicate where in subcommand 'remove'.

* fix codeclimate issue.
2020-06-25 12:17:10 +02:00
Stanislas
ce87469d0a
Fix tootctl upgrade storage-schema S3 ACL (#13768) 2020-05-17 17:27:36 +02:00
Eugen Rochko
199bbbcb9f
Fix tootctl media remove-orphans choking on unknown files in storage (#13765)
Fix #13762

Catch tootctl interrupt to prevent confusing stacktrace
2020-05-15 18:41:27 +02:00
Eugen Rochko
2b91a3dac0
Fix tootctl upgrade storage-schema misbehaving (#13761)
- Fix not moving original files of custom emojis
- Fix command failing to move any files with S3 storage
- Fix command marking records as upgraded when move failed

Fix #13594
2020-05-15 17:15:24 +02:00
Eugen Rochko
06225dee94
Bump version to 3.1.4 (#13750) 2020-05-14 07:46:11 +02:00
Takeshi Umeda
cbfb2c0e71
Fix tootctl media refresh (#13751) 2020-05-14 07:45:52 +02:00
ThibG
34756cc4e0
Fix "tootctl media remove-orphans" crashing on “Import” files (#13685)
* Fix "tootctl media remove-orphans" crashing on “Import” files

* Also remove empty directories when removing orphaned media
2020-05-09 21:06:55 +02:00
Hanage999
ff72c0472f
Fix tootctl upgrade storage-schema failing to delete empty directories (#13593) 2020-05-04 13:51:34 +02:00
Lerk
157850577f
Add tootctl emoji export (#13534)
* add emoji export command to cli

* fix codeclimate issues

* add error when no matching category was found

* add other suggestions

* exit 1 when no matching category is found

* changes according to suggestions

* 👀

* RubyNein

Y u always autoformat :c
2020-04-27 22:17:49 +02:00
Eugen Rochko
c3ca3801f2
Add separate cache directory for non-local uploads (#12821) 2020-04-26 23:29:08 +02:00
Eugen Rochko
c99bc5d51b
Fix regression in tootctl media remove-orphans (#13405)
Fix #13401
2020-04-06 14:11:47 +02:00
Eugen Rochko
a889756dd5
Bump version to 3.1.3 (#13389) 2020-04-05 06:23:46 +02:00
Eugen Rochko
a9a063c0e9
Fix tootctl media remove-orphans ignoring PAPERCLIP_ROOT_PATH (#13375)
Fix #13371
2020-04-02 05:28:51 +02:00
Eugen Rochko
0c8945e5ff
Change tootctl media remove-orphans to work for all classes (#13316)
Change `tootctl media lookup` to not use an interactive prompt
2020-03-26 01:56:41 +01:00
tateisu
4e524218d3
Add --skip-media-remove option to tootctl statuses remove (#13080)
* Add skip_media_remove option to tootctl statuses remove

* Add skip_media_remove option to tootctl statuses remove

Co-authored-by: tateisu <tateisu@juggler.jp>
2020-03-08 16:01:07 +01:00
ThibG
ce17cea221
Fix installation failing when Redis password contains special characters (#13156)
* Add support for special characters in Redis passwords

Fixes #13154

* Refactor
2020-02-29 03:00:43 +01:00
Eugen Rochko
aa8568b7f2
Bump version to 3.1.2 (#13162) 2020-02-27 12:34:07 +01:00
ThibG
ca7ee9f1cf
Add --reset-password option to tootctl accounts modify (#13126) 2020-02-22 01:29:14 +01:00
Marcin Cieślak
bd4bbba98a
Do not suggest to login into the GitLab database (#13084)
As pointed out on Discourse:

https://discourse.joinmastodon.org/t/obscure-wtf-error-message-running-migrations-for-3-1/2524

The message the administrators were getting was telling them
to log in to the "GitLab database" and also mentions MySQL
2020-02-15 12:52:57 +01:00
Eugen Rochko
b752666e3f
Bump version to 3.1.1 (#13069) 2020-02-10 02:55:40 +01:00
Eugen Rochko
d2bcef7f6f Bump version to 3.1.0 (#13053) 2020-02-09 00:02:08 +01:00
Eugen Rochko
60e160a02d
Bump version to 3.1.0rc2 (#12999) 2020-01-28 20:56:32 +01:00
Eugen Rochko
4bae4e972d
Bump version to 3.1.0rc1 (#12932) 2020-01-24 01:14:51 +01:00
Gomasy
3b08535463 Keep statuses bookmarked by local users in tootctl statuses remove (#12818) 2020-01-11 06:41:37 +01:00
ThibG
817d4a9372 Add --remote-only option to emoji purge (#12810)
Fixes #12804
2020-01-10 00:10:17 +01:00
ThibG
7fe45e7c03 Changes to tootctl statuses remove (#11267)
* Fix comments in statuses_cli

The comment for the part of query keeping statuses which have replies was
incorrect, explaining the part of the query kept favourited statuses instead

* Keep statuses favourited by local users in tootctl statuses remove

* Do delete old replies that are not direct interactions with local users

* Skip reblogs of local statuses instead of all reblogs

* Optimize some queries

* Keep old statuses with recent reblogs

* Add option to clear toots from followed accounts too
2019-12-17 13:33:36 +01:00
trwnh
1c10283217 Fix typo in tootctl statuses remove (#12603) 2019-12-12 19:50:08 +01:00
Eugen Rochko
d7bcc0930c
Fix error handling in tootctl media remove-orphans (#12571) 2019-12-09 04:26:00 +01:00
Eugen Rochko
f3d232381d
Add tootctl media remove-orphans (#12568) 2019-12-08 15:37:12 +01:00
Nathaniel Suchy
23ed9303b8 Add tootctl media lookup command (#12283)
* Add a lookup tool to the media cli

* Improved lookup logic

* Clarified wording in the output

* Code style changes

* Code style changes

* Code style changes

* Code style changes

* Add error handling code incase an attachment isn't found

* Code style changes

* Code style changes

* Make requested changes

* Fix styling issues

* Handle other media types

* Remove an inadvertently added log

* Make requested changes

* Make the code safe no matter what the path, S3 or not

* Code style changes

* Code style changes

* Replace select method with Ruby Enumerable grep method
2019-11-04 12:55:20 +01:00
Eugen Rochko
e6d111f38b
Bump version to 3.0.1 (#12116) 2019-10-09 07:36:57 +02:00
Eugen Rochko
4a98e77d0e
Change tootctl media refresh to skip already downloaded attachments (#12118) 2019-10-08 05:59:10 +02:00
Eugen Rochko
0aaa3afc2d
Add tootctl media usage command (#12115) 2019-10-07 20:04:56 +02:00
Eugen Rochko
6c9b4f6b77
Fix tootctl not allocating enough database connections for main thread (#12097) 2019-10-07 05:05:02 +02:00
Eugen Rochko
38b6c34e32
Fix issues with tootctl's parallelization and progress reporting (#12093) 2019-10-07 04:24:05 +02:00
Jeong Arm
c4fbfaf0b8 Remove unused option from tootctl accounts cull (#12074) 2019-10-06 04:30:07 +02:00
Eugen Rochko
83d3e7733d
Bump version to 3.0.0 (#12000) 2019-10-03 22:44:22 +02:00
Eugen Rochko
bae268b2f6
Bump version to 3.0.0rc3 (#12063) 2019-10-03 04:13:22 +02:00
Takeshi Umeda
0ce0baa9b5 Add parallelization to tootctl search deploy (#12051)
* Add parallel gem

* Modify parallel option in tootctl search deploy

* Add paralell option to tootctl search deploy

* Change 1 to false

* Clean up

* Rename --parallel to --processes
2019-10-02 21:50:43 +02:00
Eugen Rochko
c7adf80827
Change tootctl domains purge to accept multiple domains at once (#12046) 2019-10-02 04:49:53 +02:00
Eugen Rochko
13b06d4b3b
Bump version to 3.0.0rc2 (#11999) 2019-09-29 18:50:16 +02:00
Eugen Rochko
e87bcaa107
Bump version to 3.0.0rc1 (#11900) 2019-09-21 20:02:14 +02:00
Eugen Rochko
3c8372fa81
Bump version to 2.9.3 (#11899) 2019-09-20 22:59:29 +02:00
Eugen Rochko
c5d37f18cb
Change deletes to preserve soft-deleted statuses in unresolved reports (#11805)
Change all account actions except "none" to resolve all unresolved reports

Refactor `SuspendAccountService` to be more readable
2019-09-11 16:32:44 +02:00
Eugen Rochko
031ca25014
Add retry for failed media downloads and tootctl media refresh (#11775) 2019-09-10 15:29:12 +02:00
Eugen Rochko
8674814825
Change tootctl to use inline parallelization instead of Sidekiq (#11776)
- Remove --background option
- Add --concurrency(=5) option
- Add progress bars
2019-09-10 13:48:48 +02:00
Eugen Rochko
96702e7f67
Add tootctl cache recount command (#11597) 2019-08-18 14:55:03 +02:00
Stanislas
3a77090d01 Limit "tootctl accounts follow" to local accounts (#11592)
To (somewhat) limit mass remote follow. Fix #11360
2019-08-17 22:59:40 +02:00
Darius Kazemi
f96f45ef12 Add option to exclude suspended domains/subdomains from tootctl domains crawl (#11454)
* Add "--exclude-suspended" to tootctl domains crawl

This new option ignores any instances suspended server-wide as
well as their associated subdomains. This queries all domain
blocks up front, then runs a regexp on each domain. This improves
performance over what may be the obvious implementation, which is
to ask `DomainBlocks.blocked?(domain)` for each domain -- this
hits the DB many times, slowing things down considerably.

* cleaning up code style

* Compiling regex

* Removing ternary operator
2019-08-03 19:11:09 +02:00
Eugen Rochko
24552b5160
Add whitelist mode (#11291) 2019-07-30 11:10:46 +02:00
mayaeh
0d80f686d8 Add tootctl preview_cards remove (#11320)
* Add `tootctl preview_cards remove`

* fix code style

* Remove `Scheduler::PreviewCardsCleanupScheduler` file

* fix code style again
Add exclude case where image_file_name is blank

* Added a function to output confirmation if the specified number of days is less than 2 weeks
2019-07-28 13:48:19 +02:00
Clar Fon
a6b4440115 Remove pre from version, add extra suffix variable (#11407) 2019-07-26 07:57:27 +02:00
Eugen Rochko
e64e6a03dd
Add categories for custom emojis (#11196)
Fix #7940
2019-06-28 15:54:10 +02:00
Eugen Rochko
b5c772c3d4
Bump version to 2.9.2 (#11152) 2019-06-22 17:28:26 +02:00
Eugen Rochko
84f945d64c
Bump version to 2.9.1 (#11143) 2019-06-22 01:51:27 +02:00
Eugen Rochko
c9eeb2e832
Bump version to 2.9.0 (#11074) 2019-06-13 20:19:21 +02:00
Eugen Rochko
275f09ccab
Bump version to 2.9.0rc2 (#11070) 2019-06-13 00:43:59 +02:00
Eugen Rochko
0949c43ab3
Bump version to 2.9.0rc1 (#11004) 2019-06-09 15:53:08 +02:00
Darius Kazemi
4431ce52a6 Specify gzip required in tootctl emoji help (#11000) 2019-06-08 12:43:11 -04:00
Eugen Rochko
5cdb4c483f
Forward port version bumps to 2.8.3 and 2.8.4 (#10819)
* Bump version to 2.8.3

* Bump version to 2.8.4
2019-05-24 15:57:31 +02:00
ThibG
14f6ce2885 Record account suspend/silence time and keep track of domain blocks (#10660)
* Record account suspend/silence time and keep track of domain blocks

* Also unblock users who were suspended/silenced before dates were recorded

* Add tests

* Keep track of suspending date for users suspended through the CLI

* Show accurate number of accounts that would be affected by unsuspending an instance

* Change migration to set silenced_at and suspended_at

* Revert "Also unblock users who were suspended/silenced before dates were recorded"

This reverts commit a015c65d2d1e28c7b7cfab8b3f8cd5fb48b8b71c.

* Switch from using suspended and silenced to suspended_at and silenced_at

* Add post-deployment migration script to remove `suspended` and `silenced` columns

* Use Account#silence! and Account#suspend! instead of updating the underlying property

* Add silenced_at and suspended_at migration to post-migration

* Change account fabricator to translate suspended and silenced attributes

* Minor fixes

* Make unblocking domains always retroactive
2019-05-14 19:05:02 +02:00
Jeong Arm
03801fcaa5 Remove custom emojis on "tootctl domains purge" (#10721)
* Remove custom emojis on domains purge

* Change message "Removing" to "Removed"
2019-05-09 22:03:20 +02:00
Eugen Rochko
2508370f44
Bump version to 2.8.2 (#10703) 2019-05-05 17:14:15 +02:00
Ushitora Anqou
56880fa76a Add SOURCE_TAG to show source repository's tag (#10698) 2019-05-04 17:39:17 +02:00
Eugen Rochko
8025a41a1f
Add tootctl cache clear (#10689) 2019-05-04 01:02:57 +02:00
Eugen Rochko
5f9f610a23
Bump version to 2.8.1 (#10687) 2019-05-04 00:31:06 +02:00
Eugen Rochko
d77ee3f276
Fix accounts created through tootctl not being always pre-approved (#10684)
Add `--approve` option to `tootctl accounts modify`
2019-05-03 20:49:27 +02:00
Sho Kusano
3e6c7f3617 Configrationable repository url (#10600)
* config: Add GITHUB_REPOSITORY for repository name

* config: Add SOURCE_BASE_URL for repository url

* Show source_url and repository name on getting started
2019-04-21 04:41:34 +02:00
Eugen Rochko
6afab2587d
Bump version to 2.8.0 (#10550) 2019-04-10 23:32:08 +02:00
Eugen Rochko
830c2a9ccb
Bump version to 2.8.0rc3 (#10535) 2019-04-10 04:25:09 +02:00
Eugen Rochko
c03f926cf3
Bump version to 2.8.0rc2 (#10501) 2019-04-09 05:04:11 +02:00
Takeshi Umeda
08ba69b538 Add tootctl accounts reset-relationships (#10483)
* Add `tootctl accounts reset`

* Rename reset to reset-relationships

* Improve command description
2019-04-08 07:46:55 +02:00
Eugen Rochko
20d301c383
Fix missing long description on tootctl statuses remove (#10482) 2019-04-06 04:47:26 +02:00
Eugen Rochko
6689e572f3
Add tootctl emoji purge (#10481)
Fix #10441
2019-04-06 04:47:16 +02:00
Eugen Rochko
e007c7a99b
Add tootctl accounts approve (#10480) 2019-04-06 04:47:05 +02:00
Ben Lubar
e6814a332c Fix tootctl accounts cull (#10460)
* List the actual accounts that would have been culled during a dry run.

Otherwise, the dry run mode is basically useless.

* Prevent unreachable domains from inheriting the previous status code.

* Update CHANGELOG.md for #10460.
2019-04-04 16:46:27 +02:00
Jeong Arm
67eb47e25f Simplify culling and fix race condition (#10440) 2019-04-01 19:17:21 +02:00
Eugen Rochko
ac650d7a13
Bump version to 2.8.0rc1 (#10340)
Update AUTHORS.md
Update CHANGELOG.md
2019-03-30 02:13:42 +01:00
Eugen Rochko
5d41532d94
Add tootctl search deploy to avoid ugly rake task syntax (#10403) 2019-03-28 02:16:11 +01:00
Eugen Rochko
51e154f5e8
Admission-based registrations mode (#10250)
Fix #6856
Fix #6951
2019-03-14 05:28:30 +01:00