Commit graph

849 commits

Author SHA1 Message Date
Claire
610731b03d
Merge pull request from GHSA-55j9-c3mp-6fcq 2023-07-06 15:06:49 +02:00
Claire
c5929798bf
Merge pull request from GHSA-9pxv-6qvf-pjwc
* Fix timeout handling of outbound HTTP requests

* Use CLOCK_MONOTONIC instead of Time.now
2023-07-06 15:06:23 +02:00
Claire
eb2417ce99
Fix OAuth apps page crashing when listing apps with certain admin API scopes (#25713) 2023-07-04 18:58:23 +02:00
Eugen Rochko
ba06a2f104
Revert "Rails 7 update" (#25667) 2023-07-02 11:14:22 +02:00
Matt Jankowski
50c2a03695
Rails 7 update (#24241) 2023-07-02 10:38:53 +02:00
Claire
0ccf6c0eb7
Fix batch attachment deletion leaving empty directories (#25587) 2023-06-27 09:36:11 +02:00
S.H
7a25af64dd
Remove media attachment only when file was exist (#25586) 2023-06-26 16:38:19 +02:00
Eugen Rochko
bb4756c823
Change files to be deleted in batches instead of one-by-one (#23302) 2023-06-26 14:17:41 +02:00
Claire
c78280a8ce
Add translate="no" to outgoing mentions and links (#25524) 2023-06-20 18:10:19 +02:00
Matt Jankowski
e1c9d52e91
Reduce sleep time in request pool spec (#25470) 2023-06-19 13:48:25 +02:00
Nick Schonning
c66250abf1
Autofix Rubocop Regex Style rules (#23690)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-06-06 14:50:51 +02:00
Eugen Rochko
4eda233e09
Add webhook templating (#23289)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-06-06 10:42:47 +02:00
Matt Jankowski
af135bddd0
Update Admin::Metrics::Measure classes for Rails 7 (#25236) 2023-06-05 20:46:04 +02:00
Matt Jankowski
3b21c13dcc
Rails 7 compatibility fix for Admin::Metrics::Dimension classes (#25277) 2023-06-05 16:52:33 +02:00
Darius Kazemi
bacb674921
Add exclusive lists (#22048)
Co-authored-by: Liam Cooke <liam@liamcooke.com>
Co-authored-by: John Holdun <john@johnholdun.com>
Co-authored-by: Effy Elden <effy@effy.space>
Co-authored-by: Lina Reyne <git@lina.pizza>
Co-authored-by: Lina <20880695+necropolina@users.noreply.github.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-06-05 09:37:02 +02:00
Matt Jankowski
f84037ae2b
Remove unused #hash_to_object method in SettingsHelper (#25213) 2023-06-01 09:47:05 +02:00
Claire
e9385e93e9
Add a confirmation screen when suspending a domain (#25144) 2023-06-01 09:37:38 +02:00
Christian Schmidt
69057467cb
Translate CW, poll options and media descriptions (#24175)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-06-01 00:10:21 +02:00
Matt Jankowski
00c222377d
Remove unused Settings::Extend module (#25214) 2023-05-31 23:34:48 +02:00
Matt Jankowski
9f5deb310b
Fix Performance/MapCompact cop (#24797)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-05-23 10:49:12 +02:00
Matt Jankowski
0664704cd9
Fix Performance/StartWith cop (#24818) 2023-05-23 10:16:50 +02:00
Claire
e13d2edd47
Fix “Authorized applications” inefficiently and incorrectly getting last use date (#25060) 2023-05-22 14:03:38 +02:00
Emelia Smith
19f9098551
Allow reports with long comments from remote instances, but truncate (#25028) 2023-05-22 13:15:21 +02:00
Claire
7bb8030cc1
Change OpenGraph-based embeds to allow fullscreen (#25058) 2023-05-22 12:25:56 +02:00
Claire
9015c2d646
Change profile updates to be sent to recently-mentioned servers (#24852) 2023-05-12 13:13:04 +02:00
Claire
6b0942d107
Change AccessTokensVacuum to also delete expired tokens (#24868) 2023-05-11 04:40:03 +02:00
Nick Schonning
ff3b7069cf
Autofix Rubocop Style/ConcatArrayLiterals (#23755) 2023-05-04 05:57:40 +02:00
Matt Jankowski
668a19a2f3
Fix Performance/DeletePrefix cop (#24796) 2023-05-02 21:07:45 +02:00
Matt Jankowski
f1c1dd0118
Rename with_lock to with_redis_lock to avoid confusion with ActiveRecord's method (#24741) 2023-05-02 18:16:07 +02:00
Matt Jankowski
bae694108a
Fix Performance/Count cop (#24793) 2023-05-02 18:06:43 +02:00
Matt Jankowski
5e060e1f44
Fix Performance/Sum cop (#24788) 2023-05-02 16:10:40 +02:00
Matt Jankowski
88d33f361f
Fix Lint/DuplicateBranch cop (#24766) 2023-05-02 12:57:11 +02:00
Claire
32a030dd74
Rewrite import feature (#21054) 2023-05-02 12:08:48 +02:00
dependabot[bot]
fa14976a5d
Bump connection_pool from 2.3.0 to 2.4.0 (#24290)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-04-30 15:45:02 +09:00
Nick Schonning
49fad26eca
Drop EOL Ruby 2.7 (#24237) 2023-04-27 01:46:18 +02:00
Matt Jankowski
0a5f0a8b20
Remove instance variables from helper usage (#24203) 2023-04-23 22:35:54 +02:00
Eugen Rochko
a9b5598c97
Change user settings to be stored in a more optimal way (#23630)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2023-03-30 14:44:00 +02:00
Matt Jankowski
e63524f457
Update retention model sql query (#24249) 2023-03-26 00:37:57 +01:00
Claire
8fdf49b11d
Add warning for object storage misconfiguration (#24137) 2023-03-16 22:47:01 +01:00
Christian Schmidt
bd047acc35
Replace Status#translatable? with language matrix in separate endpoint (#24037) 2023-03-16 11:07:24 +01:00
Eugen Rochko
630436ab2d
Refactor monkey-patching of PrivateAddressCheck (#24122) 2023-03-16 04:33:38 +01:00
Claire
9875be09fe
Fix dashboard crash on ElasticSearch server error (#23751) 2023-03-15 04:01:49 +01:00
Claire
1d0ad558ff
Change sidekiq-bulk's batch size from 10,000 to 1,000 jobs in one Redis call (#24034) 2023-03-15 03:45:15 +01:00
Christian Schmidt
684a970b3c
Unescape HTML entities (#24019) 2023-03-08 19:56:41 +01:00
Matt Jankowski
dfa1be4ab4
Remove unused TOCGenerator (#23986) 2023-03-06 17:33:58 +01:00
Jean byroot Boussier
922837dc96
Upgrade to latest redis-rb 4.x and fix deprecations (#23616)
Co-authored-by: Jean Boussier <jean.boussier@gmail.com>
2023-03-04 16:38:28 +01:00
Christian Schmidt
5a8c651e8f
Only offer translation for supported languages (#23879) 2023-03-03 21:06:31 +01:00
Nick Schonning
0cfdd1a401
Enable Rubocop Style/StringConcatenation defaults (#23792) 2023-02-22 09:54:36 +09:00
Nick Schonning
59c8d43d94
Autofix Rubocop Style/RescueStandardError (#23745) 2023-02-20 11:01:20 +01:00
Nick Schonning
717683d1c3
Autofix Rubocop remaining Layout rules (#23679) 2023-02-20 06:58:28 +01:00
Nick Schonning
21bf326356
Autofix Rubocop Rails/Pluck (#23730) 2023-02-20 02:28:40 +01:00
Nick Schonning
81ad6c2e39
Autofix Rubocop Style/StringLiterals (#23695) 2023-02-19 07:38:14 +09:00
Nick Schonning
2177daeae9
Autofix Rubocop Style/RedundantBegin (#23703) 2023-02-19 07:09:40 +09:00
Nick Schonning
e2a3ebb271
Autofix Rubocop Style/IfUnlessModifier (#23697) 2023-02-18 12:37:47 +01:00
Nick Schonning
8aee4de034
Autofix Rubocop Style/PerlBackrefs (#23649) 2023-02-18 14:20:20 +09:00
Nick Schonning
a6f77aa28a
Autofix Rubocop Lint/AmbiguousOperatorPrecedence (#23681) 2023-02-18 04:30:23 +01:00
Nick Schonning
634368c491
Autofix Rubocop Lint/SymbolConversion (#23683) 2023-02-18 03:23:49 +01:00
Nick Schonning
669f6d2c0a
Run rubocop formatting except line length (#23632) 2023-02-18 06:56:20 +09:00
Claire
0c9eac80d8
Fix unbounded recursion in post discovery (#23506)
* Add a limit to how many posts can get fetched as a result of a single request

* Add tests

* Always pass `request_id` when processing `Announce` activities

---------

Co-authored-by: nametoolong <nametoolong@users.noreply.github.com>
2023-02-10 22:16:37 +01:00
Nick Schonning
0592937264
Apply Rubocop Rails/WhereNot (#23448)
* Apply Rubocop Rails/WhereNot

* Update spec for where.not
2023-02-08 10:39:57 +01:00
Nick Schonning
0d1f192c54
Apply Rubocop Performance/BlockGivenWithExplicitBlock (#23441)
* Apply Rubocop Performance/BlockGivenWithExplicitBlock

* Unprefix used block parameter
2023-02-08 10:36:23 +01:00
Nick Schonning
203739dd3a
Apply Rubocop Performance/StringIdentifierArgument (#23444) 2023-02-08 02:36:20 +01:00
Nick Schonning
ef8988aef2
Apply Rubocop Performance/DoubleStartEndWith (#23442) 2023-02-08 10:06:48 +09:00
Nick Schonning
ed570050c6
Autofix Rails/EagerEvaluationLogMessage (#23429)
* Autofix Rails/EagerEvaluationLogMessage

* Update spec for debug block syntax
2023-02-07 03:44:36 +01:00
Eugen Rochko
4c919812b9
Change number of stored items in home feed from 400 to 800 (#23349) 2023-02-01 11:24:22 +01:00
Claire
41517a4845
Fix spurious admin dashboard warning when using ElasticSearch 7.x (#23064)
Some 7.x ElasticSearch versions support some 6.x nodes, thus the version check
is inadequate. I am not sure there is a good way to check if a server
implements all the 7.x APIs, so check server version and minimum wire version
instead.
2023-01-18 16:21:48 +01:00
Markus Unterwaditzer
0c689b9d01
fix: allow verification when page size exceeds 1MB (using HTML5 parser) (#22879)
* fix: allow verification when page size exceeds 1MB
Truncates the page after 1MB instead

Closes #15316

* switch to HTML5 parser, fix rubocop errors

* undo rubocop fixes

Co-authored-by: Chris Zubak-Skees <chriszs@gmail.com>
2023-01-11 21:59:13 +01:00
Kaspar V
ae62e5fa53
Fix/remove calling private method with send in model (#22951)
* fix(status): remove send usage for private unlink_from_conversations

- make unlink_from_conversations public method
- rename unlink_from_conversations to unlink_from_conversations!
- fix send call on private method in statuses_vacuum and batched_remove_status_service

* fix(feeds_vacuum): replace find_in_batches with in_batches

because active record query results should be a little more efficient than
itterating with map and each. Postgres can grasp such lists of ids much quicker
than ruby can.
Will probably make allmost no difference, but cannot hurt either.
2023-01-11 21:57:24 +01:00
Vyr Cossont
06f979098c
Check OpenSearch compatibility version instead of regular version (#22422)
Fixes #18535.
2023-01-05 13:44:31 +01:00
Jeong Arm
fdd1facba1
Fix home TL could contain post from who blocked me (#22849)
* Fix home tl contains post from who blocked me

* Add test

* Fix feed_manager's build_crutches

blocked_by was not includes status' owner

* Add test for status from I blocked

* Fix typo
2023-01-05 13:30:38 +01:00
Claire
d13702ac06
Fix status cache hydration discrepancy (#19879) 2022-12-15 18:55:55 +01:00
Claire
673c54f114
Fix inability to use local LibreTranslate without setting ALLOWED_PRIVATE_ADDRESSES (#21926)
Fixes #20029
2022-12-15 17:04:38 +01:00
s0
52540771b0
Fix crash in elasticsearch_check.rb (#21006)
Nil unwrap causes the admin dashboard to crash/500 when the Chewy client info version number value is nil.
This occurs when running another ES-compatible backend such as MeiliSearch.
Obviously it would be good for chewy to recognise upstream but at least avoiding the crash would be fine.
2022-12-15 15:56:48 +01:00
Jeong Arm
e8cc63105f
Don't delivery a reply to domains which are blocked by author (#22117)
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-12-15 15:52:06 +01:00
Claire
c8849d6cee
Fix unbounded recursion in account discovery (#22025)
* Fix trying to fetch posts from other users when fetching featured posts

* Rate-limit discovery of new subdomains

* Put a limit on recursively discovering new accounts
2022-12-07 00:15:24 +01:00
Claire
68d1df8bc3
Fix some performance issues with /admin/instances (#21907)
/admin/instances?availability=failing remains wholly unefficient
2022-12-01 10:32:10 +01:00
Kaspar V
47f0d7021e
refactor(vacuum statuses): reduce amount of db queries and load for each query - improve performance (#21487)
* refactor(statuses_vacuum): remove dead code - unused

Method is not called inside class and private.
Clean up dead code.

* refactor(statuses_vacuum): make retention_period present test explicit

This private method only hides functionality.
It is best practice to be as explicit as possible.

* refactor(statuses_vacuum): improve query performance

- fix statuses_scope having sub-select for Account.remote scope by
  `joins(:account).merge(Account.remote)`
- fix statuses_scope unnecessary use of `Status.arel_table[:id].lt`
  because it is inexplicit, bad practice and even slower than normal
  `.where('statuses.id < ?'`
- fix statuses_scope remove select(:id, :visibility) for having reusable
  active record query batches (no re queries)
- fix vacuum_statuses! to use in_batches instead of find_in_batches,
  because in_batches delivers a full blown active record query result,
  in stead of an array - no requeries necessary
- send(:unlink_from_conversations) not to perform another db query, but
  reuse the in_batches result instead.
- remove now obsolete remove_from_account_conversations method
- remove_from_search_index uses array of ids, instead of mapping
  the ids from an array - this should be more efficient
- use the in_batches scope to call delete_all, instead of running
  another db query for this - because it is again more efficient
- add TODO comment for calling models private method with send

* refactor(status): simplify unlink_from_conversations

- add `has_many through:` relation mentioned_accounts
- use model scope local instead of method call `Status#local?`
- more readable add account to inbox_owners when account.local?

* refactor(status): searchable_by way less sub selects

These queries all included a sub-select. Doing the same with a joins
should be more efficient.
Since this method does 5 such queries, this should be significant,
since it technically halves the query count.

This is how it was:

```ruby
[3] pry(main)> Status.first.mentions.where(account: Account.local, silent: false).explain
  Status Load (1.6ms)  SELECT "statuses".* FROM "statuses" WHERE "statuses"."deleted_at" IS NULL ORDER BY "statuses"."id" DESC LIMIT $1  [["LIMIT", 1]]
  Mention Load (1.5ms)  SELECT "mentions".* FROM "mentions" WHERE "mentions"."status_id" = $1 AND "mentions"."account_id" IN (SELECT "accounts"."id" FROM "accounts" WHERE "accounts"."domain" IS NULL) AND "mentions"."silent" = $2  [["status_id", 109382923142288414], ["silent", false]]
=> EXPLAIN for: SELECT "mentions".* FROM "mentions" WHERE "mentions"."status_id" = $1 AND "mentions"."account_id" IN (SELECT "accounts"."id" FROM "accounts" WHERE "accounts"."domain" IS NULL) AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
                                                    QUERY PLAN
------------------------------------------------------------------------------------------------------------------
 Nested Loop  (cost=0.15..23.08 rows=1 width=41)
   ->  Seq Scan on accounts  (cost=0.00..10.90 rows=1 width=8)
         Filter: (domain IS NULL)
   ->  Index Scan using index_mentions_on_account_id_and_status_id on mentions  (cost=0.15..8.17 rows=1 width=41)
         Index Cond: ((account_id = accounts.id) AND (status_id = '109382923142288414'::bigint))
         Filter: (NOT silent)
(6 rows)
```

This is how it is with this change:

```ruby
[4] pry(main)> Status.first.mentions.joins(:account).merge(Account.local).active.explain
  Status Load (1.7ms)  SELECT "statuses".* FROM "statuses" WHERE "statuses"."deleted_at" IS NULL ORDER BY "statuses"."id" DESC LIMIT $1  [["LIMIT", 1]]
  Mention Load (0.7ms)  SELECT "mentions".* FROM "mentions" INNER JOIN "accounts" ON "accounts"."id" = "mentions"."account_id" WHERE "mentions"."status_id" = $1 AND "accounts"."domain" IS NULL AND "mentions"."silent" = $2  [["status_id", 109382923142288414], ["silent", false]]
=> EXPLAIN for: SELECT "mentions".* FROM "mentions" INNER JOIN "accounts" ON "accounts"."id" = "mentions"."account_id" WHERE "mentions"."status_id" = $1 AND "accounts"."domain" IS NULL AND "mentions"."silent" = $2 [["status_id", 109382923142288414], ["silent", false]]
                                                    QUERY PLAN
------------------------------------------------------------------------------------------------------------------
 Nested Loop  (cost=0.15..23.08 rows=1 width=41)
   ->  Seq Scan on accounts  (cost=0.00..10.90 rows=1 width=8)
         Filter: (domain IS NULL)
   ->  Index Scan using index_mentions_on_account_id_and_status_id on mentions  (cost=0.15..8.17 rows=1 width=41)
         Index Cond: ((account_id = accounts.id) AND (status_id = '109382923142288414'::bigint))
         Filter: (NOT silent)
(6 rows)
```
2022-11-27 20:41:18 +01:00
Claire
71c92d3f56
Fix emoji substitution not applying only to text nodes in backend code (#20641)
Signed-off-by: Claire <claire.github-309c@sitedethib.com>

Signed-off-by: Claire <claire.github-309c@sitedethib.com>
2022-11-14 20:26:21 +01:00
Bearice Ren
28cda42af5
fixes ArgumentError when proxy is used (#20420)
* fixes ArgumentError when proxy is used

* Update app/lib/request.rb

Co-authored-by: Claire <claire.github-309c@sitedethib.com>

Co-authored-by: Eugen Rochko <eugen@zeonfederated.com>
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
2022-11-11 21:31:03 +01:00
Eugen Rochko
f8e8e622e5
Change incoming activity processing to happen in ingress queue (#20264) 2022-11-10 14:21:51 +01:00
James Tucker
78a6b871fe
Improve performance by avoiding regex construction (#20215)
```ruby
10.times { p /#{FOO}/.object_id }
10.times { p FOO_RE.object_id }
```
2022-11-10 05:49:30 +01:00
Claire
c989faaa62
Change Request connection logic to try both IPv6 and IPv4 when available (#20108)
Fixes #19751
2022-11-08 16:36:26 +01:00
James Tucker
833d9c2f1c
Improve performance by avoiding method cache busts (#19957)
Switch to monkey-patching http.rb rather than a runtime extend of each
response, so as to avoid busting the global method cache. A guard is
included that will provide developer feedback in development and test
environments should the monkey patch ever collide.
2022-11-08 04:00:27 +01:00
Claire
3114c826a7
Fix filter handling in status cache hydration (#19963) 2022-11-07 19:47:48 +01:00
Claire
bb89f83cc0
Fix additional issues with status cache hydration (#19747)
* Spare one SQL query when hydrating polls

* Improve tests

* Fix more discrepancies

* Fix possible crash when the status has no application set
2022-11-04 20:01:33 +01:00
Claire
03b991de6c
Fix various issues with store hydration (#19746)
- Improve tests
- Fix possible crash when application of a reblogged post isn't set
- Fix discrepancies around favourited and reblogged attributes
- Fix discrepancies around pinned attribute
- Fix polls not being hydrated
2022-11-04 19:33:16 +01:00
Claire
c2170991c7
Fix reblogs being discarded after the reblogged status (#19731) 2022-11-04 16:31:44 +01:00
Eugen Rochko
5f9e47be34
Add caching for payload serialization during fan-out (#19642) 2022-11-04 13:21:06 +01:00
Eugen Rochko
1546538de9
Fix improperly checking for blocked domain on followed hashtags (#19472)
Fix #19469
2022-10-26 20:40:56 +02:00
Eugen Rochko
bf0ab3e0fa
Fix vacuum scheduler missing lock, locks never expiring (#19458)
Remove vacuuming of orphaned preview cards
2022-10-26 12:10:48 +02:00
Eugen Rochko
8f07381856
Revert "Remove preference to aggregate reblogs in home/list feeds (#18112)" (#19463)
This reverts commit af396fa35f.
2022-10-26 09:10:18 +02:00
Claire
30453fab80
Add mention of the translation provider when translating a post (#19433) 2022-10-24 18:37:57 +02:00
Claire
8046cf34d6
Change “Translate” button to only show up when a translation backend is configured (#19434)
* Change “Translate” button to only show up when a translation backend is configured

Fixes #19346

* Add `translation` attribute to /api/v2/instance to expose whether the translation feature is enabled

Fixes #19328
2022-10-24 18:30:58 +02:00
Takeshi Umeda
1d34eff63f
Add featured tag add/remove activity handler (#19408) 2022-10-22 11:49:41 +02:00
Eugen Rochko
839f893168
Change public accounts pages to mount the web UI (#19319)
* Change public accounts pages to mount the web UI

* Fix handling of remote usernames in routes

- When logged in, serve web app
- When logged out, redirect to permalink
- Fix `app-body` class not being set sometimes due to name conflict

* Fix missing `multiColumn` prop

* Fix failing test

* Use `discoverable` attribute to control indexing directives

* Fix `<ColumnLoading />` not using `multiColumn`

* Add `noindex` to accounts in REST API

* Change noindex directive to not be rendered by default before a route is mounted

* Add loading indicator for detailed status in web UI

* Fix missing indicator appearing while account is loading in web UI
2022-10-20 14:35:29 +02:00
Eugen Rochko
43b5d5e38d
Add logged-out access to the web UI (#18961) 2022-09-29 04:39:33 +02:00
Yamagishi Kazutoshi
d86dd067ce
Fix auto detect language for translate service (#19244) 2022-09-27 23:33:56 +02:00
Eugen Rochko
5c9abdeff1
Add retention policy for cached content and media (#19232) 2022-09-27 03:08:19 +02:00
Eugen Rochko
0d6b878808
Add user content translations with configurable backends (#19218) 2022-09-23 23:00:12 +02:00
Claire
8cf7006d4e
Refactor ActivityPub handling to prepare for non-Account actors (#19212)
* Move ActivityPub::FetchRemoteAccountService to ActivityPub::FetchRemoteActorService

ActivityPub::FetchRemoteAccountService is kept as a wrapper for when the actor is
specifically required to be an Account

* Refactor SignatureVerification to allow non-Account actors

* fixup! Move ActivityPub::FetchRemoteAccountService to ActivityPub::FetchRemoteActorService

* Refactor ActivityPub::FetchRemoteKeyService to potentially return non-Account actors

* Refactor inbound ActivityPub payload processing to accept non-Account actors

* Refactor inbound ActivityPub processing to accept activities relayed through non-Account

* Refactor how Account key URIs are built

* Refactor Request and drop unused key_id_format parameter

* Rename ActivityPub::Dereferencer `signature_account` to `signature_actor`
2022-09-21 22:45:57 +02:00