Commit graph

391 commits

Author SHA1 Message Date
Eugen Rochko
a2cda74ba3 Fix connect timeout not being enforced (#9329)
* Fix connect timeout not being enforced

The loop was catching the timeout exception that should stop execution, so the next IP would no longer be within a timed block, which led to requests taking much longer than 10 seconds.

* Use timeout on each IP attempt, but limit to 2 attempts

* Fix code style issue

* Do not break Request#perform if no block given

* Update method stub in spec for Request

* Move timeout inside the begin/rescue block

* Use Resolv::DNS with timeout of 1 to get IP addresses

* Update Request spec to stub Resolv::DNS instead of Addrinfo

* Fix Resolve::DNS stubs in Request spec
2018-11-23 22:17:04 +01:00
Eugen Rochko
ecc58c0f23 Prevent multiple handlers for Delete of Actor from running (#9292) 2018-11-23 22:15:12 +01:00
Eugen Rochko
6d4438a6ae Remove intermediary arrays when creating hash maps from results (#9291) 2018-11-23 22:15:03 +01:00
ThibG
886ef1cc38 Fix emoji update date processing (#9255) 2018-11-23 22:14:15 +01:00
Eugen Rochko
be202f9377
Accept the same payload in multiple inboxes and deliver (#9150) 2018-10-30 15:03:55 +01:00
m.b
2b18f5f85d Add Page AP type support (#9121) 2018-10-29 13:23:29 +01:00
ThibG
e53cc673e7 Ignore invalid hashtags on remote statuses instead of rejecting them (#9118)
Fixes #9115
2018-10-26 22:48:35 +02:00
ThibG
82e7988afc Fix missing mention argument when processing incoming Create activities (#9114)
* Fix missing `mention` argument when processing incoming Create activities

* Fix typo (param → params)
2018-10-26 12:59:59 +02:00
Eugen Rochko
d4cf963749
Allow inbox owner to view implicitly targeted ActivityPub payload (#9093)
Fix #9091
2018-10-25 18:12:22 +02:00
Eugen Rochko
fd5285658f
Add option to block reports from domain (#8830) 2018-10-20 08:02:44 +02:00
Eugen Rochko
ddd30f331c
Improve support for aspects/circles (#8950)
* Add silent column to mentions

* Save silent mentions in ActivityPub Create handler and optimize it

Move networking calls out of the database transaction

* Add "limited" visibility level masked as "private" in the API

Unlike DMs, limited statuses are pushed into home feeds. The access
control rules between direct and limited statuses is almost the same,
except for counter and conversation logic

* Ensure silent column is non-null, add spec

* Ensure filters don't check silent mentions for blocks/mutes

As those are "this person is also allowed to see" rather than "this
person is involved", therefore does not warrant filtering

* Clean up code

* Use Status#active_mentions to limit returned mentions

* Fix code style issues

* Use Status#active_mentions in Notification

And remove stream_entry eager-loading from Notification
2018-10-17 17:13:04 +02:00
Eugen Rochko
21ad21cb50
Improve signature verification safeguards (#8959)
* Downcase signed_headers string before building the signed string

The HTTP Signatures draft does not mandate the “headers” field to be downcased,
but mandates the header field names to be downcased in the signed string, which
means that prior to this patch, Mastodon could fail to process signatures from
some compliant clients. It also means that it would not actually check the
Digest of non-compliant clients that wouldn't use a lowercased Digest field
name.

Thankfully, I don't know of any such client.

* Revert "Remove dead code (#8919)"

This reverts commit a00ce8c92c.

* Restore time window checking, change it to 12 hours

By checking the Date header, we can prevent replaying old vulnerable
signatures. The focus is to prevent replaying old vulnerable requests
from software that has been fixed in the meantime, so a somewhat long
window should be fine and accounts for timezone misconfiguration.

* Escape users' URLs when formatting them

Fixes possible HTML injection

* Escape all string interpolations in Formatter class

Slightly improve performance by reducing class allocations
from repeated Formatter#encode calls

* Fix code style issues
2018-10-12 00:15:55 +02:00
Eugen Rochko
61d44dd11f
Fix typo in ActivityPub Create handler (#8952)
Regression from #8951
2018-10-11 02:10:15 +02:00
Eugen Rochko
790d3bc637
Move network calls out of transaction in ActivityPub handler (#8951)
Mention and emoji code may perform network calls, but does not need
to do that inside the database transaction. This may improve availability
of database connections when using pgBouncer in transaction mode.
2018-10-11 00:50:18 +02:00
Eugen Rochko
774ac47373
Add conversations API (#8832)
* Add conversations API

* Add web UI for conversations

* Add test for conversations API

* Add tests for ConversationAccount

* Improve web UI

* Rename ConversationAccount to AccountConversation

* Remove conversations on block and mute

* Change last_status_id to be a denormalization of status_ids

* Add optimistic locking
2018-10-07 23:44:58 +02:00
Jeong Arm
144d73730d Leave unknown language as nil if account is remote (#8861)
* Force use language detector if account is remote

* Set unknown remote toot's language as nil
2018-10-05 19:17:46 +02:00
ThibG
ba444797d2 Fix handling of ActivityPub activities lacking some attributes (#8864) 2018-10-03 23:44:13 +02:00
Eugen Rochko
ef69c655cc
Fix class autoloading issue in ActivityPub::Activity::Create (#8820) 2018-09-28 17:02:53 +02:00
cbayerlein
4b78546135 Exclude replies from list timelines (#8683)
* Changed list behaviour

I added the following line to the FeedManager (app/lib/feed_manager.rb) in the push_to_list function:

`return false if status.reply?`

Now all posts that are replies are filtered out, so that now only "genuine" posts are displayed in the list.

This is a first approach to solve issue #5916

* Update feed_manager.rb

As suggested by @Gargron
2018-09-28 00:37:21 +02:00
ふぁぼ原
f7a6f9489d Add a new preference to always hide all media (#8569) 2018-09-25 05:09:35 +02:00
Matt Sweetman
1889526e23 Add user preference to always expand toots marked with content warnings (#8762) 2018-09-24 05:44:01 +02:00
Eugen Rochko
f4d549d300
Redesign forms, verify link ownership with rel="me" (#8703)
* Verify link ownership with rel="me"

* Add explanation about verification to UI

* Perform link verifications

* Add click-to-copy widget for verification HTML

* Redesign edit profile page

* Redesign forms

* Improve responsive design of settings pages

* Restore landing page sign-up form

* Fix typo

* Support <link> tags, add spec

* Fix links not being verified on first discovery and passive updates
2018-09-18 16:45:58 +02:00
Renato "Lond" Cerqueira
fe56d26f7b Fix autoplay issue with spoiler tag (#8540)
Add tests to avoid similar issues in the future
2018-08-31 15:16:59 +02:00
Eugen Rochko
e3764bdb52
Do not sign useless User-Agent or Accept-Encoding headers (#8533)
Fix #8080
2018-08-31 04:22:52 +02:00
Renato "Lond" Cerqueira
11658d8653 Add animate custom emoji param to embed pages (#8507)
* Add animate custom emoji param to embed pages

* Rename param, use it for avatars and gifs

* Fix issues pointed by codeclimate and breaking test

* Ignore brakeman warning
2018-08-30 23:14:01 +02:00
Eugen Rochko
cabdbb7f9c
Add CLI task for rotating keys (#8466)
* If an Update is signed with known key, skip re-following procedure

Because it means the remote actor did *not* lose their database

* Add CLI method for rotating keys

    bin/tootctl accounts rotate [USERNAME]

Generates a new RSA key per account and sends out an Update activity
signed with the old key.

* Key rotation: Space out Update fan-outs every 5 minutes per 1000 accounts

* Skip suspended accounts in key rotation
2018-08-26 20:21:03 +02:00
Quint Guvernator
da13fa5021 Fix low-hanging rubocop gripes (#8458)
* rubocop: quit being so picky

* rubocop: miscellany

* rubocop: prefer present to blank
2018-08-26 19:22:46 +02:00
Eugen Rochko
b4ba4b1b5d
Spread out crawling randomly to avoid DDoSing the link (#8445)
* Spread out crawling randomly to avoid DDoSing the link

Fix #4486

* Remove trailing whitespace
2018-08-26 00:33:57 +02:00
M Somerville
95bd0d4528 Support ActivityStreams’ summaryMap. (#8422)
In the same way as contentMap and nameMap.
2018-08-25 13:27:34 +02:00
Jakub Mendyk
6cb3514d64 Add ability to change an instance default theme from the administration panel (#7092) (#8381)
* Add default_settings class method to ScopedSettings

ScopedSettings was extended to use value of unscoped setting instead of
only using defaults set in config/settings.yml for selected settings.
This adds possibility for admins to set default values of users' settings,
for example default theme (as requested in #7092).

* Add ability to change an instance default theme

Closes #7092
2018-08-23 14:17:35 +02:00
ThibG
42573b76f1 Do not crash if remote custom emoji does not define updated date (fixes #8376) (#8377) 2018-08-23 00:27:58 +02:00
Eugen Rochko
802cf6a4c5
Improve federated ID validation (#8372)
* Fix URI not being sufficiently validated with prefetched JSON

* Add additional id validation to OStatus documents, when possible
2018-08-22 20:55:14 +02:00
ThibG
59f7f4c923 Implement Undo { Accept { Follow } } (fixes #8234) (#8245)
* Add Follow#revoke_request!

* Implement Undo { Accept { Follow } } (fixes #8234)
2018-08-17 16:24:56 +02:00
ThibG
1ee675d68b Use correct activity id in Accept when receiving duplicate Follow (fixes #8218) (#8244) 2018-08-17 14:08:17 +02:00
Eugen Rochko
39e361a56d
Expect relays to answer with accept/reject (#8179) 2018-08-12 18:16:26 +02:00
ThibG
c52bcc0331 Serialize text-less statuses as '.' over OStatus (fixes #7856) (#8126) 2018-08-05 12:22:23 +02:00
Eugen Rochko
07b799468d
Fix incorrect context definition for the "featured" keyword (#8090)
* Fix incorrect context definition for the "featured" keyword

Fix #8077

* Adjust context definition for movedTo
2018-07-30 19:33:37 +02:00
Eugen Rochko
a23ac107e4
Fix activity:interactions counter to count all interactions (#8037) 2018-07-16 18:35:43 +02:00
Eugen Rochko
8f64b17d98 Whitelist dat/ipfs/gopher links in sanitizer (#8034)
Fix #7994
2018-07-16 22:17:43 +09:00
Eugen Rochko
2354b10eb5
Add admin dashboard (#8029) 2018-07-16 01:11:53 +02:00
Eugen Rochko
38e9662d78
Disable language detection for texts shorter than 140 characters (#8010)
If the input text is blank after preparation (only mention, or
only URL, or empty as in a media post), then use nil as language,
since it's OK to show to everyone.

Otherwise, always fall back to the server's default locale
2018-07-14 04:05:36 +02:00
Eugen Rochko
47c59591e0 Do not count self in potential friendships (#8014) 2018-07-14 10:44:23 +09:00
Eugen Rochko
20fefdb714
Make whole-word filter regex consistent between Ruby and JS (#7987) 2018-07-10 03:01:50 +02:00
ThibG
1ca4e51eb3 Add option to not consider word boundaries when processing keyword filtering (#7975)
* Add option to not consider word boundaries when filtering phrases

* Add a few tests for keyword/phrase filtering
2018-07-09 02:22:09 +02:00
Eugen Rochko
404c7702ec
In keyword filter, account for reblogs, HTML and whole-words (#7960)
* In keyword filter, account for reblogs, HTML and whole-words

* Match whole words in JS filter, too

* Fix typo
2018-07-06 02:15:44 +02:00
Eugen Rochko
4b198b172d
Check reblogged status for blocked/muted mentions (#7957) 2018-07-05 18:43:37 +02:00
Eugen Rochko
da8fe8079e
Re-add follow recommendations API (#7918)
* Re-add follow recommendations API

    GET /api/v1/suggestions

Removed in 8efa081f21 due to Neo4J
dependency. The algorithm uses triadic closures, takes into account
suspensions, blocks, mutes, domain blocks, excludes locked and moved
accounts, and prefers more recently updated accounts.

* Track interactions with people you don't follow

Replying to, favouriting and reblogging someone you're not following
will make them show up in follow recommendations. The interactions
have different weights:

- Replying is 1
- Favouriting is 10 (decidedly positive interaction, but private)
- Reblogging is 20

Following them, muting or blocking will remove them from the list,
obviously.

* Remove triadic closures, ensure potential friendships are trimmed
2018-07-03 01:47:56 +02:00
MIYAGI Hikaru
ddd0bb69e1 Merge HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY into ALLOW_ACCESS_TO_HIDDEN_SERVICE (#7901)
If Mastodon accesses to the hidden service via transparent proxy, it's needed to avoid checking whether it's a private address, since `.onion` is resolved to a private address.
I was previously using the `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` to provide that function. However, I realized that using `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` is redundant, since this specification is always used with `ALLOW_ACCESS_TO_HIDDEN_SERVICE`. Therefore, I decided to integrate the setting of `HIDDEN_SERVICE_VIA_TRANSPARENT_PROXY` into` ALLOW_ACCESS_TO_HIDDEN_SERVICE`.
2018-06-29 15:36:02 +02:00
Eugen Rochko
cdb101340a
Keyword/phrase filtering (#7905)
* Add keyword filtering

    GET|POST       /api/v1/filters
    GET|PUT|DELETE /api/v1/filters/:id

- Irreversible filters can drop toots from home or notifications
- Other filters can hide toots through the client app
- Filters use a phrase valid in particular contexts, expiration

* Make sure expired filters don't get applied client-side

* Add missing API methods

* Remove "regex filter" from column settings

* Add tests

* Add test for FeedManager

* Add CustomFilter test

* Add UI for managing filters

* Add streaming API event to allow syncing filters

* Fix tests
2018-06-29 15:34:36 +02:00
Eugen Rochko
a58ec29631
Allow selecting default posting language instead of auto-detect (#7828)
* Allow selecting default posting language instead of auto-detect

* Enable default language setting in credentials API

* Fix form saving
2018-06-17 18:57:31 +02:00