Commit graph

536 commits

Author SHA1 Message Date
Eugen Rochko
658cbc9425
Improve PostStatusService performance (#7317)
Offload creation of local notifications to a worker. Remove two
redundant SQL queries from ProcessMentionsService, remove n+1
XML/JSON serialization via memoization
2018-05-02 22:10:57 +02:00
Eugen Rochko
cb5b5cb5f7
Slightly reduce RAM usage (#7301)
* No need to re-require sidekiq plugins, they are required via Gemfile

* Add derailed_benchmarks tool, no need to require TTY gems in Gemfile

* Replace ruby-oembed with FetchOEmbedService

Reduce startup by 45382 allocated objects

* Remove preloaded JSON-LD in favour of caching HTTP responses

Reduce boot RAM by about 6 MiB

* Fix tests

* Fix test suite by stubbing out JSON-LD contexts
2018-05-02 18:58:48 +02:00
abcang
71a7cea73f Keep notification when muting_notifications is true (#7311)
* Keep notification when muting_notifications is true

* Retrun mute object

* Fix test
2018-05-02 16:14:51 +02:00
Eugen Rochko
f62539ce5c
Remove most behaviour disparities between blocks and mutes (#7231)
* Remove most behaviour disparities between blocks and mutes

The only differences between block and mute should be:

- Mutes can optionally NOT affect notifications
- Mutes should not be visible to the muted

Fix #7230
Fix #5713

* Do not allow boosting someone you blocked

Fix #7248

* Do not allow favouriting someone you blocked

* Fix nil error in StatusPolicy
2018-05-02 15:50:20 +02:00
Eugen Rochko
965345316f
Guard against nil URLs in Request class (#7284)
Fix #7265
2018-05-02 15:44:22 +02:00
Surinna Curtis
dc786c0cf4 Support Actors/Statuses with multiple types (#7305)
* Add equals_or_includes_any? helper in JsonLdHelper

* Support arrays in JSON-LD type fields for actors/tags/objects.

* Spec for resolving accounts with extension types

* Style tweaks for codeclimate
2018-05-02 12:40:24 +02:00
Emelia Smith
495303d9b8 Prevent suspended accounts from appearing in AccountSearchService (#7246) 2018-04-23 21:27:18 +02:00
Eugen Rochko
75c4ab9d12
Remove "nsfw" category for sensitive statuses in OStatus serializer (#7048)
Fix #7011
2018-04-22 22:09:03 +02:00
Eugen Rochko
ff87d1bc3e
Rescue SSL errors when processing mentions, remove useless line (#7184) 2018-04-19 00:53:31 +02:00
Kaito Sinclaire
156b916caf Direct messages column (#4514)
* Added a timeline for Direct statuses
* Lists all Direct statuses you've sent and received
* Displayed in Getting Started
* Streaming server support for direct TL

* Changes to match other timelines in 2.0
2018-04-18 13:09:06 +02:00
ThibG
1c379b7ef4 Remove extra spaces from search API queries and public account headers (fixes #7129) (#7152) 2018-04-16 17:19:04 +02:00
Eugen Rochko
78ed4ab75f
Add bio fields (#6645)
* Add bio fields

- Fix #3211
- Fix #232
- Fix #121

* Display bio fields in web UI

* Fix output of links and missing fields

* Federate bio fields over ActivityPub as PropertyValue

* Improve how the fields are stored, add to Edit profile form

* Add rel=me to links in fields

Fix #121
2018-04-14 12:41:08 +02:00
Eugen Rochko
778562c223
Ensure SynchronizeFeaturedCollectionWorker is unique and clean up (#7043)
* Ensure SynchronizeFeaturedCollectionWorker is unique and clean up

Fix #7041

* Fix code style issue
2018-04-13 01:27:22 +02:00
Renato "Lond" Cerqueira
0c52654b52 When creating status, if no sensitive status is given, use default (#7057)
Clients using the API that do not provide the sensitive flag are always
posting with false sensitive option.
2018-04-09 23:02:42 +02:00
Nolan Lawson
6ff3b3e4db Fix nil account issue in ProcessAccountService (#7019) 2018-04-03 13:08:11 +02:00
Eugen Rochko
f890d2a766
Support all ActivityPub actor types (#6997)
Fix #6973
2018-04-02 02:10:53 +02:00
David Underwood
123a343d11 [WIP] Enable custom emoji on account pages and in the sidebar (#6124)
Federate custom emojis with accounts
2018-04-01 23:55:42 +02:00
ThibG
e573bb0990 Fix compatibility with PeerTube (#6968)
* Support fetching objects of convertible types by URL (fixes #6924)

* Ignore invalid hashtags
2018-03-30 15:44:54 +02:00
Akihiko Odaki
40e5d2303b Validate HTTP response length while receiving (#6891)
to_s method of HTTP::Response keeps blocking while it receives the whole
content, no matter how it is big. This means it may waste time to receive
unacceptably large files. It may also consume memory and disk in the
process. This solves the inefficency by checking response length while
receiving.
2018-03-26 14:02:10 +02:00
Akihiko Odaki
54b273bf99 Close http connection in perform method of Request class (#6889)
HTTP connections must be explicitly closed in many cases, and letting
perform method close connections makes its callers less redundant and
prevent them from forgetting to close connections.
2018-03-24 12:49:54 +01:00
Eugen Rochko
9381a7d9d5
Use username/domain to match existing accounts in ActivityPub (#6842)
See also: #6837, #6667
2018-03-20 14:57:46 +01:00
nightpool
566ace2d64 Add entropy to download filenames (#6811)
pretty quick fix, and with the 1 week expiration i don't think we need to be too worried about the existing files

closes #6798
2018-03-17 17:39:28 +01:00
Eugen Rochko
510c9049c7
For now, put a "." into no-text statuses with media for backcompat (#6691) 2018-03-08 08:20:49 +01:00
Eugen Rochko
64db9ed5f6
After blocking domain with reject_media, invalidate cache (#6679)
Media attachments are part of the association cache of statuses,
since they are presumed to be immutable. Unless this cache is
cleared manually, the statuses will continue to look like they
have media embedded.
2018-03-08 06:59:42 +01:00
Eugen Rochko
cfa9b6e13a
Remove text requirement when media attached from statuses (#6672) 2018-03-07 08:28:52 +01:00
Eugen Rochko
9110db41c5
Federate pinned statuses over ActivityPub (#6610)
* Federate pinned statuses over ActivityPub

* Display pinned toots in web UI

Fix #6117

* Fix migration

* Fix tests

* Update outbox_serializer.rb

* Update remove_serializer.rb

* Update add_serializer.rb

* Update fetch_featured_collection_service.rb
2018-03-04 09:19:11 +01:00
Aboobacker MK
49092945ab Fix 500 while searching after deleting a post (#6604)
Fixes #6602
2018-03-03 19:45:06 +01:00
Eugen Rochko
41a01bec23
Federated reports (#6570)
* Fix #2176: Federated reports

* UI for federated reports

* Add spec for ActivityPub Flag handler

* Add spec for ReportService
2018-02-28 06:54:55 +01:00
Eugen Rochko
61ed133fea
Account archive download (#6460)
* Fix #201: Account archive download

* Export actor and private key in the archive

* Optimize BackupService

- Add conversation to cached associations of status, because
  somehow it was forgotten and is source of N+1 queries
- Explicitly call GC between batches of records being fetched
  (Model class allocations are the worst offender)
- Stream media files into the tar in 1MB chunks
  (Do not allocate media file (up to 8MB) as string into memory)
- Use #bytesize instead of #size to calculate file size for JSON
  (Fix FileOverflow error)
- Segment media into subfolders by status ID because apparently
  GIF-to-MP4 media are all named "media.mp4" for some reason

* Keep uniquely generated filename in Paperclip::GifTranscoder

* Ensure dumped files do not overwrite each other by maintaing directory partitions

* Give tar archives a good name

* Add scheduler to remove week-old backups

* Fix code style issue
2018-02-21 23:21:32 +01:00
Eugen Rochko
f7765acf9d
Fix #5173: Click card to embed external content (#6471) 2018-02-15 07:04:28 +01:00
Eugen Rochko
3ebc0ad4d3
Full-text search for authorized statuses (#6423)
* Add full-text search for authorized statuses

- Search API will return statuses that match the query
- Only for logged in users
- Only if you are author of the status,
- Or you were mentioned in it
- Or you favourited or reblogged it
- Configuration over `ES_ENABLED`, `ES_HOST`, `ES_PORT`, `ES_PREFIX`
- Run `rails chewy:deploy` to create & populate index

Fix #5880
Fix #4293
Fix #1152

* Add commented out docker-compose configuration for ES container

* Optimize index import, filter search results

* Add basic normalization to the index

* Add better stemming and normalization to the index

* Skip webfinger request if search query includes both @ and a space

* Fix code style

* Visually separate search result sections

* Fix code style issues
2018-02-09 23:04:47 +01:00
abcang
7e5c433dfc Fix saving of oEmbed image (#6409) 2018-02-02 11:57:59 +01:00
Akihiko Odaki
613e7c7521 Rename ResolveRemoteAccountService to ResolveAccountService (#6327)
The service used to be named ResolveRemoteAccountService resolves local
accounts as well.
2018-01-22 14:25:09 +01:00
Akihiko Odaki
17cecd75ca Rename FetchRemoteResourceService to ResolveURLService (#6328)
The service used to be named FetchRemoteResourceService resolves local
URL as well.
2018-01-22 14:24:22 +01:00
Renato "Lond" Cerqueira
e56404be41 When must_be_following_dm is on, only notify if recipient dm'ed user (#6283)
* When must_be_following_dm is on, only notify if recipient dm'ed user
Currently, when must_be_following_dm is on, if a user sends a direct
message replying to any status from the recipient, the recipient gets a
notification. This should not be the case, as if the recipient posted
something publicly this can be used to spam their notifications.

* Refactor replied_to_status_is_direct_message?
Following suggestion in PR
2018-01-18 16:12:10 +01:00
Eugen Rochko
7badad7797
Fix home regeneration (#6251)
* Fix regeneration marker not being removed after completion

* Return HTTP 206 from /api/v1/timelines/home if regeneration in progress
Prioritize RegenerationWorker by putting it into default queue

* Display loading indicator and poll home timeline while it regenerates

* Add graphic to regeneration message

* Make "not found" indicator consistent with home regeneration
2018-01-17 23:56:03 +01:00
puckipedia
74c1c9ec01 Allow attributedTo in a status to be an embedded object (#6238) 2018-01-15 06:51:46 +01:00
Eugen Rochko
e4a241abef
Fix bad URL schemes being accepted (#6219)
* Fix actors accepting invalid URI schemes or different host between URI and URL

* Fix statuses accepting invalid URI scheme or different host to actor

* Adjust tests to new requirements

* Improve readability of mismatching_origin?/invalid_origin? methods
2018-01-08 05:00:23 +01:00
ThibG
a594139115 When fetching an ActivityPub-enabled status, do not re-request it as text/html (#6196) 2018-01-05 22:42:50 +01:00
Eugen Rochko
02ba03d6db
Send one Delete of Actor in ActivityPub when account is suspended (#6172) 2018-01-04 14:40:49 +01:00
Eugen Rochko
7d4f4f9aab
Fix FetchAtomService not finding alternatives if there's a Link header (#6170)
without them, such as is the case with GNU social

Fixes the ability to find GNU social accounts via URL in search and
when using remote follow function
2018-01-04 04:56:04 +01:00
Eugen Rochko
d319b3dbe4
Update moved-to property when it's removed too (#6160)
* Fix #6140 - Update moved-to property when it's removed too

* Remove trailing whitespace
2018-01-03 00:38:20 +01:00
ThibG
cea98e0c12 Reduce the number of synchronous resolves when posting toots (#6075) 2017-12-22 02:15:08 +01:00
Eugen Rochko
a8deb6648b
Fix redundant HTTP request in FetchLinkCardService (#6002) 2017-12-13 12:15:28 +01:00
Eugen Rochko
5706fe18c2
Fix #5952 - NameError (regression from #5762) (#5999)
* Fix #5952 - NameError (regression from #5762)

* Fix
2017-12-13 04:12:38 +01:00
Akihiko Odaki
f9f6918148 Store preview image for embedded photo in preview cards (#5986)
The preview image would be useful to embed in timeline.
2017-12-12 15:54:38 +01:00
Renato "Lond" Cerqueira
19257d91bf Return false if object does not respond to url (#5988)
Avoid error when the service returns a mostly valid oembed, but has no
url in it, causing a MethodError: undefined method `url'
for #<OEmbed::Response::Photo:0x000056505def9620>
2017-12-12 15:12:09 +01:00
Renato "Lond" Cerqueira
fe180f18ff Change conditional to avoid nil into string error in sidekiq (#5987)
* Change conditional to avoid nil into string error in sidekiq
When obtaining information about users with mastodon in a different
subdomain, sidekiq was giving out a 'no implicit conversion of nil into String'

* Use presence instead of blank? with ternary.
Following suggestion on PR
2017-12-12 15:11:13 +01:00
Andrea Scarpino
6a82939adb Fix account and tag searches with leading/trailing spaces (#5965)
* Strip leading & trailing spaces from account query

* Strip leading & trailing spaces from tag search
2017-12-10 19:35:46 +01:00
Yamagishi Kazutoshi
e7d55df38d Ignore HEAD method if does not support (#5949) 2017-12-09 16:53:40 +01:00