Replace Gson library with Moshi (#4309)
**! ! Warning**: Do not merge before testing every API call and database read involving JSON ! **Gson** is obsolete and has been superseded by **Moshi**. But more importantly, parsing Kotlin objects using Gson is _dangerous_ because Gson uses Java serialization and is **not Kotlin-aware**. This has two main consequences: - Fields of non-null types may end up null at runtime. Parsing will succeed, but the code may crash later with a `NullPointerException` when trying to access a field member; - Default values of constructor parameters are always ignored. When absent, reference types will be null, booleans will be false and integers will be zero. On the other hand, Kotlin-aware parsers like **Moshi** or **Kotlin Serialization** will validate at parsing time that all received fields comply with the Kotlin contract and avoid errors at runtime, making apps more stable and schema mismatches easier to detect (as long as logs are accessible): - Receiving a null value for a non-null type will generate a parsing error; - Optional types are declared explicitly by adding a default value. **A missing value with no default value declaration will generate a parsing error.** Migrating the entity declarations from Gson to Moshi will make the code more robust but is not an easy task because of the semantic differences. With Gson, both nullable and optional fields are represented with a null value. After converting to Moshi, some nullable entities can become non-null with a default value (if they are optional and not nullable), others can stay nullable with no default value (if they are mandatory and nullable), and others can become **nullable with a default value of null** (if they are optional _or_ nullable _or_ both). That third option is the safest bet when it's not clear if a field is optional or not, except for lists which can usually be declared as non-null with a default value of an empty list (I have yet to see a nullable array type in the Mastodon API). Fields that are currently declared as non-null present another challenge. In theory, they should remain as-is and everything will work fine. In practice, **because Gson is not aware of nullable types at all**, it's possible that some non-null fields currently hold a null value in some cases but the app does not report any error because the field is not accessed by Kotlin code in that scenario. After migrating to Moshi however, parsing such a field will now fail early if a null value or no value is received. These fields will have to be identified by heavily testing the app and looking for parsing errors (`JsonDataException`) and/or by going through the Mastodon documentation. A default value needs to be added for missing optional fields, and their type could optionally be changed to nullable, depending on the case. Gson is also currently used to serialize and deserialize objects to and from the local database, which is also challenging because backwards compatibility needs to be preserved. Fortunately, by default Gson omits writing null fields, so a field of type `List<T>?` could be replaced with a field of type `List<T>` with a default value of `emptyList()` and reading back the old data should still work. However, nullable lists that are written directly (not as a field of another object) will still be serialized to JSON as `"null"` so the deserializing code must still be handling null properly. Finally, changing the database schema is out of scope for this pull request, so database entities that also happen to be serialized with Gson will keep their original types even if they could be made non-null as an improvement. In the end this is all for the best, because the app will be more reliable and errors will be easier to detect by showing up earlier with a clear error message. Not to mention the performance benefits of using Moshi compared to Gson. - Replace Gson reflection with Moshi Kotlin codegen to generate all parsers at compile time. - Replace custom `Rfc3339DateJsonAdapter` with the one provided by moshi-adapters. - Replace custom `JsonDeserializer` classes for Enum types with `EnumJsonAdapter.create(T).withUnknownFallback()` from moshi-adapters to support fallback values. - Replace `GuardedBooleanAdapter` with the more generic `GuardedAdapter` which works with any type. Any nullable field may now be annotated with `@Guarded`. - Remove Proguard rules related to Json entities. Each Json entity needs to be annotated with `@JsonClass` with no exception, and adding this annotation will ensure that R8/Proguard will handle the entities properly. - Replace some nullable Boolean fields with non-null Boolean fields with a default value where possible. - Replace some nullable list fields with non-null list fields with a default value of `emptyList()` where possible. - Update `TimelineDao` to perform all Json conversions internally using `Converters` so no Gson or Moshi instance has to be passed to its methods. - ~~Create a custom `DraftAttachmentJsonAdapter` to serialize and deserialize `DraftAttachment` which is a special entity that supports more than one json name per field. A custom adapter is necessary because there is not direct equivalent of `@SerializedName(alternate = [...])` in Moshi.~~ Remove alternate names for some `DraftAttachment` fields which were used as a workaround to deserialize local data in 2-years old builds of Tusky. - Update tests to make them work with Moshi. - Simplify a few `equals()` implementations. - Change a few functions to `val`s - Turn `NetworkModule` into an `object` (since it contains no abstract methods). Please test the app thoroughly before merging. There may be some fields currently declared as mandatory that are actually optional.
This commit is contained in:
Christophe Beyls
GitHub
parent
5343766886
commit
df7b11afc3
87 changed files with 767 additions and 992 deletions
|
|
@ -17,38 +17,40 @@ package com.keylesspalace.tusky.db
|
|||
|
||||
import androidx.room.ProvidedTypeConverter
|
||||
import androidx.room.TypeConverter
|
||||
import com.google.gson.Gson
|
||||
import com.google.gson.reflect.TypeToken
|
||||
import com.keylesspalace.tusky.TabData
|
||||
import com.keylesspalace.tusky.components.conversation.ConversationAccountEntity
|
||||
import com.keylesspalace.tusky.createTabDataFromId
|
||||
import com.keylesspalace.tusky.entity.Attachment
|
||||
import com.keylesspalace.tusky.entity.Card
|
||||
import com.keylesspalace.tusky.entity.Emoji
|
||||
import com.keylesspalace.tusky.entity.FilterResult
|
||||
import com.keylesspalace.tusky.entity.HashTag
|
||||
import com.keylesspalace.tusky.entity.NewPoll
|
||||
import com.keylesspalace.tusky.entity.Poll
|
||||
import com.keylesspalace.tusky.entity.Status
|
||||
import com.squareup.moshi.Moshi
|
||||
import com.squareup.moshi.adapter
|
||||
import java.net.URLDecoder
|
||||
import java.net.URLEncoder
|
||||
import java.util.Date
|
||||
import javax.inject.Inject
|
||||
import javax.inject.Singleton
|
||||
|
||||
@OptIn(ExperimentalStdlibApi::class)
|
||||
@ProvidedTypeConverter
|
||||
@Singleton
|
||||
class Converters @Inject constructor(
|
||||
private val gson: Gson
|
||||
private val moshi: Moshi
|
||||
) {
|
||||
|
||||
@TypeConverter
|
||||
fun jsonToEmojiList(emojiListJson: String?): List<Emoji>? {
|
||||
return gson.fromJson(emojiListJson, object : TypeToken<List<Emoji>>() {}.type)
|
||||
fun jsonToEmojiList(emojiListJson: String?): List<Emoji> {
|
||||
return emojiListJson?.let { moshi.adapter<List<Emoji>?>().fromJson(it) }.orEmpty()
|
||||
}
|
||||
|
||||
@TypeConverter
|
||||
fun emojiListToJson(emojiList: List<Emoji>?): String {
|
||||
return gson.toJson(emojiList)
|
||||
fun emojiListToJson(emojiList: List<Emoji>): String {
|
||||
return moshi.adapter<List<Emoji>>().toJson(emojiList)
|
||||
}
|
||||
|
||||
@TypeConverter
|
||||
|
|
@ -83,55 +85,52 @@ class Converters @Inject constructor(
|
|||
|
||||
@TypeConverter
|
||||
fun accountToJson(account: ConversationAccountEntity?): String {
|
||||
return gson.toJson(account)
|
||||
return moshi.adapter<ConversationAccountEntity?>().toJson(account)
|
||||
}
|
||||
|
||||
@TypeConverter
|
||||
fun jsonToAccount(accountJson: String?): ConversationAccountEntity? {
|
||||
return gson.fromJson(accountJson, ConversationAccountEntity::class.java)
|
||||
return accountJson?.let { moshi.adapter<ConversationAccountEntity?>().fromJson(it) }
|
||||
}
|
||||
|
||||
@TypeConverter
|
||||
fun accountListToJson(accountList: List<ConversationAccountEntity>?): String {
|
||||
return gson.toJson(accountList)
|
||||
fun accountListToJson(accountList: List<ConversationAccountEntity>): String {
|
||||
return moshi.adapter<List<ConversationAccountEntity>>().toJson(accountList)
|
||||
}
|
||||
|
||||
@TypeConverter
|
||||
fun jsonToAccountList(accountListJson: String?): List<ConversationAccountEntity>? {
|
||||
return gson.fromJson(
|
||||
accountListJson,
|
||||
object : TypeToken<List<ConversationAccountEntity>>() {}.type
|
||||
)
|
||||
fun jsonToAccountList(accountListJson: String?): List<ConversationAccountEntity> {
|
||||
return accountListJson?.let { moshi.adapter<List<ConversationAccountEntity>?>().fromJson(it) }.orEmpty()
|
||||
}
|
||||
|
||||
@TypeConverter
|
||||
fun attachmentListToJson(attachmentList: List<Attachment>?): String {
|
||||
return gson.toJson(attachmentList)
|
||||
fun attachmentListToJson(attachmentList: List<Attachment>): String {
|
||||
return moshi.adapter<List<Attachment>>().toJson(attachmentList)
|
||||
}
|
||||
|
||||
@TypeConverter
|
||||
fun jsonToAttachmentList(attachmentListJson: String?): List<Attachment>? {
|
||||
return gson.fromJson(attachmentListJson, object : TypeToken<List<Attachment>>() {}.type)
|
||||
fun jsonToAttachmentList(attachmentListJson: String?): List<Attachment> {
|
||||
return attachmentListJson?.let { moshi.adapter<List<Attachment>?>().fromJson(it) }.orEmpty()
|
||||
}
|
||||
|
||||
@TypeConverter
|
||||
fun mentionListToJson(mentionArray: List<Status.Mention>?): String? {
|
||||
return gson.toJson(mentionArray)
|
||||
fun mentionListToJson(mentionArray: List<Status.Mention>): String {
|
||||
return moshi.adapter<List<Status.Mention>>().toJson(mentionArray)
|
||||
}
|
||||
|
||||
@TypeConverter
|
||||
fun jsonToMentionArray(mentionListJson: String?): List<Status.Mention>? {
|
||||
return gson.fromJson(mentionListJson, object : TypeToken<List<Status.Mention>>() {}.type)
|
||||
fun jsonToMentionArray(mentionListJson: String?): List<Status.Mention> {
|
||||
return mentionListJson?.let { moshi.adapter<List<Status.Mention>?>().fromJson(it) }.orEmpty()
|
||||
}
|
||||
|
||||
@TypeConverter
|
||||
fun tagListToJson(tagArray: List<HashTag>?): String? {
|
||||
return gson.toJson(tagArray)
|
||||
fun tagListToJson(tagArray: List<HashTag>?): String {
|
||||
return moshi.adapter<List<HashTag>?>().toJson(tagArray)
|
||||
}
|
||||
|
||||
@TypeConverter
|
||||
fun jsonToTagArray(tagListJson: String?): List<HashTag>? {
|
||||
return gson.fromJson(tagListJson, object : TypeToken<List<HashTag>>() {}.type)
|
||||
return tagListJson?.let { moshi.adapter<List<HashTag>?>().fromJson(it) }
|
||||
}
|
||||
|
||||
@TypeConverter
|
||||
|
|
@ -145,45 +144,47 @@ class Converters @Inject constructor(
|
|||
}
|
||||
|
||||
@TypeConverter
|
||||
fun pollToJson(poll: Poll?): String? {
|
||||
return gson.toJson(poll)
|
||||
fun pollToJson(poll: Poll?): String {
|
||||
return moshi.adapter<Poll?>().toJson(poll)
|
||||
}
|
||||
|
||||
@TypeConverter
|
||||
fun jsonToPoll(pollJson: String?): Poll? {
|
||||
return gson.fromJson(pollJson, Poll::class.java)
|
||||
return pollJson?.let { moshi.adapter<Poll?>().fromJson(it) }
|
||||
}
|
||||
|
||||
@TypeConverter
|
||||
fun newPollToJson(newPoll: NewPoll?): String? {
|
||||
return gson.toJson(newPoll)
|
||||
fun newPollToJson(newPoll: NewPoll?): String {
|
||||
return moshi.adapter<NewPoll?>().toJson(newPoll)
|
||||
}
|
||||
|
||||
@TypeConverter
|
||||
fun jsonToNewPoll(newPollJson: String?): NewPoll? {
|
||||
return gson.fromJson(newPollJson, NewPoll::class.java)
|
||||
return newPollJson?.let { moshi.adapter<NewPoll?>().fromJson(it) }
|
||||
}
|
||||
|
||||
@TypeConverter
|
||||
fun draftAttachmentListToJson(draftAttachments: List<DraftAttachment>?): String? {
|
||||
return gson.toJson(draftAttachments)
|
||||
fun draftAttachmentListToJson(draftAttachments: List<DraftAttachment>): String {
|
||||
return moshi.adapter<List<DraftAttachment>>().toJson(draftAttachments)
|
||||
}
|
||||
|
||||
@TypeConverter
|
||||
fun jsonToDraftAttachmentList(draftAttachmentListJson: String?): List<DraftAttachment>? {
|
||||
return gson.fromJson(
|
||||
draftAttachmentListJson,
|
||||
object : TypeToken<List<DraftAttachment>>() {}.type
|
||||
)
|
||||
fun jsonToDraftAttachmentList(draftAttachmentListJson: String?): List<DraftAttachment> {
|
||||
return draftAttachmentListJson?.let { moshi.adapter<List<DraftAttachment>?>().fromJson(it) }.orEmpty()
|
||||
}
|
||||
|
||||
@TypeConverter
|
||||
fun filterResultListToJson(filterResults: List<FilterResult>?): String? {
|
||||
return gson.toJson(filterResults)
|
||||
fun filterResultListToJson(filterResults: List<FilterResult>?): String {
|
||||
return moshi.adapter<List<FilterResult>?>().toJson(filterResults)
|
||||
}
|
||||
|
||||
@TypeConverter
|
||||
fun jsonToFilterResultList(filterResultListJson: String?): List<FilterResult>? {
|
||||
return gson.fromJson(filterResultListJson, object : TypeToken<List<FilterResult>>() {}.type)
|
||||
return filterResultListJson?.let { moshi.adapter<List<FilterResult>?>().fromJson(it) }
|
||||
}
|
||||
|
||||
@TypeConverter
|
||||
fun cardToJson(card: Card?): String {
|
||||
return moshi.adapter<Card?>().toJson(card)
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -21,10 +21,10 @@ import androidx.core.net.toUri
|
|||
import androidx.room.Entity
|
||||
import androidx.room.PrimaryKey
|
||||
import androidx.room.TypeConverters
|
||||
import com.google.gson.annotations.SerializedName
|
||||
import com.keylesspalace.tusky.entity.Attachment
|
||||
import com.keylesspalace.tusky.entity.NewPoll
|
||||
import com.keylesspalace.tusky.entity.Status
|
||||
import com.squareup.moshi.JsonClass
|
||||
import kotlinx.parcelize.Parcelize
|
||||
|
||||
@Entity
|
||||
|
|
@ -46,21 +46,18 @@ data class DraftEntity(
|
|||
val statusId: String?
|
||||
)
|
||||
|
||||
/**
|
||||
* The alternate names are here because we accidentally published versions were DraftAttachment was minified
|
||||
* Tusky 15: uriString = e, description = f, type = g
|
||||
* Tusky 16 beta: uriString = i, description = j, type = k
|
||||
*/
|
||||
@JsonClass(generateAdapter = true)
|
||||
@Parcelize
|
||||
data class DraftAttachment(
|
||||
@SerializedName(value = "uriString", alternate = ["e", "i"]) val uriString: String,
|
||||
@SerializedName(value = "description", alternate = ["f", "j"]) val description: String?,
|
||||
@SerializedName(value = "focus") val focus: Attachment.Focus?,
|
||||
@SerializedName(value = "type", alternate = ["g", "k"]) val type: Type
|
||||
val uriString: String,
|
||||
val description: String?,
|
||||
val focus: Attachment.Focus?,
|
||||
val type: Type
|
||||
) : Parcelable {
|
||||
val uri: Uri
|
||||
get() = uriString.toUri()
|
||||
|
||||
@JsonClass(generateAdapter = false)
|
||||
enum class Type {
|
||||
IMAGE,
|
||||
VIDEO,
|
||||
|
|
|
|||
|
|
@ -21,7 +21,11 @@ import androidx.room.Insert
|
|||
import androidx.room.OnConflictStrategy.Companion.REPLACE
|
||||
import androidx.room.Query
|
||||
import androidx.room.TypeConverters
|
||||
import com.google.gson.Gson
|
||||
import com.keylesspalace.tusky.entity.Attachment
|
||||
import com.keylesspalace.tusky.entity.Card
|
||||
import com.keylesspalace.tusky.entity.Emoji
|
||||
import com.keylesspalace.tusky.entity.HashTag
|
||||
import com.keylesspalace.tusky.entity.Poll
|
||||
import com.keylesspalace.tusky.entity.Status
|
||||
|
||||
@Dao
|
||||
|
|
@ -89,13 +93,13 @@ AND
|
|||
)
|
||||
abstract suspend fun deleteRange(accountId: Long, minId: String, maxId: String): Int
|
||||
|
||||
suspend fun update(accountId: Long, status: Status, gson: Gson) {
|
||||
suspend fun update(accountId: Long, status: Status) {
|
||||
update(
|
||||
accountId = accountId,
|
||||
statusId = status.id,
|
||||
content = status.content,
|
||||
editedAt = status.editedAt?.time,
|
||||
emojis = gson.toJson(status.emojis),
|
||||
emojis = status.emojis,
|
||||
reblogsCount = status.reblogsCount,
|
||||
favouritesCount = status.favouritesCount,
|
||||
repliesCount = status.repliesCount,
|
||||
|
|
@ -105,13 +109,13 @@ AND
|
|||
sensitive = status.sensitive,
|
||||
spoilerText = status.spoilerText,
|
||||
visibility = status.visibility,
|
||||
attachments = gson.toJson(status.attachments),
|
||||
mentions = gson.toJson(status.mentions),
|
||||
tags = gson.toJson(status.tags),
|
||||
poll = gson.toJson(status.poll),
|
||||
attachments = status.attachments,
|
||||
mentions = status.mentions,
|
||||
tags = status.tags,
|
||||
poll = status.poll,
|
||||
muted = status.muted,
|
||||
pinned = status.pinned ?: false,
|
||||
card = gson.toJson(status.card),
|
||||
pinned = status.pinned,
|
||||
card = status.card,
|
||||
language = status.language
|
||||
)
|
||||
}
|
||||
|
|
@ -141,12 +145,12 @@ AND
|
|||
WHERE timelineUserId = :accountId AND (serverId = :statusId OR reblogServerId = :statusId)"""
|
||||
)
|
||||
@TypeConverters(Converters::class)
|
||||
abstract suspend fun update(
|
||||
protected abstract suspend fun update(
|
||||
accountId: Long,
|
||||
statusId: String,
|
||||
content: String?,
|
||||
editedAt: Long?,
|
||||
emojis: String?,
|
||||
emojis: List<Emoji>,
|
||||
reblogsCount: Int,
|
||||
favouritesCount: Int,
|
||||
repliesCount: Int,
|
||||
|
|
@ -156,13 +160,13 @@ AND
|
|||
sensitive: Boolean,
|
||||
spoilerText: String,
|
||||
visibility: Status.Visibility,
|
||||
attachments: String?,
|
||||
mentions: String?,
|
||||
tags: String?,
|
||||
poll: String?,
|
||||
attachments: List<Attachment>,
|
||||
mentions: List<Status.Mention>,
|
||||
tags: List<HashTag>?,
|
||||
poll: Poll?,
|
||||
muted: Boolean?,
|
||||
pinned: Boolean,
|
||||
card: String?,
|
||||
card: Card?,
|
||||
language: String?
|
||||
)
|
||||
|
||||
|
|
@ -243,7 +247,8 @@ AND serverId = :statusId"""
|
|||
"""UPDATE TimelineStatusEntity SET poll = :poll
|
||||
WHERE timelineUserId = :accountId AND (serverId = :statusId OR reblogServerId = :statusId)"""
|
||||
)
|
||||
abstract suspend fun setVoted(accountId: Long, statusId: String, poll: String)
|
||||
@TypeConverters(Converters::class)
|
||||
abstract suspend fun setVoted(accountId: Long, statusId: String, poll: Poll)
|
||||
|
||||
@Query(
|
||||
"""UPDATE TimelineStatusEntity SET expanded = :expanded
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue