Delta Features - Syncing
- 9 minsJust like GBA4iOS, Delta will support syncing data between all your devices, so you will be able to resume any progress made on another device completely seamlessly.
However, this solution has some notable differences from GBA4iOS’ Dropbox Sync, mostly due to feedback I’ve heard from everyone who had used that feature. I’ve divided this post into two main sections, user-facing features and the actual implementation, so you can read only the parts you are most interested in.
So, let’s dive in to what exactly Delta’s syncing functionality will be like!
User-Facing Features
Fundamentally, the system is very similar to the one used in GBA4iOS. Whenever you save your game or add/delete save states or cheats, the data will by synced to your other devices in the background. The goal is for this process to be fully automatic, but allow for some control when necessary.
Dropbox and Google Drive as Backends
GBA4iOS used Dropbox for all of its syncing functionality. While this made sense at the time, Dropbox has become increasingly less popular, and almost everyone has a Google account. As a result, Delta will fully support both Dropbox and Google Drive. This allows users to choose the option that makes the most sense for them, which is important since this is a feature I think everyone should have enabled.
Why, however, am I not supporting the obvious syncing backend used by practically every other iOS app: iCloud? Unfortunately, it’s because Delta is not just a normal app. iCloud is officially only supported for apps that are distributed through the App Store, and while there are ways around this, ultimately Apple has the power to disable iCloud support for any app they want very easily.
Maybe Apple will relax their policies on iCloud usage in the future, but for now I consider this far too risky, especially since we’re dealing with game data that would be disastrous to lose, so I do not think it is worth the risk.
Syncing ROMs
In GBA4iOS, I made the decision to sync all game data except the ROMs themselves. This was because Dropbox’s free tier was rather small at 2GB, and I didn’t want to risk filling up users’ storage quota with ROMs that could just be downloaded individually onto each device.
Unfortunately, in practice this was rather confusing. People wondered why after enabling syncing on one device their games didn’t sync to their other devices, and no matter how many times I answered the question, it was clear it just wasn’t intuitive why this was the case. Additionally, Google Drive comes with 15GB for the free tier, which I feel is far more than enough for the added storage taken up by ROMs.
So, for Delta we will be syncing the ROMs themselves, which I do believe is the better default. I’m debating whether this will be an option for users to disable, but am leaning towards no (at least for version 1.0). That being said, if you strongly prefer it to be an option, please let me know in the comments.
File Versions
A feature I ended up cutting from GBA4iOS was the ability to view versions of synced data (such as game saves), since it was possible to access this information from the Dropbox website directly if people wanted to.
However, as people used Dropbox Sync, I saw it being used more and more as a backup system for their data rather than just for syncing, which I think is great! But, any backup system should be able to show users past versions of their files, especially if they resolved a conflict by accidentally selecting the wrong version of the file.
Since this functionality is already built into Dropbox and Google Drive, I am planning on simply having a way to hook into their implementations directly from the app. Most importantly, this should bring more users peace about their save data, since they know they can always restore a previous save file if they mess something up themselves.
Safety First
Just like with GBA4iOS, data integrity is the #1 priority for syncing. As a result, Delta will defer as much as possible to the user for decisions when it is not 100% sure what the right answer is. While this means there may be more “Resolve Conflict” screens than similar syncing approaches, I believe this is essential to ensure no data is lost.
I’m very happy that in the 4 years since GBA4iOS, I have yet to hear a story about how someone lost their data due to a bug in the syncing logic. I truly believe this is because of the very conservative approach taken by GBA4iOS when making conflict resolution decisions, and I aim for Delta to be just as reliable as GBA4iOS.
Implementation - Harmony
The syncing functionality of GBA4iOS was always tied directly to Dropbox, and as a result the syncing code itself is tightly coupled with the Dropbox SDK. This made sense due to the time constraints and my limited expertise of syncing logic in general, but it now means I can use practically none of the same code as before to build the syncing functionality for Delta.
Rather than follow the same approach and closely tying the logic with Dropbox and Google Drive directly, I’ve opted instead to separate the logic into it’s own framework I’ve named Harmony, with support for arbitrary backends. This results in much more modular (and testable!) code, and I’ll be able to use it in any future projects I work on without having to do this all over again.
Core Data-Based
Delta uses Core Data to store its information locally, and I’ve been extremely happy with this; I cannot express how wonderful and powerful Core Data is. So, when designing Harmony, it made sense to make sure it integrated well with Core Data, so Delta and any other Core Data-based apps could easily adopt it.
Additionally, Core Data provides practically everything necessary for a syncing framework. It’s easy to know when anything is added/deleted/modified, and since apps using Core Data typically react to these updates automatically (using NSFetchedResultsController, for example), there really is almost no additional work for a Core Data app to support syncing via Harmony.
Harmony itself will have it’s own Core Data stack that will keep track of the status of both local and remote data. Additionally, it will have access to the app’s own Core Data stack, will is used to retrieve and update local data. This is important, because this means the app can be blissfully unaware of Harmony, and continue to work as if it wasn’t there.
Support for Files
While most apps that use Core Data typically store all information in the local database file itself, Delta is slightly different. Almost everything synced between devices is actually one (or more) files on disk, and Core Data is used to store metadata about that file. For example, when you create a save state, Delta saves a thumbnail and a save state generated by the emulation core to disk, and then creates a SaveState (custom NSManagedObject subclass) with relevant metadata such as the name, creation date, and whether the save state is “locked” or not.
Most syncing solutions based on database persistence treat the database entries as the only data necessary to sync, but since for Delta the entries are only the metadata of the files, Harmony allows you to associate a Record with any number of File objects, and will ensure they are synced together as one “unit” to ensure data integrity.
Fully Automatic
As mentioned before, Core Data apps typically use a reactive approach to display data; that is, rather than manually updating the UI, they instead listen for notifications sent out by the Core Data framework, and update the UI whenever they receive these notifications.
As it turns out, Harmony is able to use these very same notifications to determine when local data has been changed. Whenever an NSManagedObject is inserted, updated, or deleted, Harmony will receive a notification and update it’s private database with this information. When Harmony performs its next sync, it can easily query the state of all local objects and compare them with the state of remote objects to determine what operations are necessary to get everything in sync.
Additionally, when Harmony updates the app’s own database, it makes sure that the appropriate notifications are sent out. This way, when data is downloaded from the server, the app will react the same way as if it had saved the data itself, with 0 additional code changes. Magic!
Backend-Agnostic
Harmony was designed to be flexible and support any potential backend. To accomplish this, as much syncing logic as possible is kept separate from the notion of backends; Harmony knows how to serialize Core Data entities to and from syncable representations, determine exactly what operations are necessary to keep datasets in sync, and handle any conflicts as they arise.
Ultimately, it does need to connect to specific backends, so there is a very simple Service protocol that backend-implementations can conform to. This protocol is limited to very primitive functionality, such as authentication, uploading/download files, and fetching all changed remote files. For now, there are two concrete implementations of this Service protocol, DropboxService and DriveService, which handle the backend-specific logic for Dropbox and Google Drive, respectively.
Current State of Syncing
The majority of the core Harmony logic has been written, and I am currently working on adding the backend-specific logic. Once I’ve finished this, I can then incorporate the framework into Delta, and begin testing it.
Once the framework is incorporated in Delta, I will begin working on the necessary UI for users to interact with it. I expect this to take a couple weeks, but since it’s mostly writing code that calls through to the already written syncing code, I expect it will be fairly straightforward.
The goal is for this to seem extremely simple to the end-user, and so the less complicated it seems, the better I’ll feel I’ve done. However, this will undergo a lot of beta testing, so anything listed above may change as I respond to feedback. Ultimately though, Delta syncing should do exactly what’s expected from users and never lose data, and I won’t release Delta until I am positive those expectations hold true.
Riley Testut
Independent iOS developer, USC student, and Apple fan.