Dark as of today (4/18/2020) does not support changing Datastore schemas. Therefore, you need to do a migration if you want to make schema changes.

Migrations in Dark are similar to migrations in traditional backend systems. The following steps outline a double-write strategy, which allows you to move data from Datastore A → B without taking down the service or losing data. If these are not concerns, you can use a REPL to move data from A → B and incrementally update callsites. However, you may want to use the following strategy regardless to have a less stressful migration 😬.

We'll use the simple example of adding an email field to a Users Datastore.

Double Write

• Create the new Datastore. We'll call it Users2. Our goal is to end up in a place where we are running on Users2 and Users can be deleted. Note that Datastores cannot be renamed after they hold data (the next step).

  

• Update all your DB::set Users callsites to also write to Users2.

  

  Note that if your newer schema contains data not in the older schema (in this example the email field), you'll need to retrieve the value from the new Datastore before updating it.

  If you have a lot of DB::set callsites, you may consider extracting logic into a function like updateUser which performs both DB::set calls.

• Once all your DB::set callsites are writing to both Datastores, we're ready to do the big migration. Create a REPL that iterates through Users and writes them to Users2.

  

  At this point, both your Datastores should be in a usable state. Now's a good time to use a REPL to double check that both Datastores contain what you expect.

Redirect reads

• Update your readers to read from the new Datastore. This is a good place to use feature flags!

  

• Confirm that your APIs are still working as expected. In this example, we'd expect the email field to now show up in the updated endpoint. If everything is going well, we can begin our cleanup.

  You may want to stay in this state for a little bit to build confidence. Once we continue, we can't easily undo the previous steps because it will be missing data.

Cleanup

We want to get a state where the old Datastore has no more dependents.

• Remove the writes to the old Datastore.

  

• Once your old Datastore has no more dependents, you can clear it with a REPL.

  

• Last but not least, delete your old Datastore!

  

Some migrations may be trickier, such as ones that migrate multiple Datastores at the same time. However, the same mental model applies of getting to a state where you have two usable up-to-date Datastores and then flipping the switch to read from the new Datastore.