Upgrading to Mezzanine 4

This post assumes your project is currently using Mezzanine 3.1.10 and up to date with South migrations.

To get started install Mezzanine 4 into a fresh virtualenv and create a Mezzanine 4 project.

$ virtualenv MEZ4
$ source MEZ4/bin/active
$ pip install mezzanine
$ mezzanine-project mez4proj

We will refer to mez4proj throughout the process of upgrading your project to Mezzanine 4.

As I write this blog post I'm updating Adept to support Mezzanine 4 so I may refer to the project being updated as adept. Right now adept has the old project structure and looks like this:

__init__.py
deploy
    crontab
    gunicorn.conf.py
    live_settings.py
    nginx.conf
    supervisor.conf
dev.db
fabfile.py
local_settings.py
manage.py
settings.py
static
templates
theme
    __init__.py
    admin.py
    blog_mods
        __init__.py
        admin.py
        migrations
            # migration files
        models.py
    defaults.py
    migrations
        # migration files
    models.py
    page_processors.py
    portfolio
        __init__.py
        admin.py
        migrations
            # migration files
        models.py
        page_processors.py
        templates
    static
        # static files
    templates
        # template files
    templatetags
        __init__.py
        adept_tags.py
urls.py
wsgi.py

There is a single top level app, theme which contains two sub apps blog_mods and portfolio. Let's get started!

Update your project's layout

Create a folder in your project with the same name as the project, move everything into it except your static directory and database (if using SQLite). Add an empty file called __init__.py to your top level project folder
```
$ touch __init__.py
```
Within the new folder delete deploy, manage.py, wsgi.py and fabfile.py

If you have made changes to any of the files you are instructed to delete above, do not delete them! Instead move them into the correct place. I've decided to delete them to upgrade to the latest and greatest from Mezzanine.
Copy deploy, manage.py, and fabfile.py from mez4proj to the top level of your project. Copy wsgi.py from mez4proj/mez4proj to the newly created folder.
Move any apps located in your project up to the top level, in my case I move theme from the newly created adept/adpet up one directory to reside in the top level adept directory.

If you are using SQLite also move your database file to the top level. The top level of adept now looks like:

$ ls
__init__.py      deploy           fabfile.py       requirements.txt
adept            dev.db           manage.py        theme

Update files

manage.py and wsgi.py

Open the copied manage.py and wsgi.py and replace any references to mez4proj with the correct name of your project.

settings.py

At this point things get a bit dicey. It's likely that you have made many modifications to your settings.py file but there have also been changes to the settings.py file that comes with Mezzanine. Personally I want the settings.py that ships with adept to be as close to the one that comes with Mezzanine as possible. Here are the steps I use to merge them.

Rename settings.py to settings_old.py

mv adept/settings.py adept/settings_old.py

Copy settings.py from mez4proj/mez4proj to adept/adept
Open the newly copied settings.py and settings_old.py in a text editor and copy over any needed changes from settings_old.py to settings.py

In my case the changes I made to the newly copied settings.py included:

Copy ADMIN_MENU_ORDER from settings_old.py
Copy PAGE_MENU_TEMPLATES from settings_old.py
Copy EXTRA_MODEL_FIELDS from settings_old.py
uncomment BLOG_USE_FEATURED_IMAGE = True
Update INSTALLED_APPS with apps I had in settings_old.py

Do not copy USE_SOUTH = True, we will be updating to Django migrations

urls.py

Merge your project's urls.py and urls.py from mez4proj. You can use a similar process to what we did above with settings.py or just eyeball it. My urls.py files tend to be smaller and easier to merge than settings.py.

Migrations

As of Django 1.7 migrations are a core feature of Django. South is a thing of the past and since Mezzanine 4 requires Django 1.7+ we will need Django migrations. These are the Django docs on upgrading from South, I will borrow from them heavily.

The Django docs tell you to delete all your current migrations. You can do that but I prefer to rename the current migration folders from migrations to south_migrations. If you prefer to do exactly as the Django docs say read them, delete all numbered migrations files from your migrations folder[s] and skip to step 3

Create Django Migrations

If you use EXTRA_MODEL_FIELDS make sure to check out the section with that title below before starting these steps

Rename migrations to south_migrations
Everywhere you now have a south_migrations directory create a new directory called migrations and add an empty __init__.py file to it.
At this point it is important that you have migrations folders that are empty except for an __init__.py, i.e. they look like this:
```
migrations
    __init__.py
```
Make Django migrations:
```
$ python manage.py makemigrations
```
Fake the migrations since your database is already up to date
```
$ python manage.py migrate --fake-initial
```

EXTRA_MODEL_FIELDS

Using south, it was pretty easy to have arbitrary migrations for one app in another app. This made making migrations for EXTRA_MODEL_FIELDS fairly straightforward. Django migrations are not flexible in the same way and expect all migrations for an app to be in one place. The following is how I've gotten EXTRA_MODEL_FIELDS to work with Django migrations. It's ugly, hacky and brittle. If you know of a better way please let me know in the comments below!

Steps

Prior to doing the "Create Django Migrations" steps above comment out EXTRA_MODEL_FIELDS in your project's settings.py.
Do the migration steps above except do not complete step 5 yet.
Uncomment EXTRA_MODEL_FIELDS
Run
```
$ python manage.py makemigrations [APPNAME] --dry-run --verbosity 3
```
where [APPNAME] is the name of the app your EXTRA_MODEL_FIELDS modifies. You will need to repeat this for each app EXTRA_MODEL_FIELDS modifies

In my case EXTRA_MODEL_FIELDS adds a field to BlogPost and I get the following output:

$ python manage.py makemigrations blog --dry-run --verbosity 3

Migrations for 'blog':
  0003_blogpost_featured_video.py:
    - Add field featured_video to blogpost
Full migrations file '0003_blogpost_featured_video.py':
# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import models, migrations

class Migration(migrations.Migration):

    dependencies = [
        ('blog', '0002_auto_20150527_1555'),
    ]

    operations = [
        migrations.AddField(
            model_name='blogpost',
            name='featured_video',
            field=models.TextField(help_text='Optional, an iframe here will override any featured image above', verbose_name='Featured video', blank=True),
        ),
    ]

Copy everything starting with # -*- coding: utf-8 -*- and paste it into a new file in a migrations folder in one of your apps. In the case of adept I create adept/theme/blog_modes/migrations/0002_blogpost_featured_video.py

Django doesn't actually care about the numbers at the beginning of migrations, those are just to make it easier for humans to tell the order of migrations.

Add the following to the top, just under the imports, of your new migrations file

class AddExtraField(migrations.AddField):

    def __init__(self, *args, **kwargs):
        if 'app_label' in kwargs:
            self.app_label = kwargs.pop('app_label')
        else:
            self.app_label = None
        super(AddExtraField, self).__init__(*args, **kwargs)

    def state_forwards(self, app_label, state):
        super(AddExtraField, self).state_forwards(self.app_label or app_label, state)

    def database_forwards(self, app_label, schema_editor, from_state, to_state):
        super(AddExtraField, self).database_forwards(
            self.app_label or app_label, schema_editor, from_state, to_state)

    def database_backwards(self, app_label, schema_editor, from_state, to_state):
        super(AddExtraField, self).database_backwards(
            self.app_label or app_label, schema_editor, from_state, to_state)

Change instances of migrations.AddField to AddExtraField.
To the end of any AddExtraField call add a new kwarg, app_label and set it equal to the string of the app this migration is modifying.

In my case the migration file ends up looking like this:

# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from django.db import models, migrations

class AddExtraField(migrations.AddField):

    def __init__(self, *args, **kwargs):
        if 'app_label' in kwargs:
            self.app_label = kwargs.pop('app_label')
        else:
            self.app_label = None
        super(AddExtraField, self).__init__(*args, **kwargs)

    def state_forwards(self, app_label, state):
        super(AddExtraField, self).state_forwards(self.app_label or app_label, state)

    def database_forwards(self, app_label, schema_editor, from_state, to_state):
        super(AddExtraField, self).database_forwards(
            self.app_label or app_label, schema_editor, from_state, to_state)

    def database_backwards(self, app_label, schema_editor, from_state, to_state):
        super(AddExtraField, self).database_backwards(
            self.app_label or app_label, schema_editor, from_state, to_state)

class Migration(migrations.Migration):

    dependencies = [
        ('blog', '0002_auto_20150527_1555'),
    ]

    operations = [
        AddExtraField(
            model_name='blogpost',
            name='featured_video',
            field=models.TextField(help_text='Optional, an iframe here will override any featured image above', verbose_name='Featured video', blank=True),
            app_label="blog"
        ),
    ]

All it does is add a featured_video field to blog posts.

Fake all migration:

$ python manage.py migrate --fake-initial

Closing

Run your project, it should work! At this point you can delete settings_old.py from your project. You may want to keep a copy around somewhere to refer to just in case.

The layout of the upgraded adept now looks like this:

__init__.py
adept
    __init__.py
    local_settings.py
    settings.py
    urls.py
    wsgi.py
deploy
    crontab
    gunicorn.conf.py
    live_settings.py
    nginx.conf
    supervisor.conf
dev.db
fabfile.py
manage.py
static
theme
    __init__.py
    admin.py
    blog_mods
        __init__.py
        admin.py
        migrations
            # migration files
        models.py
    defaults.py
    migrations
        # migration files
    models.py
    page_processors.py
    portfolio
        __init__.py
        admin.py
        migrations
            # migration files
        models.py
        page_processors.py
        templates
    static
        # static files
    templates
        # template files
    templatetags
        __init__.py
        adept_tags.py

Gotchas

Certain types of monkey patches can no longer be in models.py so if you run into errors complaining of models not being loaded, that may be it.