如何加速从 Django 中的大型数据库迁移数据

Question

我尝试从我的 Django 1.7 应用程序中删除模型继承

models.py

class Element(models.Model):
    ELEMENT_A = 'ELEMENT_A'
    ELEMENT_B = 'ELEMENT_B'
    TYPES = (
        (ELEMENT_A, 'Element A'),
        (ELEMENT_B, 'Element B')
    )
    number = models.CharField(max_length=255)
    type = models.CharField(max_length=10, choices=TYPES, default=ELEMENT_A)
    quantity = models.IntegerField(default=1)

class ChildElement(Element):
    old_number = models.CharField(max_length=30, unique=True)
    old_quantity = models.IntegerField()

我的迁移文件

def forwards_remove_heritance(apps, schema_editor):
    childs = apps.get_model("core", "ChildElement")
    type = Element.ELEMENT_A
    for c in childs.objects.all():
        c.number = c.old_number
        c.quantity = c.old_quantity
        c.type = type
        c.save()
    return


def backward_remove_heritance(apps, schema_editor):
    return


class Migration(migrations.Migration):
    dependencies = [

    ]

    operations = [
        migrations.RunPython(
            forwards_remove_heritance, backward_remove_heritance
        ),
    ]

我的迁移需要几个小时，我在 app_ChildElement table

中有超过 750k 个元素

有什么想法吗？

Answer 1

将查询集 update 与 F expressions 结合使用，示例：

ChildElement.objects.all().update(number=F('old_number'))

Answer 2

我找到的解决方案是在操作

中使用原始 SQL 直接更新我的数据库

class RemoveChild(Operation):
    reversible = True

    def __init__(self):
        pass

    def state_forwards(self, app_label, state):
        pass

    def database_forwards(self, app_label, schema_editor, from_state, to_state):
        schema_editor.execute("""update Element
SET Element.number = ChildElement.old_number, Element.quantity=ChildElement.old_quantity, Element.type='ELEMENT_B'
FROM Element
INNER JOIN ChildElement
ON ChildElement.element_ptr_id = Element.id;
""")

    def database_backwards(self, app_label, schema_editor, from_state, to_state):
        pass

    def describe(self):
        return "Remove child heritance model"

如何加速从 Django 中的大型数据库迁移数据

how can I accelerate data migration from a large database in django

django

performance

django-models