Django MySql 全文搜索有效但在测试中无效
Django MySql Fulltext search works but not on tests
我使用这个 SO question 在 Django 应用程序的 mysql 数据库上启用全文搜索。
# models.py
class CaseSnapshot(BaseModel):
text = models.TextField(blank=True)
class Search(models.Lookup):
lookup_name = "search"
def as_mysql(self, compiler, connection):
lhs, lhs_params = self.process_lhs(compiler, connection)
rhs, rhs_params = self.process_rhs(compiler, connection)
params = lhs_params + rhs_params
return f"MATCH (%s) AGAINST (%s IN BOOLEAN MODE)" % (lhs, rhs), params
models.TextField.register_lookup(Search)
因为Django在mysql上不支持全文搜索,所以我们要添加我们的索引,所以我们修改生成的迁移文件:
# Generated by Django 2.2 on 2020-04-28 03:41
from django.db import migrations, models
# Table details
table_name = "by_api_casesnapshot"
field_name = "text"
index_name = f"{table_name}_{field_name}_index"
class Migration(migrations.Migration):
dependencies = [("by_api", "0033_add_tag_color")]
operations = [
migrations.CreateModel(...), # As auto-generated
migrations.RunSQL(
f"CREATE FULLTEXT INDEX {index_name} ON {table_name} ({field_name})",
f"DROP INDEX {index_name} ON {table_name}",
),
]
检查作业
# dbshell
mysql> SHOW INDEX FROM by_api_casesnapshot;
+---------------------+--------------------------------+-------------+-----+------------+-----+
| Table | Key_name | Column_name | ... | Index_type | ... |
+---------------------+--------------------------------+-------------+-----+------------+-----+
| by_api_casesnapshot | PRIMARY | id | ... | BTREE | ... |
| by_api_casesnapshot | by_api_casesnapshot_text_index | text | ... | FULLTEXT | ... |
+---------------------+--------------------------------+-------------+-----+------------+-----+
#shell
>>> snap = CaseSnapshot.objects.create(text="XXX")
# Regular query first
>>> CaseSnapshot.objects.filter(text__contains="X")
<QuerySet [<CaseSnapshot pk=1>]>
# Now test custom search lookup
>>> CaseSnapshot.objects.filter(text__search="XXX")
<QuerySet [<CaseSnapshot pk=1>]>
>>> CaseSnapshot.objects.filter(text__search="X*")
<QuerySet [<CaseSnapshot pk=1>]>
>>> CaseSnapshot.objects.filter(text__search="X*").query.__str__()
'SELECT `by_api_casesnapshot`.`id`, `by_api_casesnapshot`.`text` FROM `by_api_casesnapshot` WHERE MATCH (`by_api_casesnapshot`.`text`) AGAINST (X* IN BOOLEAN M
ODE)'
到目前为止,一切都完全按照预期工作。
问题
这是失败的地方 - 所有查询都使用 __search
查找 在测试期间 运行s return 查询集空集,甚至认为生成的查询是相同的。
测试
from django.test import TestCase
from by_api.models import CaseSnapshot
class TestModels(TestCase):
def test_search(self):
CaseSnapshot.objects.create(text="XXX")
rv1 = CaseSnapshot.objects.filter(text__contains="XXX")
# Passes
self.assertEqual(rv1.count(), 1)
rv2 = CaseSnapshot.objects.filter(text__search="XXX")
# Fails - count i
self.assertEqual(rv2.count(), 1)
我尝试在测试 运行 中使用 pdb 对其进行调试,并注意生成的查询与上面在 shell 环境中生成的查询相同:
>>> rv2.query.__str__())
'SELECT `by_api_casesnapshot`.`id`, `by_api_casesnapshot`.`text` FROM `by_api_casesnapshot` WHERE MATCH (`by_api_casesnapshot`.`text`) AGAINST (XXX IN BOOLEAN MODE)'
问题
为什么在测试期间使用 __search
查找会产生空查询集?
我试过的
- 调查了
- 确保测试数据库和 shell 数据库具有相同的值
- 尝试使用游标执行生成的查询 - 也是空的
更新
我刚刚注意到如果我使用 unittest.TestCase
而不是 django.test.TestCase
测试通过 - 这是否与 django 的测试用例如何包装数据库事务有关?
来自 django-mysql 库的 Adam Chainz 的回答:
InnoDB doesn't make changes to full text indexes until commits. You can use TransactionTestCase to get around this.
我将测试 class 更改为使用 TransactionTestCase
,测试现在通过了。
我使用这个 SO question 在 Django 应用程序的 mysql 数据库上启用全文搜索。
# models.py
class CaseSnapshot(BaseModel):
text = models.TextField(blank=True)
class Search(models.Lookup):
lookup_name = "search"
def as_mysql(self, compiler, connection):
lhs, lhs_params = self.process_lhs(compiler, connection)
rhs, rhs_params = self.process_rhs(compiler, connection)
params = lhs_params + rhs_params
return f"MATCH (%s) AGAINST (%s IN BOOLEAN MODE)" % (lhs, rhs), params
models.TextField.register_lookup(Search)
因为Django在mysql上不支持全文搜索,所以我们要添加我们的索引,所以我们修改生成的迁移文件:
# Generated by Django 2.2 on 2020-04-28 03:41
from django.db import migrations, models
# Table details
table_name = "by_api_casesnapshot"
field_name = "text"
index_name = f"{table_name}_{field_name}_index"
class Migration(migrations.Migration):
dependencies = [("by_api", "0033_add_tag_color")]
operations = [
migrations.CreateModel(...), # As auto-generated
migrations.RunSQL(
f"CREATE FULLTEXT INDEX {index_name} ON {table_name} ({field_name})",
f"DROP INDEX {index_name} ON {table_name}",
),
]
检查作业
# dbshell
mysql> SHOW INDEX FROM by_api_casesnapshot;
+---------------------+--------------------------------+-------------+-----+------------+-----+
| Table | Key_name | Column_name | ... | Index_type | ... |
+---------------------+--------------------------------+-------------+-----+------------+-----+
| by_api_casesnapshot | PRIMARY | id | ... | BTREE | ... |
| by_api_casesnapshot | by_api_casesnapshot_text_index | text | ... | FULLTEXT | ... |
+---------------------+--------------------------------+-------------+-----+------------+-----+
#shell
>>> snap = CaseSnapshot.objects.create(text="XXX")
# Regular query first
>>> CaseSnapshot.objects.filter(text__contains="X")
<QuerySet [<CaseSnapshot pk=1>]>
# Now test custom search lookup
>>> CaseSnapshot.objects.filter(text__search="XXX")
<QuerySet [<CaseSnapshot pk=1>]>
>>> CaseSnapshot.objects.filter(text__search="X*")
<QuerySet [<CaseSnapshot pk=1>]>
>>> CaseSnapshot.objects.filter(text__search="X*").query.__str__()
'SELECT `by_api_casesnapshot`.`id`, `by_api_casesnapshot`.`text` FROM `by_api_casesnapshot` WHERE MATCH (`by_api_casesnapshot`.`text`) AGAINST (X* IN BOOLEAN M
ODE)'
到目前为止,一切都完全按照预期工作。
问题
这是失败的地方 - 所有查询都使用 __search
查找 在测试期间 运行s return 查询集空集,甚至认为生成的查询是相同的。
测试
from django.test import TestCase
from by_api.models import CaseSnapshot
class TestModels(TestCase):
def test_search(self):
CaseSnapshot.objects.create(text="XXX")
rv1 = CaseSnapshot.objects.filter(text__contains="XXX")
# Passes
self.assertEqual(rv1.count(), 1)
rv2 = CaseSnapshot.objects.filter(text__search="XXX")
# Fails - count i
self.assertEqual(rv2.count(), 1)
我尝试在测试 运行 中使用 pdb 对其进行调试,并注意生成的查询与上面在 shell 环境中生成的查询相同:
>>> rv2.query.__str__())
'SELECT `by_api_casesnapshot`.`id`, `by_api_casesnapshot`.`text` FROM `by_api_casesnapshot` WHERE MATCH (`by_api_casesnapshot`.`text`) AGAINST (XXX IN BOOLEAN MODE)'
问题
为什么在测试期间使用 __search
查找会产生空查询集?
我试过的
- 调查了
- 确保测试数据库和 shell 数据库具有相同的值
- 尝试使用游标执行生成的查询 - 也是空的
更新
我刚刚注意到如果我使用 unittest.TestCase
而不是 django.test.TestCase
测试通过 - 这是否与 django 的测试用例如何包装数据库事务有关?
来自 django-mysql 库的 Adam Chainz 的回答:
InnoDB doesn't make changes to full text indexes until commits. You can use TransactionTestCase to get around this.
我将测试 class 更改为使用 TransactionTestCase
,测试现在通过了。