Let's create a Group (this model is a plain Django model) and some Musician instances:

In [1]:
beatles, created = Group.objects.get_or_create(name="The Beatles")
for name in (
"John Lennon",
"Paul McCartney",
"Ringo Starr",
"George Harrison"
):
beatles.members.get_or_create(name=name)


Try some simple things with the models. How many members in The Beatles? We can ask a few different ways.

In [2]:
assert beatles.members.count() == 4
assert beatles.members.all().count() == 4


If we delete a musician, things keep working as we'd expect:

In [3]:
beatles.members.first().delete()
assert beatles.members.all().count() == 3
assert beatles.members.count() == 3


So far, so good. What if we try the same sort of thing with soft-deletable musicians?

In [4]:
queen, created = SoftDeletableGroup.objects.get_or_create(name="Queen")
for name in (
"Freddie Mercury",
"John Deacon",
"Roger Taylor",
"Brian May"
):
queen.members.get_or_create(name=name)

assert queen.members.count() == 4
assert queen.members.all().count() == 4

queen.members.first().delete()
assert queen.members.all().count() == 3


Things seem OK. But see:

In [5]:
try:
assert queen.members.count() == 3
except AssertionError:
print("The assertion failed.")

The assertion failed.


We get AssertionError. If we forget to use .all(), it seems that the results include soft-deleted band members. Let's investigate.

In [6]:
queen.members.filter(name__isnull=False)

Out[6]:
<SoftDeletableQuerySet [<SoftDeletableMusician: John Deacon>, <SoftDeletableMusician: Roger Taylor>, <SoftDeletableMusician: Brian May>]>

Weird: it seems that all() is somehow being used behind the scenes, or something. I don't know. I just expect queen.members.count() to equal 3.

Things get weirder if we try annotations on queries:

In [7]:
groups = SoftDeletableGroup.objects \
.annotate(num_members=Count('members')) \
.filter(num_members__gt=3)
print(groups)

<SoftDeletableQuerySet [<SoftDeletableGroup: Queen>]>


The query should have returned an empty result set; we asked for SoftDeletableGroup objects having more than 3 members. So annotations behave weirdly (incorrectly?) as well.

In [8]:
groups.first().num_members

Out[8]:
4
In [9]:
groups.first().members.count()

Out[9]:
4
In [10]:
groups.first().members.all().count()

Out[10]:
3

These things are really, really difficult to track down when writing migrations, queries, basically anything. I've lost hours and hours to these under-the-hood behaviours.

Personally speaking, I'd prefer to be explicit all the way through my codebase and write (say) .available_objects instead of .objects wherever I want only "normal" model instances.

It's possible to make annotations work, but this doesn't rely on the magic behaviour inside SoftDeletableModel. The query below would work fine if is_removed was just implemented as a straightforward BooleanField.

In [11]:
SoftDeletableGroup.objects \
.annotate(num_members=Count('members', filter=Q(members__is_removed=False))) \
.first() \
.num_members

Out[11]:
3

See for yourself how confusing things get. Try running this notebook more than once against a database, and see what happens. Clue: things break down at python cell [4]. Can you understand why? Would you have expected that before you saw it for yourself?