Let's create a Group (this model is a plain Django model) and some Musician instances:

In [1]:
beatles, created = Group.objects.get_or_create(name="The Beatles")
for name in (
    "John Lennon",
    "Paul McCartney",
    "Ringo Starr",
    "George Harrison"
    ):
    beatles.members.get_or_create(name=name)

Try some simple things with the models. How many members in The Beatles? We can ask a few different ways.

In [2]:
assert beatles.members.count() == 4
assert beatles.members.all().count() == 4

If we delete a musician, things keep working as we'd expect:

In [3]:
beatles.members.first().delete()
assert beatles.members.all().count() == 3
assert beatles.members.count() == 3

So far, so good. What if we try the same sort of thing with soft-deletable musicians?

In [4]:
queen, created = SoftDeletableGroup.objects.get_or_create(name="Queen")
for name in (
    "Freddie Mercury",
    "John Deacon",
    "Roger Taylor",
    "Brian May"
    ):
    queen.members.get_or_create(name=name)

assert queen.members.count() == 4
assert queen.members.all().count() == 4

queen.members.first().delete()
assert queen.members.all().count() == 3

Things seem OK. But see:

In [5]:
try:
    assert queen.members.count() == 3
except AssertionError:
    print("The assertion failed.")
The assertion failed.

We get AssertionError. If we forget to use .all(), it seems that the results include soft-deleted band members. Let's investigate.

In [6]:
queen.members.filter(name__isnull=False)
Out[6]:
<SoftDeletableQuerySet [<SoftDeletableMusician: John Deacon>, <SoftDeletableMusician: Roger Taylor>, <SoftDeletableMusician: Brian May>]>

Weird: it seems that all() is somehow being used behind the scenes, or something. I don't know. I just expect queen.members.count() to equal 3.

Things get weirder if we try annotations on queries:

In [7]:
groups = SoftDeletableGroup.objects \
    .annotate(num_members=Count('members')) \
    .filter(num_members__gt=3)
print(groups)
<SoftDeletableQuerySet [<SoftDeletableGroup: Queen>]>

The query should have returned an empty result set; we asked for SoftDeletableGroup objects having more than 3 members. So annotations behave weirdly (incorrectly?) as well.

In [8]:
groups.first().num_members
Out[8]:
4
In [9]:
groups.first().members.count()
Out[9]:
4
In [10]:
groups.first().members.all().count()
Out[10]:
3

These things are really, really difficult to track down when writing migrations, queries, basically anything. I've lost hours and hours to these under-the-hood behaviours.

Personally speaking, I'd prefer to be explicit all the way through my codebase and write (say) .available_objects instead of .objects wherever I want only "normal" model instances.

It's possible to make annotations work, but this doesn't rely on the magic behaviour inside SoftDeletableModel. The query below would work fine if is_removed was just implemented as a straightforward BooleanField.

In [11]:
SoftDeletableGroup.objects \
    .annotate(num_members=Count('members', filter=Q(members__is_removed=False))) \
    .first() \
    .num_members
Out[11]:
3

See for yourself how confusing things get. Try running this notebook more than once against a database, and see what happens. Clue: things break down at python cell [4]. Can you understand why? Would you have expected that before you saw it for yourself?