Redesigning Django's generic class based views

Tom Christie

2nd September 2013

Introducing django-vanilla-views: a simplified redesign of Django's generic class-based views.

A recent thread on the Django developers mailing list has been discussing adding class hierarchy diagrams to the documentation for generic class based views.

For me, this thread highlighted just how awkward the current implementation is.

Let's take a look at the class hierarchy for one of the views. The following is the hierarchy diagram for CreateView.

Yowzers, that's quite something.

It gets even more scary if we take a look at the inheritance diagram for the complete set of generic views. I'm not going to include that here, because it'll hurt your eyes.

The wonderful CCBV site makes working with the views much less painful, but it's still difficult to untangle the behaviour.

As an example, let's take a look through what happens when we make a GET request to CreateView. The calling hierarchy is listed below.

As an example, let's take a look through what happens when we make a GET request to CreateView. The calling hierarchy is listed below.

CreateView.get()
|
+-- ProcessFormView.get()
    |
    +-- ModelFormMixin.get_form_class()
    |   |
    |   +-- SingleObjectMixin.get_queryset()
    |
    +-- FormMixin.get_form()
    |   |
    |   +-- ModelFormMixin.get_form_kwargs()
    |   |   |
    |   |   +-- FormMixin.get_form_kwargs()
    |   |
    |   +-- FormMixin.get_initial()
    |
    +-- ModelFormMixin.get_context_data()
    |   |
    |   +-- SingleObjectMixin.get_context_object_name()
    |   |
    |   +-- SingleObjectMixin.get_context_data()
    |       |
    |       +-- SingleObjectMixin.get_context_object_name()
    |       |
    |       +-- ContextMixin.get_context_data()
    |
    +-- TemplateResponseMixin.render_to_response()
        |
        +-- SingleObjectTemplateResponseMixin.get_template_names()
        |
        +-- TemplateResponseMixin.get_template_names()

The logic spans 8 classes, and 3 source files.

<sadface>.

Here's Adrian Holovaty, co-creator of Django, commenting on the current state of affairs.

I'd suggest not using class-based views. They're way over-abstracted and not worth the cognitive burden.

Again, <sadface>.

It doesn't have to be this way.

The Zen of Python says:

Simple is better than complex. Complex is better than complicated. Flat is better than nested.

I'm a big fan of class based views. We use Django's GCBVs extensively in our work at DabApps, and they save a lot of time and code. I don't enjoy the mixin style implementation, but they do what they need to and I wouldn't want to be without them.

Django REST framework also uses a derivative of generic class based views, where they're indespensible to super-fast API development with very little code. In REST framework, we've dropped the Django base implementations entirely in favor of a slightly more simplified version. There's only a single generic base class, and various parts of the API and implementation have been slimmed down. I find them vastly more pleasant to work with, and they're easy to manage from a maintenance perspective - questions on the mailing list don't tend to focus on usage of the generic views, and those that do are easily answered.

I've been meaning to take a similar approach to Django's standard GCBVs, and see if they could be redesigned with a nicer API and more obvious implementation.

Here's the result: django-vanilla-views.

The django-vanilla-views package provides essentially the same set of functionality as Django's existing GCBVs, but with:

No mixin classes.
No calls to super().
A sane class hierarchy.
A stripped down API.
Simpler method implementations, with less magical behavior.

The date-based generic views haven't yet been implemented, but all the existing model and base generic views are replicated.

The best way to get an idea of how django-vanilla-views differs from the current GCBV implementation is to take a look at its complete view hierarchy.

View --+------------------------- RedirectView
       |
       +-- GenericView -------+-- TemplateView
       |                      |
       |                      +-- FormView
       |
       +-- GenericModelView --+-- ListView
                              |
                              +-- DetailView
                              |
                              +-- CreateView
                              |
                              +-- UpdateView
                              |
                              +-- DeleteView

The django-vanilla-views package also weighs in with substantially less lines of code than the existing implementation. Around 50% for the same set of functionality, compared against Django's existing implementation:

Existing GCBVs

Source file	# lines of code
base.py (excluding View class)	139
detail.py	171
edit.py	280
list.py	197
total	787

Vanilla GCBVs

Source file	# lines of code
generic_view.py	90
generic_model_view.py	300
total	390

Compare and contrast.

We've seen the inheritance hierarchy for the current implementation of CreateView, so let's see how the CreateView from django-vanilla-views looks by comparison.

CreateView --> GenericModelView --> View

Let's take a look at the vanilla-style calling hierarchy.

CreateView.get()
|
+-- GenericModelView.get_form()
|   |
|   +-- GenericModelView.get_form_class()
|
+-- GenericModelView.get_context()
|
+-- GenericModelView.get_response()
    |
    +-- GenericModelView.get_template_names()

The difference is dramatic.

Less magic please

The method implementations in Django's current GCBV implementations contain a fair amount of implicit behavior, with more explicit cases gradually falling back to implicit shortcuts.

1. Determining the form class.

If .form_class exists, use that.
Else if .model exists, create a ModelForm class based on the model class.
Else if .object has been set, create a ModelForm class based on the model instance.
Else call .get_queryset(), and create a ModelForm class based on the queryset.

2. Determining the template name.

If .template_name exists add that.
If .object has been set, and .template_name_field exists, determine a template name based on a field on the instance, and add that.
If .object has been set, add a template name based on the model class of the instance.
Else if .model has been set, add a template name based on the model class.

3. Object lookup.

If a queryset argument is passed to .get_object() use that as the base queryset.
Else use a base queryset by calling .get_queryset().
If .pk_url_kwarg is provided in the view keyword arguments, use that as the object lookup against the primary key.
Else if .slug_url_kwarg is provided in the view keyword arguments, use that as the object lookup, looking up against the model field as determined by .get_slug_field(), which defaults to returning .slug_field.
Otherwise raise a configuration error.

4. Pagination.

Parent calls paginate_by() to determine the page size.
Parent calls paginate_queryset() passing the queryset and page size. The method returns a 4-tuple of (paginator, page, queryset, is_paginated).

The vanilla style

In django-vanilla-views we use broadly the same set of behavior, but simplify the implementations where possible. The end result is that the behavior is easier to understand and easier to override.

1. Determining the form class.

If .form_class exists, use that.
Else if .model exists, create a ModelForm class based on the model class.
Otherwise raise a configuration error.

2. Determining the template name.

If .template_name exists use that.
Else if .model exists, use a template name based on the model class.
Otherwise raise a configuration error.

3. Object lookup.

Use a base queryset by calling .get_queryset().
If .lookup_field is provided in the view keyword arguments, use that as the object lookup. The default value for .lookup_field is 'pk'.
Otherwise raise a configuration error.

4. Pagination.

Parent calls .paginate_queryset(), passing the queryset. The method returns a page object, or None if pagination is turned off.

Less API please

The generic view APIs have also been slimmed down slightly. The following view attributes that have been removed or replaced.

Object lookup arguments

pk_url_kwarg = 'pk'
slug_url_kwarg = 'slug'
slug_field = 'slug'

Replaced with:

lookup_field = 'pk'

If you need slug based lookup, just set the attribute.

class MyView(DetailView):
    model = MyModel
    lookup_field = 'slug'

Template name arguments

template_name_field = None

Removed. If you need a view that returns template names based on a field on the object instance, just write some code.

def get_template_names(self):
    return [self.object.template_name]

Pagination arguments

allow_empty = True

Removed. If you need a view that returns 404 responses for empty querysets, just write some code.

def get_queryset(self):
    queryset = super(MyView, self).get_queryset()
    if queryset.empty():
        raise Http404
    return queryset

Response arguments

content_type = None
response_cls = TemplateResponse

Removed. If you need a view that returns a different style of response, just write some code.

def get_response(context):
    # Note: Would typically need something a more complex than `json.dumps`
    # in order to handle serializing model instances and querysets.
    data = json.dumps(context)
    return HttpResponse(data, content_type='application/json')

Aside: Building better API behavior directly into the class based views is a topic for another day's discussion, but it's worth noting that you might want to consider Django REST framework if you're building APIs with class based views.

Let's make things better

It's okay that we as a community ended up with an awkward design for the generic class based views. The mixin design seemed like a good idea at the time, and no-one came up with a cleaner implementation. They fulfil their purpose, and it's a testment to the fundamental soundness of the concept that they are now so widely used. However, we do need to recognize that the current implementation and API is unacceptably complex, and figure out how we can improve the situation.

The current django-vanilla-views package is a first pass. It still needs documentation, tests, and perhaps also the date generic views. Any real proposal to modifying Django would also need to take into account a deprecation policy or migration guide. From my point of view django-vanilla-views seems like a huge improvement, and I'd really like community feedback on if and how we could go about getting something like this into core.

If you found this article and the django-vanilla-views package interesting, you might want to follow Tom Christie on Twitter, here, and DabApps here

Need a top-class engineering team? DabApps offers Web, API and Mobile development using Django and other technologies.

You might also be interested in our Python for Programmers training course, running in October and November this year.

To find out more, get in touch.

Marc Tamlyn
2nd September 2013

I'm very glad you wrote this. I also have absolutely no intention of using it.
I appreciate that the built in GCBVs can have some confusing points, but I personally don't find the API difficult to grok.
Firstly, I'd like to point out that your code size metrics are misleading - your code has no docstrings or comments which account for a significant number of lines in Django's implementation.
My other issue is that you seem to me to have chosen an arbitrary set of features to remove - some of which (such as `slug_url_kwarg` or related functionality) I find really useful. Some of it is kinda junk, but is there for consistency between the various views (what makes sense on a detail/list view may not seem to on update, but allows for greater interoperability).
I also believe that your mixin-less structure is likely to fall down at some point as you get conflict between different areas. Why does *every* view need to have the get_form function? There is no separation of concerns.
I don't necessarily like the style of your views, but I think it's a good thing that you could (fairly easily) do this. Generic means generic - they shouldn't necessarily be used everywhere. If you have your own implementation you're more comfortable with - go for it! The best thing is that this is still based of the standard View class - which points the gun away from people's feet.
I'm not saying that in the future the GCBV API won't change - there's an argument that it's due a rewrite and it has some crucial features (e.g. multiple forms) missing. It's good to see attempted rewrites, but I can make just as many arguments as to how this implementation is "broken" as you can about Django's.
- tom christie
  2nd September 2013
  
  Hi Marc,
  > but I personally don't find the API difficult to grok.
  You're a Django Core Developer. I wouldn't expect you to find it difficult to work with. But it does come up as an issue, time and time again. I don't think it's particular contentious to say that there are aspects of the GCBVs that make them not a strong point in Django's API.
  And again, look at that calling graph for CreateView. Even using CCBV I had difficulty pulling that together and making sure I hadn't missed anything.
  > I'd like to point out that your code size metrics are misleading
  Granted. Taking out docstrings I make it more like 268 lines of code vs 443 lines of code. It'd probably narrow further based on further tweaks to django-vanilla-views, but I still think you'd see a decent difference.
  Another measure is, say, the number of source lines of code that you can potentially run through in a GET request to `CreateView` - Here I counted 29 individual code lines for django-vanilla-views, against 72 for the current implementation in core.
  Yes, they're doing somewhat different things, but the intention is that there's essentially still feature-parity.
  > I also believe that your mixin-less structure is likely to fall down at some point as you get conflict between different areas.
  I get the root of that criticism, sure, but I think this is a case of practicality beat purity. It's worth noting that even in the mixin case, CreateView (as an example) still pulls in `get_object` (which it never uses), as well as `get_queryset` which it uses only in an edge case that I don't believe should actually be supported in order to automagically determine a form class to use if none of `model`, `form_class` or `queryset` are set, as well as a couple of base class methods that are overridden and not used.
  Given a single base class with 9 methods that's used throughout, versus 8 base classes providing 14 methods, plus multiple parent methods, I think it's clear which style is less complex.
  It's also worth re-iterating that this style is similar to what we use in Django REST framework. Personally I've found it substantially easier to use. I've also never seen a case of any practical issues that the single generic view base class runs into there, and as long as the API of the base class is kept minimal, I can't see it ever being an issue.
  > you seem to me to have chosen an arbitrary set of features to remove
  This is core: What I'm trying to do is very specifically not remove *any* functionality.
  Where any bits of API have been removed I've chosen to do so because the represent a less common use case (based on the current defaults, and my own experience) and only if it's trivial to add the behaviour back in through method overrides. The 'Less API please' section of the post covers that.
  If there are any use cases that django-vanilla-views cannot easily meet that the current views can then I'd consider that a bug and address it as such.
  > I'm not saying that in the future the GCBV API won't change
  It's not necessarily the case that a re-write would have to be wholesale, personally I'd like to see a fairly concerted attempt to simplify things, but I can see incremental approaches to be had too. There are plenty of improvements to be had, and IMHO we should be looking towards a less granular API, with a little less automagical behaviour.
  > I can make just as many arguments as to how this implementation is "broken" as you can about Django's.
  Take a step back and compare the two call diagrams for the existing CreateView, and for django-vanilla-view's CreateView. Or the class hierarchies for both packages. I can't see how there's any objective case for it not being a substantial improvement.
  I'm not trying to be negative by pointing to the issues with GCBVs - I use them all the time and I'm super glad we have them, but they do have some issues. You build something, and discover the pain points afterwards, nothing new there. It consistently comes up as a problematic bit of Django to work with, in particular to newcomers. Now that we've all been working with GCBVs for a while, I think we can be doing better.
  - Marc Tamlyn
    3rd September 2013
    
    > This is core: What I'm trying to do is very specifically not remove *any* functionality. Where any bits of API have been removed I've chosen to do so because the represent a less common use case (based on the current defaults, and my own experience).
    That's the thing, the things you've removed are based on your experience. In particular, I use the ability for the URL kwarg and the attribute of the model being different in almost every site I write - they normally have hierarchical URL structures and the urls are much cleaner by always using "foo_slug" - so when I reverse a book page the kwarg is "book_slug", when I reverse the chapter page, the kwargs are "book_slug" and "chapter_slug". This keeps consistence. When you build mostly APIs you don't need this feature so much as the URL structure tends to be much flatter, with fewer links between pages. I would be happy with two options though - "lookup_attribute" and "lookup_url_kwarg".
    If I were to redesign based on my experience, I'd remove all automatic form generation, as I always write a separate form class as I like to be explicit about which fields I want to use in all cases. Also most objects are related to the user, so create views need their own form or custom logic anyway.
    You're correct in that the views work in the same way, but you have removed API from them. Almost every ticket about GCBVs is to *add* functionality which someone finds useful in their 70% use cases. Sometimes it gets in (like auto generating forms based on the queryset), other times it doesn't. Once there, it's very difficult to remove.
    Your mixin-less structure works because the implementations are simple. The impression from many users is that practicality does beat purity here. They want the ease of configuration rather than a clean hierarchy. I'd like to see a fully featured version of the views with this structure - I feel it would result in a fair bit more duplication that we might be happy with. Then again, maybe that's ok?
    - tom christie
      3rd September 2013
      
      I've added a `url_kwarg` to meet the differing lookup field case you've described. You're right - that needs to be supported.
      Elsewhere I believe that any bits of API that are removed are all covered by behaviour that can be added in by trivial overrides. I'll aim to document the differences thoroughly at some point and show that we're not losing any functionality.
      It'd also be interesting to see a branch that doesn't remove any class attributes, but does use the different hierarchy, and more explicit behaviour. If I have the chance I'll push something along those lines up for further comparison.
      And yes, sure user tickets will tend towards requesting extra API. I'd expect that a simplified version of the GCBVs would make it easier to explain how to achieve the desired use-cases without requiring changes in core.
      I'll take this discussion to django-dev at some point, would be interesting to get some more feedback and see if there's anything around this worth working towards in Django core.
Daniel Roy Greenfeld
4th September 2013

This is a great idea for a library. CBVs are powerful, but for the uninitiated the library is scary. That much lasagna code is unpleasant for anyone and the learning curve for mortals like me who aren't core developers is hard. In fact, until django-mongonaut and the CBV docs refactor of DjangoCon EU 2012, I was against them.
This is the next, important step. Keep it up!
megaman821
10th September 2013

I like CBVs but the deep inheritance chain does not need to exist. A little bit of composition could go a long way.
A base view class implementing a typical response lifecycle could look like:
class View(object):
authenticate = None
authorize = None
dispatcher = None
handlers = []
renders = []
Andrey Kaygorodov
14th February 2014

First time when the CBVs were added to django, they seemed to me very complex and unclear (and If I ran into your article at that time, I would try your implementation in some pet projects). But after sometime working with them, I found the original CBV hierarchy pretty logic and flexible for many cases.
Matija Kolarić
27th March 2014

This is, again, a very interesting article. I do think that this discussion you and Marc started is quite constructive. But...
Models are abstractions of reality. We take whatever segments of reality are important for our app and then make a model out of it. We do it for ourselves (or clients for whom we develop).
Views are ... well, views. Its like looking out of a window. There is a bunch of different objects of different classes there. We identify them in views and then paint them with a template. We do it for end users.
I, like Marc, will not use your vanilla, but probably neither the canonical class based views. At least not often. The real magic is in satisfying both end users and ourselves/our clients. In my case, they are mostly very different. Perhaps a bit less for Marc.
However, when You develop with REST, then you do not deal with end users. You develop for yourselves (your frontend developers) or your clients (their frontend developers). So the "magic" is not within your scope. Its only data. Of course, class-based anythings fit wonderfully.
dfrankow
5th July 2015

I'd love to use these, but I'll look first at django-extra-views because it has inline formsets.

Commenting is now closed

It doesn't have to be this way.

Compare and contrast.

Less magic please

1. Determining the form class.

2. Determining the template name.

3. Object lookup.

4. Pagination.

The vanilla style

1. Determining the form class.

2. Determining the template name.

3. Object lookup.

4. Pagination.

Less API please

Object lookup arguments

Template name arguments

Pagination arguments

Response arguments

Let's make things better

Get in touch