Abhijeet Pal: Adding Pagination With Django

While working on a modern web application quite often you will need to paginate the app be it for better user experience or performance. Fortunately, Django comes with built-in pagination classes for managing paginating data of your application.

In this article, we will go through the pagination process with class-based views and function based views in Django.

Prerequisite

For the sake of this tutorial I am using a blog application  – Github repo

The above project is made on Python 3.7, Django 2.1 and Bootstrap 4.3. This is a very basic blog application displaying a list of posts on the homepage but when the number of posts increases we need to split them up.

Recommended Article:  Building A Blog Application With Django

Adding Pagination Using Class-Based-Views [ ListView ]

The Django ListView class comes with built-in support for pagination so all we need to do is take advantage of it. Pagination is controlled by the GET parameter that controls which page to show.

First, open the views.py file of your app.

from django.views import generic from .models import Post   class PostList(generic.ListView):     queryset = Post.objects.filter(status=1).order_by('-created_on')     template_name = 'index.html'  class PostDetail(generic.DetailView):     model = Post     template_name = 'post_detail.html' 

Now in the PostList view we will introduce a new attribute paginate_by which takes an integer specifying how many objects should be displayed per page. If this is given, the view will paginate objects with paginate_by objects per page. The view will expect either a page query string parameter (via request.GET) or a page variable specified in the URLconf.

class PostList(generic.ListView):     queryset = Post.objects.filter(status=1).order_by('-created_on')     template_name = 'index.html'     paginate_by = 3

Now our posts are paginated by 3 posts a page.

Next, to see the pagination in action, we need to edit the template which for this application is the index.html file paste the below snippet.

{% if is_paginated %}   <nav aria-label="Page navigation conatiner"></nav>   <ul class="pagination justify-content-center">     {% if page_obj.has_previous %}     <li><a href="?page={{ page_obj.previous_page_number }}" class="page-link">&laquo; PREV </a></li>     {% endif %}     {% if page_obj.has_next %}     <li><a href="?page={{ page_obj.next_page_number }}" class="page-link"> NEXT &raquo;</a></li>      {% endif %}   </ul>   </nav> </div> {% endif %}

Note that we are using Bootstrap 4.3 for this project, if you are using any other frontend framework you may change the classes.

Now run the server and visit http://127.0.0.1:8000/ you should see the page navigation buttons below the posts.

Adding pagination with class based views in django

 

Adding Pagination Using Function-Based-Views

Equivalent function based view for the above PosList class would be.

def PostList(request):     object_list = Post.objects.filter(status=1).order_by('-created_on')     paginator = Paginator(object_list, 3)  # 3 posts in each page     page = request.GET.get('page')     try:         post_list = paginator.page(page)     except PageNotAnInteger:             # If page is not an integer deliver the first page         post_list = paginator.page(1)     except EmptyPage:         # If page is out of range deliver last page of results         post_list = paginator.page(paginator.num_pages)     return render(request,                   'index.html',                   {'page': page,                    'post_list': post_list}) 

So in the view, we instantiate the Paginator class with the number of objects to be displayed on each page i.e 3. Then we have the request.GET.get('page') parameter which returns the current page number. The page() method is used to obtain the objects from the desired page number. Below that we have two exception statements for PageNotAnInteger and EmptyPage both are subclasses of InvalidPage finally at the end we are rendering out the HTML file.

Now in your templates paste the below snippet.

{% if post_list.has_other_pages %}   <nav aria-label="Page navigation conatiner"></nav>   <ul class="pagination justify-content-center">     {% if post_list.has_previous %}     <li><a href="?page={{ post_list.previous_page_number }}" class="page-link">&laquo; PREV </a></li>     {% endif %}     {% if post_list.has_next %}     <li><a href="?page={{ post_list.next_page_number }}" class="page-link"> NEXT &raquo;</a></li>   </ul>   </nav>   {% endif %}

Save the files and run the server you should see the NEXT button below the post list.

Adding Pagination Using Function-Based-Views

The post Adding Pagination With Django appeared first on Django Central.

Planet Python

Django Weblog: Unauthenticated Remote Code Execution on djangoci.com

Yesterday the Django Security and Operations teams were made aware of a remote code execution vulnerability in the Django Software Foundation’s Jenkins infrastructure, used to run tests on the Django code base for GitHub pull requests and release branches. In this blog post, the teams want to outline the course of events.

Impact

The Django Security and Operations teams want to assure that at no point was there any risk about issuing or uploading malicious releases of Django to PyPI or the Django Project website. Official Django releases have always been issued manually by releasers. Neither was there any risk to any user data related to the Django Project website or the Django bug tracker.

Timeline

On May 14th, 2019 at 07:48 UTC the Django Security team was made aware by Ai Ho through its HackerOne project that the Django’s Continuous Integration service was susceptible to a remote code execution vulnerability, allowing unauthenticated users to execute arbitrary code.

At 08:01 UTC, the Django Security team acknowledged the report and took immediate steps to mitigate the issue by shutting down the primary Jenkins server. The Jenkins master server was shut down by 08:10 UTC.

At 08:45 UTC, the Operations team started provisioning a new server. In cases of a compromised server, it is almost always impractical to clean it up. Starting with a fresh, clean installation is a considerably better and safer approach.

At 14:59 UTC, the new Jenkins master server was up and running again, with some configuration left to do to get Jenkins jobs working again. About 10 minutes later, at 15:09 UTC, that was the case.

At 15:44 UTC, Jenkins started running tests against GitHub pull requests again.

At 16:00 UTC, the Operations team discussed the necessity of revoking various Let’s Encrypt certificates or keys. However, since there was no indication that either the account or the certificate’s private key was exposed, it was deemed sufficient to rely on the auto-expiration of the Let’s Encrypt certificate. However, a new private key for the djangoci.com certificate was generated during the bootstrapping of the new Jenkins master server.

At 16:50 UTC, the Jenkins Windows nodes were working again and started to process jobs.

General notes regarding security reporting

As always, we ask that potential security issues be reported via private email to security@djangoproject.com or HackerOne, and not via Django’s Trac instance or the django-developers list. Please see our security policies for further information.

Planet Python

Zero-with-Dot (Oleg Żero): AJAX calls in Django 2.2

Introduction

AJAX (Asyncroneous JavaScript and XML) is a great way of updating client content without the need to reload the whole webpage. When working with Django, the front-end code is rendered form the backend as a part of template generation. So is any JavaScript code.

Although the problem of creating AJAX calls in Django has been solved in the past, the framework constantly evolves. At the time of writing of this post, the most recent version of Django was 2.2. Therefore, this post is written with an intention to outline a clear implementation recipe and bring it more up to date.

Basic scenario – GET

The most basic scenario assumes that we have want to request some data form a specific end-point. This tasks fits in very much with a GET method. To set it up, all we need to do is to:

  • define an end-point (in urls.py),
  • define the corresponding function (in views.py),
  • create a routine in JavaScript and include it in the template we generate.

Here is how we do it:

urls.py

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 
from django.urls import path from django.conf import settings from djamgo.cong.urls.static import static  from . import views   urlpatterns = [     ...     path('stuff/<int:item_id>', views.Stuff.as_view(), name='stuff')     ... ]  if settings.DEBUG:     urlpatterns += static(settings.STATIC_URL,         document_root=settings.STATIC_ROOT)

Here, we choose to go on with class-based views, mainly to avoid having to branch the functions with conditions like if request.method == _GET_ and so on. Also, already at this stage, we account for designated paths for .js files we will include, instead of writing JavaScript code alongside with HTML. Although this approach works, it doesn’t scale.

views.py

1 2 3 4 5 6 7 8 9 10 11 12 
from django.views.generic import ListView from django.https import JsonResponse  from .models import StuffModel   class Stuff(ListView):     model = StuffModel      def get(self, request, item_id):         stuff_item = StuffModel.objects.get(id=item_id)         return JsonResponse({'item': stuff_item})

templates/yourapp/list.html

1 2 3 4 5 6 7 8 9 10 11 12 
{% extends 'youapp/base.html' %} {% load static %}   <!-- to trigger the call --> <button id="42" onclick="getStuff(elem)">Get Stuff</button>  <!-- to fill out from AJAX response --> <div id="something"></div>  <!-- include the call's logic in a separate script --> <script src="{% static 'yourapp/js/calls.js' %}"></script>

static/yourapp/js/calls.js

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 
function getStuff(elem) {   const id = $  (elem).attr('id');      $  .ajax({     url:            window.location.pathname + id,     type:           'GET',     contentType:    'application/json; charset=utf-8',     dataType:       'json',     success:        function (result) {       $  ('#something').prepend(         "<p>" + result.item + "</p>"       );     }   }); } 

The essence of this logic is to trigger the call, synthesize the correct URL to hit out end point and do something with the result once response is received. Note that we rely on jQuery in this example (see: here).

Adding complexity – POST

GET methods are designed, as the name suggests, to fetch data from the servers, not to add or change anything. If we want to push some information, we should use other methods, such as POST (see later for PUT and DELETE). For obvious reasons, we would like to ensure that whatever is being POST-ed, comes from a legitimate source. The way Django handles it is that it enforces all POST, PUT and DELETE methods to present a so-called cross-site request forgery (CSRF) tokens (see here).

From the developer’s point of view, all you need to do when creating of a form that uses e.g. POST method, is to paste {% csrf_token %} inside <form></form> tags, which in the background, creates an additional hidden field into the form with the token it generates. Unfortunately, we are not working with forms here and the following snippets explain the process:

views.py

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 
...  class Stuff(ListView):     model = StuffModel      def get(self, request, item_id):         ...      def post(self, request, item_id):         if request.is_ajax()             new_stuff = StuffModel(item_id=item_id)             new_stuff.save()             return JsonResponse({'message': 'OK'})         else:             return jsonResponse({'message': 'Failed'})

Here, we have added a post function to support HTTP POST method. The logic is, however, somewhat arbitrary. It is really up to you what you want to achieve. In this example, all we do is to create a new database entry (implicitly assuming that item_id is a field).

templates/yourapp/list.html

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 
{% extends 'youapp/base.html' %} {% load static %}   <!-- to trigger the call --> <button id="42" onclick="getStuff(elem)">Get Stuff</button>  <!-- to fill out from AJAX response --> <div id="something"></div>  <!-- include the call's logic in a separate script --> <script type="text/javascript"> window.CSRF_TOKEN = ""; </script> <script src="{% static 'yourapp/js/calls.js' %}"></script>

The double braces {{ ... }} (in the contary to {% ... %}) place the token in to the template literally. This value, we take using the tiny JavaScript snippet (lines 12-14) and save it into the DOM. Thanks to that, we can have it available for calls.js that is attached separately.

static/yourapp/js/calls.py

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 
function postStuff(elem) {   const id = $  (elem).attr('id');   const csrftoken = window.CSRF_TOKEN;      $  .ajax({     url:            window.location.pathname + id,     type:           'POST',     beforeSend:     function (request) {       request.setRequestHeader("X-CSRFToken", csrftoken);     },     data:           {       csrfmiddlewaretoken: csrftoken     },     contentType:    'application/json; charset=utf-8',     dataType:       'json',     success:        function (result) {       $  ('#something').prepend(         "<p>" + result.item + "</p>"       );     }   }); } 

There are important modifications that we have made:

  • First, we retrieve the CSRF value from the DOM.
  • Change the type of the method to POST (obviously).
  • Next, add a specific header line X-CSRFToken.
  • Send the token also as normal data – just as if you were sending data from a form.

Unless Django receives the token in a correct way, it will reply with 403 code.

Other methods – DELETE, PUT

The HTTP protocol defines more specific methods for altering (PUT) or deleting (DELETE) of resources. Although from the logical point of view, the goal can be achieved with POST method only, we would need to create additional URLs.

If we use class-based views, implementing both DELETE and PUT methods can be done analogically to POST:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 
class Stuff(ListView):     model = StuffModel      def get(self, request, item_id):         ...      def post(self, request, item_id):         ...      def delete(self, request, item_id):         ...      def put(self, request, item_id):         ...

Similarly, the calls.js function can be modified with changing of the type attribute, essentially leaving the CSRF implementation the same.

Unfrotunately, handling of request objects for PUT and DELETE methods is more cumbersome in Django. According to the documentation, only HttpRequest.GET and HttpRequest.POST options are supported. For other methods, we cannot use this simple dictionary-like structure to get the content, only leaving us with manual parsing of the request.body. Since the intention of Django was to work with web browsers and accept content from forms (bsically done through POST request), it is perhaps not surprising. The following example illustrates the difference.

example

1 2 3 4 5 6 7 8 9 10 11 12 13 
... def post(self, request):     param = request.POST.get('param')     ...  def delete(self, request):     body = request.body.decode('utf-8') # if utf-8 encoded...     param_list = body.split('&')     params = {}     for p in param_list:         params[p.split('=')[0]] = p.split('=')[1]     param = params['param']     ...

All in all, the exact implementation depends on the needs. Personally, I would prefer to use the action-specific methods, however the above delete function looks a bit ugly. Therefore, for consistency, I would define a separate URL and stick to using of POST method as it flows more naturally with Django. In the end, the most important part is to get the AJAX call work, which in this case it is no different for as soon as we implement the CSRF token correctly.

Planet Python

PyBites: How to Create and Serve Zipfiles from Django

We added support to our platfom for bulk downloading of all your code submissions. This feature required creating and serving up zipfiles through Django. In this article I show you how to do it creating a simple Django app collecting code snippets through the admin interface, and serving them up in a zipfile via a download endpoint. Let’s dive straight in …

Setup

First we make a virtual env, set a secret key in our venv and install Django:

[bobbelderbos@imac code]$   mkdir django-archive [bobbelderbos@imac code]$   cd $  _ [bobbelderbos@imac django-archive]$   python3.7 -m venv venv [bobbelderbos@imac django-archive]$   echo "export SECRET_KEY='abc123.;#'" >> venv/bin/activate [bobbelderbos@imac django-archive]$   source venv/bin/activate (venv) [bobbelderbos@imac django-archive]$   pip install django Collecting django ... Successfully installed django-2.2 pytz-2019.1 sqlparse-0.3.0 

Now let’s create a project and app in Django. Don’t forget the extra dot in the startproject command to not create an extra subdirectory.

(venv) [bobbelderbos@imac django-archive]$   django-admin startproject snippets . (venv) [bobbelderbos@imac django-archive]$   django-admin startapp archive (venv) [bobbelderbos@imac django-archive]$   tree -L 2 . ├── archive │   ├── __init__.py │   ├── admin.py │   ├── apps.py │   ├── migrations │   ├── models.py │   ├── tests.py │   └── views.py ├── manage.py ├── snippets │   ├── __init__.py │   ├── settings.py │   ├── urls.py │   └── wsgi.py └── venv     ... 

Make sure we add the new app to Django’s config:

snippets/settings.py

INSTALLED_APPS = [     ...     # own apps     'archive', ] 

While here, let’s also load the secret key from our venv (venv/bin/activate) as defined earlier:

SECRET_KEY = os.environ['SECRET_KEY'] 

Lastly let’s sync the pending migrations to our default sqlite DB and create a superuser to access Django’s admin back-end:

(venv) [bobbelderbos@imac django-archive]$   python manage.py migrate Operations to perform: Apply all migrations: admin, auth, contenttypes, sessions Running migrations: Applying contenttypes.0001_initial... OK Applying auth.0001_initial... OK Applying admin.0001_initial... OK Applying admin.0002_logentry_remove_auto_add... OK Applying admin.0003_logentry_add_action_flag_choices... OK Applying contenttypes.0002_remove_content_type_name... OK Applying auth.0002_alter_permission_name_max_length... OK Applying auth.0003_alter_user_email_max_length... OK Applying auth.0004_alter_user_username_opts... OK Applying auth.0005_alter_user_last_login_null... OK Applying auth.0006_require_contenttypes_0002... OK Applying auth.0007_alter_validators_add_error_messages... OK Applying auth.0008_alter_user_username_max_length... OK Applying auth.0009_alter_user_last_name_max_length... OK Applying auth.0010_alter_group_name_max_length... OK Applying auth.0011_update_proxy_permissions... OK Applying sessions.0001_initial... OK  (venv) [bobbelderbos@imac django-archive]$   python manage.py createsuperuser Username (leave blank to use 'bobbelderbos'): bob Email address: Password: Password (again): This password is too short. It must contain at least 8 characters. Bypass password validation and create user anyway? [y/N]: y Superuser created successfully. 

Create routes

In the main app snippets, that was created with the startproject command, we add the following routes:

snippets/urls.py

from django.contrib import admin from django.urls import path, include  urlpatterns = [     path('admin/', admin.site.urls),     path('', include('archive.urls', namespace='archive')), ] 

Apart from the default admin routes (admin.site.urls), we namespace the archive app’s routes, defining them in the app:

archive/urls.py

from django.urls import path  from . import views  app_name = 'archive' urlpatterns = [     path('download/', views.download, name='download') ] 

This will be the download endpoint that will serve the zipfile, we will write that code in a bit. First let’s define the model (DB table) that will hold our code snippets.

Create a Script model

In our archive app we make this simple model and sync it to the DB:

archive/models.py

from django.db import models   class Script(models.Model):     name = models.CharField(max_length=100)     code = models.TextField()     added = models.DateTimeField(auto_now_add=True)      def __str__(self):         return self.name      class Meta:         ordering = ['-added'] 

We inherit all goodness from Django’s Model class. The added datetime gets automatically populated upon insert. Defining a __str__ on the class makes it easier to inspect the objects when debugging (or in Django’s interactive shell). And we can use the inner Meta class to set further behaviors, in this case let’s show most recently added snippets first.

Now we have to commit (“migrate”) this model to the DB which is easy using Django’s manage.py. However first we need to stub out the download function we defined in archive/urls.py, otherwise we get: AttributeError: module 'archive.views' has no attribute 'download' upon migration. Add this code to archive/views.py:

archive/views.py

def download(request):     pass 

Now it should work:

(venv) [bobbelderbos@imac django-archive]$   python manage.py makemigrations Migrations for 'archive': archive/migrations/0001_initial.py     - Create model Script (venv) [bobbelderbos@imac django-archive]$   python manage.py migrate Operations to perform: Apply all migrations: admin, archive, auth, contenttypes, sessions Running migrations: Applying archive.0001_initial... OK 

I am just using the default sqlite DB, we can use schema to see what migrate created:

(venv) [bobbelderbos@imac django-archive]$   sqlite3 db.sqlite3 SQLite version 3.24.0 2018-06-04 19:24:41 Enter ".help" for usage hints. sqlite> .table ... other tables ... archive_script              <== our new table sqlite> .schema  archive_script CREATE TABLE IF NOT EXISTS "archive_script" ("id" integer NOT NULL PRIMARY KEY AUTOINCREMENT, "name" varchar(100) NOT NULL, "code" text NOT NULL, "added" datetime NOT NULL); 

Django’s admin interface

To be able to work with the new model from the admin interface we need to register it. Add this code to the archive/admin.py module:

archive/admin.py

from django.contrib import admin  from .models import Script   class ScriptAdmin(admin.ModelAdmin):     pass admin.site.register(Script, ScriptAdmin) 

Now let’s spin up the dev server. As I leave it running in the foreground I use a second terminal:

$   cd /Users/bbelderbos/code/django-archive $   source venv/bin/activate (venv) [bbelderbos@imac django-archive]$   python manage.py runserver Watching for file changes with StatReloader Performing system checks...  System check identified no issues (0 silenced). May 08, 2019 - 02:17:32 Django version 2.2, using settings 'snippets.settings' Starting development server at http://127.0.0.1:8000/ Quit the server with CONTROL-C. 

Now I can access http://localhost:8000/admin and login with the superuser I created earlier. At this point we should see the new model:

new model in admin

Let’s add some small code snippets from our tips page:

3 snippets added

adding snippet 1

adding snippet 2

adding snippet 3

Serving up a zipfile

Now let’s create a zipfile with all the code snippets stored in the DB. We do this in the download view we stubbed out earlier:

archive/views.py

import zipfile  from django.http import HttpResponse  from .models import Script  README_NAME = 'README.md' README_CONTENT = """ ## PyBites Code Snippet Archive  Here is a zipfile with some useful code snippets.  Produced for blog post https://pybit.es/django-zipfiles.html  Keep calm and code in Python! """ ZIPFILE_NAME = 'pybites_codesnippets.zip'   def download(request):     """Download archive zip file of code snippets"""     response = HttpResponse(content_type='application/zip')     zf = zipfile.ZipFile(response, 'w')      # create the zipfile in memory using writestr     # add a readme     zf.writestr(README_NAME, README_CONTENT)      # retrieve snippets from ORM and them to zipfile     scripts = Script.objects.all()     for snippet in scripts:         zf.writestr(snippet.name, snippet.code)      # return as zipfile     response['Content-Disposition'] = f'attachment; filename={ZIPFILE_NAME}'     return response 

We use Django’s HttpResponse object which we have to give a Content-Disposition attribute. To directly serve up the resulting zipfile, not writing it to disk, I use zipfile‘s writestr. Getting the snippets from Django’s ORM is as easy as: Script.objects.all(). I also added a README file.

Now visit the download endpoint: http://localhost:8000/download -> A zipfile should automatically download to your desktop:

download the zipfile

Let’s see if it worked by unzipping the obtained zipfile into a tmp directory:

[bbelderbos@imac Downloads]$   mkdir tmp [bbelderbos@imac Downloads]$   mv pybites_codesnippets.zip tmp [bbelderbos@imac Downloads]$   cd tmp [bbelderbos@imac tmp]$   unzip pybites_codesnippets.zip Archive:  pybites_codesnippets.zip extracting: README.md extracting: flatten.py extracting: zipping.py extracting: enumerate.py  [bbelderbos@imac tmp]$   cat README.md  ## PyBites Code Snippet Archive  Here is a zipfile with some useful code snippets.  Produced for blog post https://pybit.es/django-zipfile  Keep calm and code in Python!  [bbelderbos@imac tmp]$   for i in *py; do echo "== $  i =="; cat $  i; echo ; done == enumerate.py == names = 'bob julian tim sara'.split() for i, name in enumerate(names, 1):     print(i, name) == flatten.py == list_of_lists = [[1, 2], [3], [4, 5], [6, 7, 8]] flattened = sum(list_of_lists, []) print(flattened) == zipping.py == names = 'bob julian tim sara'.split() ages = '11 22 33 44'.split() print(dict(zip(names, ages))) 

Cool! So there you have it: a small Django app with a single model and view to serve zipfiles 🙂

One enhancement would be to lock this down for users that are not logged in. Django makes this easy, just add this the following code at the top of the download function, returning a 401 (and toast message) if the user is not authenticated:

from django.contrib import messages ... def download(request):     """Download archive zip file of code snippets"""     response = HttpResponse(content_type='application/zip')      # add this:     if not request.user.is_authenticated:         messages.error(request, 'Need to be logged in to access this endpoint')         return HttpResponse(status=401)     # end      ...      ... 

The full code for this blog post is here.


If you saved some code for Bite exercises on our platform you can check out this feature scrolling to the bottom of the settings page

I hope this was useful and let us know if there are other Django related topics you’d like to see covered here …

Keep Calm and Code in Python!

— Bob

Planet Python

Catalin George Festila: Python 3.7.3 and Django CMS.

From official www.django-cms.org: django CMS was originally conceived by web developers frustrated with the technical and security limitations of other systems. Its lightweight core makes it easy to integrate with other software and put to use immediately, while its ease of use makes it the go-to choice for content managers, content editors and website admins. The django-crm.readthedocs.io/en/
Planet Python