Django Tips

I love Django. The web framework, not the jazz guitarist, though he’s not bad, either. Their documentation is a bit lacking in places, mostly in the details, which may be semi-niche edge cases, but I think it’s worth documenting what I’ve found, even if just for myself (as these posts usually are).

Widget Templates

I’ve wanted to tweak a few, and have, previously to what I discovered yesterday, gone deep into the weeds to modify the templates (in django/templates/django/forms/widgets). There are much easier ways! First of all, there’s an invaluable app, django-widget-tweaks, that, even though the code is a tad stale, does enough to be worth its use. Second, if you want to be fancier about things, or if you use other apps that override widgets (like django-markdownx), newer versions of Django need additional settings that are far from clear in the documentation.

First thing, edit your settings.py file. Add FORM_RENDERER='django.forms.renderers.TemplatesSetting' as its own line somewhere in the file. If you do this, and only this, you will find yourself presented with an error page where Django is expecting you to provide templates for everything. Maybe this is what you want, but probably not. To fix this behavior, in INSTALLED_APPS, add ‘django.forms’, and make sure this is the last line of the installed apps, or at the very least, after anything that might feasibly override a widget. That’ll take care of details that you don’t feel like worrying about as you develop your app. You might reach a point where you’ve covered every widget with your own, but it’s good to have a fallback just in case one was missed!

ModelForms

So you’re useing ModelForms to help build forms that you don’t have the time or inclination to write get and put methods all the time. Great! That’s what they’re there for! But there are some things that aren’t obvious, by any stretch!

How about passing data to one of these forms via the URL?

Let’s consider the polls model from the tutorial. Say that, instead of adding answers when editing the question (which, yes, in that case, makes more sense, but go with me for a minute) we want to add answers to a question on a page with the url ‘polls/<int:question_id>/add_answer/’. We know we’re adding an answer, and because of the URL we know the question ID, but the normal ModelForm for the given Answer model will ask you to provide the question_id, because of the ForeignKey field.

Here’s what you need to do:

class AnswerAdd(generic.CreateView):
    model=Answer
    def get_form(self):
        self.question=get_object_or_404(Question,id=self.kwargs['question_id'])
        partial_ans=Answer(question=self.question)
        kwargs=self.get_form_kwargs()
        kwargs['instance']=partial_ans
        form=AnswerForm(**kwargs)
        return form

First, you can’t get away without creating an AnswerForm class, but since this only requires a couple of extra lines from what you’d need in your view, and you’d have to duplicate that in your create and update views anyway, it’s probably worth it.

Second, and annoyingly, you can’t instantiate the form by doing AnswerForm(self.get_form_kwargs(), instance=partial_ans), but this is for good reason: because there’d then be two instance keyword arguments in the constructor—not allowed. Thus the separate lines for kwargs. This also allows you to pass in pass in extra data to the form.

Why can’t we do this in get_context_data? Because that method doesn’t handle the object instance.

There is a downside here, too: you can’t use the mere existence of the object in your template to figure out if someone is adding or editing an object. There’s a simple fix, though, just update your template to use this logic instead:

{% if not object.pk %}Create{% else %}Update{% endif %}

Updating Widgets at Runtime

This may be controversial, but I don’t know why. I think it has it place, and it’s situational. I have a number field. It has a meaning, and as the software developer I know what it means, but since that meaning can vary between instances in a rather complicated way that doesn’t lend itself to creating more objects and tables, and since I want to convey that meaning to my users, I want to replace the <input> widget with a <select> widget, a drop-down that’s far more intuitive for my users.

Here’s how you do that, though:

class MyModelForm(forms.ModelForm):
    class Meta:
        model=MyModel
        fields=[
            'field1',
            'field2',
            # you get the idea
        ]
    def __init__(self,*args,**kwargs):
        obj=kwargs.pop('some_object')
        super().__init__(*args,**kwargs)
        self.fields['field1'].widget=forms.Select(
            choices=obj.get_choices()
        )

Removing Widgets Dynamically

This is answered a few places at SO, but it’s worth rehashing here for my own benefit. Sometimes, you want/need to have a field that is only displayed during the creation of an object, or only when an object is updated, but you’re lazy and don’t want to make a CreateForm and an UpdateForm. I’m with you, all the way! And, as usual, there’s a way to do this:

class MyModelForm(forms.ModelForm):
    def __init__(self,*args,**kwargs):
        super().__init__(*args,**kwargs)
        if not self.instance:
            self.fields.pop('special_field1')
        else:
            self.fields.pop('special_field2')

With the logic above, special_field1 is removed when creating a new MyModel, and special_field2 is removed when updating that MyModel. The downside is that this won’t work in conjunction with passing data via the URL, because you have an instance, even if it’s mostly empty. You have to be a teensy bit cleverer (though I gave this answer out earlier in this post, I only figured it out here, so I’m repeating it for everyone’s benefit):

class MyModelForm(forms.ModelForm):
    def __init__(self,*args,**kwargs):
        super().__init__(*args,**kwargs)
        if not self.instance.pk:
            self.fields.pop('special_field1')
        else:
            self.fields.pop('special_field2')

Note that this time I’m checking for a value of instance.pk that isn’t a Boolean false. While you create an instance, no primary key is (typically) created until the object is saved in the database. Even if you have some situation where the PK is filled out in the instance you pass to the form, it’s likely there will be something that you can use to test if the form should be an edit or an update.

Deploying Python Scripts

I write a lot of little Python scripts for work, and a few for personal use. Most of them, even the work-related ones, never leave my own machine, because I write them to make my own life easier, and the syntax is specific, confusing, or I am too lazy to put in any error checking and don’t want to have to deal with the consequences of someone else messing something up by using one of my scripts. But for those that are extremely useful, I need to get them in a format usable by the masses.

We’ll use an example of munger.py—your day-to-day involves munging a file for a particular reason—and you want to simplify the process for you and your colleagues. And the function you want to run, because again you were lazy and not terribly forward-thinking, is called mung (not main).

Python is available on target…

If Python (of the appropriate version) is available on the target machine, you could simply pass along the script as a .py file and be done with it. But you’re nicer than that.

If you’re like me, you have a folder chock-full of random Python scripts. The first thing you’ll have to do is make a folder for the one you want to deploy, and give it an appropriate name. Let’s just do this here:

mung/
|-- mung/
| |-- __init__.py
| `-- munger.py
`-- setup.py

The original script was/is munger.py; the CLI command that will, in the end, be run, will be mung (i.e. mung filename.ext). __init__.py and setup.py are empty files for now. Let’s put some code in there.

Most of the time, I don’t bother with __init__.py, but it can be important when using setuptools, or in creating any package. Most of what you put in here is just imports, though, and here will be no different:

from .mung import *

And now, setup.py:

import setuptools

setuptools.setup(
    name='mung',
    version='1.0',
    packages=setuptools.find_packages(),
    author='<your name here>',
    author_email='<your email here>',
    description='a description',
    entry_points={
        'console_scripts': [
            'mung = mung.mung:mung'
        ],
        'gui_scripts': [
            'mung_gui = mung.mung:mung_gui'
        ],
    },
)

Refer to the documentation here if you plan on deploying your script to PyPI: there are a ton more options and keywords to pass to setup, and most of them will help your project’s use if you want to make it public (e.g. a full test suite, documentation, bug tracking, etc.). If you have a larger package, specific dependencies, a more complicated directory structure, they’re all covered there. But for the basics, what’s above should be more than sufficient.

Keep in mind: you must include name, version, packages, and entry_points (technically, entry_points is not required, but it’ll make documentation and explaining/teaching the use of your script that much simpler later—it’s worth it).

Now you’re ready. With those files saved, in the top mung directory (containing setup.py), and after ensuring you have the latest version of setuptools and wheel, run the following:

python setup.py sdist bdist_wheel

You’ll find the built version of your script in dist/mung-1.0-py3-none-any.whl. The version, and ‘none’ and ‘any’ parts of the filename may be different depending on your build options. But you can pass that one file on to your colleagues, and have them run the following to install it:

pip install mung-1.0-py3-none-any.whl

Rename as you feel appropriate before passing it on!

Your built script will be a bit larger, but not by a lot; I used a 1064 byte test file on my own machine as an example and the built wheel was just 2052 bytes!

Python not available on target…

If Python is not available on the target machine, things are a bit easier and a bit harder at the same time. I recommend is getting your hands on the latest versions of PyInstaller (which does not actually create or install things) and NSIS (which does). I’ll only discuss utilizing these tools. If you want to cross-compile, this will not do the job. For that I’d recommend trying to get Python installed on the target computer—it’s probably easier in the long run.

Each of the following options have their pros and cons. If speed is an issue, and rewriting the program in C or installing Python on the target are not options, go with option 1. If portability is the main factor, go with option 2. If you’re tight on file size, try 2a.

Option 1: Installer

Using PyInstaller, building the script into an executable is fairly straightforward, although you don’t have the same flexibility as the previous method. You can’t define entry points, so you need to be able to directly run the script—no function calling after-the-fact. Add something like this to the bottom of your file if necessary:

if __name__=='__main__':
    mung()

Run the following command:

pyinstaller munger.py

You’ll wind up with munger.exe and a bazillion other files in a dist directory. You may have to mess with the spec file that’s created to make it work just right; see the documentation for details there.

With my 1064 byte test input, I wound up with 52 files in the output totaling 12,513,606 bytes.

How do we deploy 52 files? NSIS. Create mung.nsi with the following contents:

!include "MUI.nsh"
#define compression method
SetCompressor /SOLID LZMA

!define MajorVersion 1
!define MinorVersion 0
!define PatchVersion 0

#define version information
VIAddVersionKey "ProductName" "Munger"
VIAddVersionKey "CompanyName" "<Your Company Here>"
VIAddVersionKey "LegalCopyright" "©20XX All rights reserved."
VIAddVersionKey "FileVersion" "${MajorVersion}.${MinorVersion}.${PatchVersion}.0"
VIAddVersionKey "ProductVersion" "${MajorVersion}.${MinorVersion}.${PatchVersion}.0"
VIAddVersionKey "FileDescription" "<Executable Description Here>"
VIProductVersion "${MajorVersion}.${MinorVersion}.${PatchVersion}.0"
VIFileVersion "${MajorVersion}.${MinorVersion}.${PatchVersion}.0"

RequestExecutionLevel admin ;Require admin rights on NT6+ (When UAC is turned on)

!include LogicLib.nsh

Function .onInit
UserInfo::GetAccountType
pop $0
${If} $0 != "admin" ;Require admin rights on NT4+
    MessageBox mb_iconstop "Administrator rights required!"
    SetErrorLevel 740 ;ERROR_ELEVATION_REQUIRED
    Quit
${EndIf}
FunctionEnd

#define output file
!ifndef LabelVersion
  Outfile "munger-${MajorVersion}.${MinorVersion}.${PatchVersion}.exe"
!endif
!ifdef LabelVersion
  Outfile "munger-${MajorVersion}.${MinorVersion}.${PatchVersion}-${LabelVersion}.exe"
!endif

#define installation directory
# should be C:\Program Files\Munger
InstallDir "$PROGRAMFILES64\Munger" #Insert appropriate directory here

#define installer name
Caption "Munger"
Name "Munger"

#define variables
!define MUI_ICON "myicon.ico"
ShowInstDetails show

!insertmacro MUI_PAGE_WELCOME
!insertmacro MUI_PAGE_DIRECTORY
!insertmacro MUI_PAGE_INSTFILES
!insertmacro MUI_PAGE_FINISH

#Include logic functions
!include LogicLib.nsh

#default section
Section
  #define options
  SetShellVarContext current
  SetOverwrite ifnewer
  SetOutPath $INSTDIR
  #install file
  File /r dist\*
  CreateShortcut "$DESKTOP\munger.lnk" "$INSTDIR\munger.exe"
  WriteUninstaller "$INSTDIR\uninstaller.exe"
SectionEnd

Section "Uninstall"
  Delete "$INSTDIR\uninstaller.exe"
  !include /CHARSET=UTF8 "remfiles.nsh"
  RMDir "$INSTDIR"
  Delete "$DESKTOP\munger.lnk"
SectionEnd

!insertmacro MUI_LANGUAGE "English"

Add nsis_uninstall_helper.py with the following contents:

import os
import sys

#generate removal file for NSIS
print(sys.argv[1])
with open('remfiles.nsh','w') as fp:
    for root,dirs,files in os.walk(sys.argv[1],False):
        for f in files:
            t=fp.write('  Delete "$INSTDIR\\{}"\n'.format(os.path.join(root,f)[len(sys.argv[1])+1:]))
        for d in dirs:
            t=fp.write('  RMDir "$INSTDIR\\{}"\n'.format(os.path.join(root,d)[len(sys.argv[1])+1:]))

Now you should have something like the following:

somedir
|-- build/
| |-- munger/
| |-- Analysis-00.toc
| |-- COLLECT-00.toc
| |-- EXE-00.toc
| |-- PKG-00.toc
| |-- PYZ-00.pyz
| |-- PYZ-00.toc
| |-- base_library.zip
| |-- munger.exe
| |-- munger.exe.manifest
| |-- warn-munger.txt
| `-- xref-password.html
|-- dist/
| |-- munger/
| | |-- ... <--a bunch of stuff
| | `-- munger.exe
|-- munger.py
|-- munger.spec
|-- nsis_uninstall_helper.py
`-- mung.nsi

In the dist directory, run the following:

python nsis_uninstall_helper.py
makensis mung.nsi

If all goes well, you’ll wind up with munger-1.0.0.exe in the directory. My personal sample came in at a svelte 4,443,448 bytes. This you can distribute like any other installer, and, provided the target user has the appropriate rights, they shouldn’t have much trouble.

Option 2: Single File

Just like in the previous section, when using PyInstaller, building the script into an executable is fairly straightforward, although you don’t have the same flexibility as the previous method. You can’t define entry points, so you need to be able to directly run the script—no function calling after-the-fact. Add something like this to the bottom of your file if necessary:

if __name__=='__main__':
    mung()

Run the following command:

pyinstaller -F munger.py

You’ll wind up with munger.exe in a dist directory. You may have to mess with the spec file that’s created to make it work just right; see the documentation for details there. But that’s all you need! Pass along the file as-is, and you shouldn’t have too many issues.

With my 1064 byte test input, I wound up with just the one file totaling 6,094,767 bytes. Not too shabby!

Option 2a: Single File (With UPX)

You can, perhaps, compress your file even more using UPX. I won’t go into installing or configuring it, see the documentation for the details, but here are my results:

Run the following command:

pyinstaller -F --upx-dir=path\to\upx munger.py

You’ll wind up with munger.exe in a dist directory. You may have to mess with the spec file that’s created to make it work just right; see the documentation for details there. But that’s all you need! Pass along the file as-is, and you shouldn’t have too many issues.

Again, with my 1064 byte test input, I wound up with just the one file totaling 5,057,013 bytes!