The last post covered the structure of Django Model. This post covers how the model field works, what are the some important methods and functionality and properties of the field.
Object-Relational Mapper is a technique of declaring, querying the database tables using Object relationship in the programming language. Here is a sample model declaration in Django.
class Question(models.Model):
question_text = models.CharField(max_length=200)
pub_date = models.DateTimeField('date published')
Each class inherits from models.Model
becomes a table inside the SQL database unless explicitly marked as abstract. The Question
model becomes <app_name>_question
table in the database. question_text
and pub_date
become columns in the table. The properties of the each field are declared by instantiating the respective class. Below is the method resolution order for CharField
.
In [5]: models.CharField.mro()
Out[5]:
[django.db.models.fields.CharField,
django.db.models.fields.Field,
django.db.models.query_utils.RegisterLookupMixin,
object]
CharField
inherits Field
and Field
inherits RegisterLookUpMixin
.
High-level role of Field class
The role of field class is to map type of the field to SQL database type.
Serialization - to convert the Python object into relevant SQL database value.
DeSerialization - to convert the SQL database value into Python object.
Check declared validations at the field level and built-in checks before serializing the data. For example, in a
PositiveIntegerField
the value should be greater than zero - built-in constraint.
Structure of Field Class
# Find out all the classes inheriting the Field
In [7]: models.Field.__subclasses__()
Out[7]:
[django.db.models.fields.BooleanField,
django.db.models.fields.CharField,
django.db.models.fields.DateField,
django.db.models.fields.DecimalField,
django.db.models.fields.DurationField,
django.db.models.fields.FilePathField,
django.db.models.fields.FloatField,
django.db.models.fields.IntegerField,
django.db.models.fields.IPAddressField,
django.db.models.fields.GenericIPAddressField,
django.db.models.fields.TextField,
django.db.models.fields.TimeField,
django.db.models.fields.BinaryField,
django.db.models.fields.UUIDField,
django.db.models.fields.json.JSONField,
django.db.models.fields.files.FileField,
django.db.models.fields.related.RelatedField,
django.contrib.postgres.search.SearchVectorField,
django.contrib.postgres.search.SearchQueryField,
fernet_fields.fields.EncryptedField,
enumchoicefield.fields.EnumChoiceField,
django.contrib.postgres.fields.array.ArrayField,
django.contrib.postgres.fields.hstore.HStoreField,
django.contrib.postgres.fields.ranges.RangeField]
Here fernet_fields
is a third-party library which implements the EncryptedField
by inheriting the Field
class. Also these are high level fields. For example, Django implements other high-level fields which inherit the above fields.
For example, EmailField
inherits the CharField
.
In [10]: models.CharField.__subclasses__()
Out[10]:
[django.db.models.fields.CommaSeparatedIntegerField,
django.db.models.fields.EmailField,
django.db.models.fields.SlugField,
django.db.models.fields.URLField,
django.contrib.postgres.fields.citext.CICharField,
django_extensions.db.fields.RandomCharField,
django_extensions.db.fields.ShortUUIDField]
Here is the Field
class initializer signature
In [11]: models.Field?
Init signature:
models.Field(
verbose_name=None,
name=None,
primary_key=False,
max_length=None,
unique=False,
blank=False,
null=False,
db_index=False,
rel=None,
default=<class 'django.db.models.fields.NOT_PROVIDED'>,
editable=True,
serialize=True,
unique_for_date=None,
unique_for_month=None,
unique_for_year=None,
choices=None,
help_text='',
db_column=None,
db_tablespace=None,
auto_created=False,
validators=(),
error_messages=None,
)
The Field
initializer contains 22 arguments. Most of the arguments are related to SQL database column properties and rest of the arguments are for Django admin and model forms.
For example, Django provides Admin interface to browse the database records and allows you to edit. blank
parameter determines whether the field is required while filling up data in the admin interface and custom form. help_text
field is used while display the form.
The most commonly used fields are max_length, unique, blank, null, db_index, validators, default, auto_created
. null
attribute is a boolean type when set to True
, the allows null value while saving to the database. db_index=True
created a B-Tree
index on the column. default
attribute stores the default value passed on to the database, when the value for the field is missing.
validators
attribute contains list of validators passed on by the user and Django's internal validators. The function of the validator is to determine the value is valid or not. For example, in our question_text
field declaration max_length
is set to 200
. When the field value is greater than 200, Django raises ValidationError
. max_length
attribute is useful only for text field and MaxLengthValidator
will be missing in non-text fields.
In [29]: from django.core.exceptions import ValidationError
In [30]: def allow_odd_validator(value):
...: if value % 2 == 0:
...: raise ValidationError(f'{value} is not odd number')
...:
In [31]: int_field = models.IntegerField(validators=[allow_odd_validator])
In [32]: int_field.validators
Out[32]:
[<function __main__.allow_odd_validator(value)>,
<django.core.validators.MinValueValidator at 0x1305fdac0>,
<django.core.validators.MaxValueValidator at 0x1305fda30>]
In [33]: # let's look into question_text field validators
In [38]: question_text.validators
Out[38]: [<django.core.validators.MaxLengthValidator at 0x12e767fa0>]
As long as the validator function or custom class doesn't raise exception, the value is considered as valid
.
The details of each field can found in the Django model field reference
Field Methods
In [41]: import inspect
In [44]: len(inspect.getmembers(models.Field, predicate=inspect.isfunction))
Out[44]: 59
In [45]: len(inspect.getmembers(models.Field, predicate=inspect.ismethod))
Out[45]: 6
The Field
class consists of (along with inherited ones) 65 methods. Let's look at some of the important ones.
to_python
to_python
method is responsible to convert the value passed on to the model during intialization. For example, to_python
for IntegerField
will convert the value to Python integer. The original value could be string
or float
. Every field will override to_python
method. Here is an example of to_python
method invocation on an IntegerField
.
In [46]: int_field.to_python
Out[46]: <bound method IntegerField.to_python of <django.db.models.fields.IntegerField>>
In [47]: int_field.to_python('23')
Out[47]: 23
In [48]: int_field.to_python(23)
Out[48]: 23
In [49]: int_field.to_python(23.56)
Out[49]: 23
get_db_prep_value
get_db_prep_value
method is responsible to convert Python value to SQL database specific value. Each field may have a different implementation depending on field type. For example, Postgres
has a native UUID
type, whereas in SQLite
and MySQL
Django uses varchar(32)
. Here is the implementation for get_db_prep_value
from UUIDField
.
def get_db_prep_value(self, value, connection, prepared=False):
if value is None:
return None
if not isinstance(value, uuid.UUID):
value = self.to_python(value)
if connection.features.has_native_uuid_field:
return value
return value.hex
connection
is a Database Connection or Wrapper object of underlying database. Below is an example output from a Postgres Connection
and SQLite Connection
for uuid field check.
In [50]: from django.db import connection
...:
In [51]: connection
Out[51]: <django.utils.connection.ConnectionProxy at 0x10e3c8970>
In [52]: connection.features
Out[52]: <django.db.backends.postgresql.features.DatabaseFeatures at 0x1236a6a00>
In [53]: connection.features.has_native_uuid_field
Out[53]: True
In [1]: from django.db import connection
In [2]: connection
Out[2]: <django.utils.connection.ConnectionProxy at 0x10fe3b4f0>
In [3]: connection.features
Out[3]: <django.db.backends.sqlite3.features.DatabaseFeatures at 0x110ba5d90>
In [4]: connection.features.has_native_uuid_field
Out[4]: False
One thing to note, Django uses psycopg2
driver for Postgres and it will take care of handling UUID specific to Postgres because UUID Python object needs to be converted to string or bytes before sending to the Postgres server.
Similar to get_db_prep_value
, get_prep_value
which converts Python value to query value
.
formfield
Django supports ModelForm
which is one to one mapping of HTML form to Django model. The Django admin uses ModelForm
. The form consists of several fields. Each field in the form maps to field in the model. So Django can automatically construct the form with a list of validators from the model field.
Here is the implementation for the UUIDField.
def formfield(self, **kwargs):
return super().formfield(**{
'form_class': forms.UUIDField,
**kwargs,
})
When you create a custom database field, you need to create a custom form field to work with Django admin and pass it as an argument to super class method.
deconstruct
deconstruct
method returns value for creating an exact copy of the field. The method returns a tuple with 4 values.
- The first value is the
name
of the field passed during initialisation. The default value isNone
. - The import path of the field.
- The list of positonal arguments passed during the field creation.
- The dictionary of keyword arguments passed during the field creation.
In [62]: # Let's see the question_text deconstruct method return value
In [63]: question_text.deconstruct()
Out[63]: (None, 'django.db.models.CharField',
[], {'max_length': 200})
In [65]: # let's create a new integer field with a name
In [66]: int_field = models.IntegerField(name='int_field', validators=[allow_odd_validator])
In [67]: int_field.deconstruct()
Out[67]:
('int_field',
'django.db.models.IntegerField',
[],
{'validators': [<function __main__.allow_odd_validator(value)>]})
In [68]: models.IntegerField(**int_field.deconstruct()[-1])
Out[68]: <django.db.models.fields.IntegerField>
In [69]: int_2_field = models.IntegerField(default=2)
In [70]: int_2_field.deconstruct()
Out[70]: (None, 'django.db.models.IntegerField',
[], {'default': 2})
Also when you implement a custom field, you can override the deconstruct method. Here is the deconstruct
implementation for UUIDField
.
def deconstruct(self):
name, path, args, kwargs = super().deconstruct()
del kwargs['max_length']
return name, path, args, kwargs
init
__init__
method is a good place to override some of the default values. For example, UUIDField
max_length should always be 32
irrespective of the value passed on. In the decimal field, max_digits
can be modified during initialization.
Here is the UUIDField
initializer method implementation.
def __init__(self, verbose_name=None, **kwargs):
kwargs['max_length'] = 32
super().__init__(verbose_name, **kwargs)
db_type
db_type
method takes Django connection as an argument and returns the database specific implementation type for this field. The method takes connection as an argument. Here is the output of db_type for Postgres and SQLite.
In [72]: # Postgres
In [73]: uuid_field = models.UUIDField()
In [74]: uuid_field.db_type(connection)
Out[74]: 'uuid'
In [8]: # Sqlite
In [9]: uuid_field = models.UUIDField()
In [10]: uuid_field.rel_db_type(connection)
Out[10]: 'char(32)'
get_internal_type
method returns internal
Python type which is companion to the db_type
method. In practice, Django fields type and database field mapping is maintained as class variable in DatabaseWrapper
. You can find, Django fields and Postgres fields mapping in backends module. Below is the mapping taken from source code.
class DatabaseWrapper(BaseDatabaseWrapper):
vendor = 'postgresql'
display_name = 'PostgreSQL'
# This dictionary maps Field objects to their associated PostgreSQL column
# types, as strings. Column-type strings can contain format strings; they'll
# be interpolated against the values of Field.__dict__ before being output.
# If a column type is set to None, it won't be included in the output.
data_types = {
'AutoField': 'serial',
'BigAutoField': 'bigserial',
'BinaryField': 'bytea',
'BooleanField': 'boolean',
'CharField': 'varchar(%(max_length)s)',
'DateField': 'date',
'DateTimeField': 'timestamp with time zone',
'DecimalField': 'numeric(%(max_digits)s, %(decimal_places)s)',
'DurationField': 'interval',
'FileField': 'varchar(%(max_length)s)',
'FilePathField': 'varchar(%(max_length)s)',
'FloatField': 'double precision',
'IntegerField': 'integer',
'BigIntegerField': 'bigint',
'IPAddressField': 'inet',
'GenericIPAddressField': 'inet',
'JSONField': 'jsonb',
'OneToOneField': 'integer',
'PositiveBigIntegerField': 'bigint',
'PositiveIntegerField': 'integer',
'PositiveSmallIntegerField': 'smallint',
'SlugField': 'varchar(%(max_length)s)',
'SmallAutoField': 'smallserial',
'SmallIntegerField': 'smallint',
'TextField': 'text',
'TimeField': 'time',
'UUIDField': 'uuid',
}
get_internal_type
values are keys and values are Postgres field names.
I have skipped the implementation of the rest of the methods like __reduce__
and check
. You can go through the source code of Django fields in GitHub and also you will find class variables and private methods usages.
Django documentation has an excellent page on how-to write custom model field.
Summary
-
models.Field
is the root of all the model fields. - Field initializer takes configuration details like
name, default, db_index, null
for the database columns andblank, help_text
for non-column features like Django model form and Django admin. -
__init__
method in the child class can override the user passed value and(or) set custom default value. -
validators
attribute in the field contains the user-defined validators and default validators specific to the field. - Every field needs to implement a few methods to work with the specific databases. Some of the useful methods are
to_python, get_db_prep_value, get_prep_value, deconstruct, formfield, db_type
. - Django
Connection
object or wrapper contains details and features of the underlying the database.
Top comments (1)
Nice