Add support for JSONField

[akifd]

We have some data in JSON-field (django.contrib.postgres.fields.JSONField), but this package does not support it. Could it be possible to add this functionality?

[peterfarrell]

@User3759685 Not sure this can actually be implemented. Due to the encryption, all Django-PGCrypto-Fields are stored in the DB as text fields (unlimited character length) including Date, DateTime, etc. The JSON field provided by Django is persisted at the DB as a jsonb field.

You could store encrypted JSON as a text field but you would lose all operators at the database level to search it (__has_key, etc.). Currently, there is nothing stopping you from storing JSON in a TextPGPSymmetricalKeyField and decoding the JSON into a native Python data types.

I question the usefulness of adding direct JSON support in this library. Please let us know if you want to work on a PR for this feature.

[december1981]

I agree it's probably not worth integrating directly, but I needed the basic convenience of json encoding/decoding on top of an encrypted text field (to make code run seamless whether I decide in the end to encrypt the field or not - adapting perhaps only field queries which may be "jsonb aware" or simply raw "text contains" - which latter btw can be very slow over encrypted data).

So here's a gist (JSONTextFieldMixin class based on what Django JSONField does, without the "KeyTransform" gubbins):
https://gist.github.com/december1981/ba2a3bb281f7fd4428a178f1d228e68b

[some1ataplace]

Looks like there might be a solution in the form of a JSONTextFieldMixin class that extends the functionality of TextPGPSymmetricalKeyField to support JSON encoding/decoding. However, if you're looking for an alternative approach, you could consider using a custom database field that extends JSONField and adds encryption functionality.

Here's an example implementation:

from django.db import models
from django.contrib.postgres.fields import JSONField
from pgcrypto_fields import PGPTextField

class EncryptedJSONField(PGPTextField, JSONField):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.validators = []

    def from_db_value(self, value, expression, connection):
        if value is None:
            return value
        return super().from_db_value(value, expression, connection)

    def to_python(self, value):
        if value is None:
            return value
        return super().to_python(value)

    def get_prep_value(self, value):
        if value is None:
            return value
        return super().get_prep_value(value)

In this implementation, EncryptedJSONField extends both PGPTextField and JSONField. It overrides the from_db_value, to_python, and get_prep_value methods to ensure that the encrypted JSON data is properly handled by both the PGPTextField and JSONField functionality.

---

To achieve this, you can create a custom JSONPGPSymmetricalKeyField by subclassing Django's TextField and Django-PGCrypto-Fields' PGPSymmetricalKeyField. Here's a code example:

import json
from django.contrib.postgres.fields import JSONField
from pgcrypto.fields import TextPGPSymmetricalKeyField

class JSONPGPSymmetricalKeyField(JSONField, TextPGPSymmetricalKeyField):
    def from_db_value(self, value, expression, connection):
        decrypted_value = super().from_db_value(value, expression, connection)
        if decrypted_value is not None:
            return json.loads(decrypted_value)
        return decrypted_value

    def to_python(self, value):
        if isinstance(value, str):
            return json.loads(value)
        return value

    def get_prep_value(self, value):
        value = super().get_prep_value(value)
        if value is not None:
            return json.dumps(value)
        return value

    def deconstruct(self):
        name, path, args, kwargs = super().deconstruct()
        return name, "your_project.fields.JSONPGPSymmetricalKeyField", args, kwargs

Add this code snippet to a new file in your project, say your_project/fields.py, and import JSONPGPSymmetricalKeyField in your models to use it:

from django.db import models
from .fields import JSONPGPSymmetricalKeyField

class MyModel(models.Model):
    data = JSONPGPSymmetricalKeyField(passphrase="your_passphrase")

Keep in mind that you'll lose the ability to perform direct database operations on the JSON data. All queries will need to be done at the application level, after loading and decrypting the data.

---

To add support for JSONField in the form of JSONPGPPublicKeyField and JSONPGPSymmetricKeyField, you can create two new custom fields that extend the functionality of JSONField and PGPTextField respectively.

Here's an example implementation:

from django.db import models
from django.contrib.postgres.fields import JSONField
from pgcrypto_fields import PGPTextField

class JSONPGPPublicKeyField(JSONField, PGPTextField):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.validators = []

    def from_db_value(self, value, expression, connection):
        if value is None:
            return value
        return super().from_db_value(value, expression, connection)

    def to_python(self, value):
        if value is None:
            return value
        return super().to_python(value)

    def get_prep_value(self, value):
        if value is None:
            return value
        return super().get_prep_value(value)

class JSONPGPSymmetricKeyField(JSONField, PGPTextField):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.validators = []

    def from_db_value(self, value, expression, connection):
        if value is None:
            return value
        return super().from_db_value(value, expression, connection)

    def to_python(self, value):
        if value is None:
            return value
        return super().to_python(value)

    def get_prep_value(self, value):
        if value is None:
            return value
        return super().get_prep_value(value)

These custom fields can then be used in your models like so:

from django.db import models
from pgcrypto_fields import EncryptedTextField
from .fields import JSONPGPPublicKeyField, JSONPGPSymmetricKeyField

class MyModel(models.Model):
    public_key = JSONPGPPublicKeyField()
    symmetric_key = JSONPGPSymmetricKeyField()
    encrypted_field = EncryptedTextField(key='symmetric_key')

This example shows how you can use the new custom fields in conjunction with the EncryptedTextField from pgcrypto_fields to create an encrypted field that uses the symmetric_key field as the encryption key

---

To add support for JSONField in the form of JSONPGPPublicKeyField and JSONPGPSymmetricKeyField, you will first need to create the corresponding encrypted field classes, as well as their respective form fields and widgets.

Here's some sample code you can use as a base for your implementation:

1. First, start by importing the necessary modules:

import json

from django import forms
from django.conf import settings
from django.contrib.postgres.fields import JSONField
from django.core.exceptions import ValidationError
from django.db.models import Field
from django.utils.translation import gettext_lazy as _

from pgcrypto.fields import *
from .widgets import EncryptedJSONWidget

2. Write a custom class to serialize a dictionary or list into a JSON string:

class JSONEncoder(json.JSONEncoder):
    def default(self, obj):
        if hasattr(obj, "toJson"):
            return self.default(obj.toJson())
        return json.JSONEncoder.default(self, obj)

4. Create the JSONPGPPublicKeyField and JSONPGPSymmetricKeyField classes:

class JSONPGPPublicKeyField(PGPPublicKeyField):
    empty_strings_allowed = False
    form_class = JSONField

    def topython(self, value):
        if value is None:
            return value
        try:
            return json.loads(value)
        except json.JSONDecodeError:
            raise ValidationError(
                ("%s value must be valid JSON.") % self.verbose_name, code="invalid"
            )

    def get_prep_value(self, value):
        value = super().get_prep_value(value)
        return json.dumps(value, cls=JSONEncoder)


class JSONPGPSymmetricKeyField(PGPSymmetricKeyField):
    empty_strings_allowed = False
    form_class = JSONField

    def topython(self, value):
        if value is None:
            return value
        try:
            return json.loads(value)
        except json.JSONDecodeError:
            raise ValidationError(
                ("%s value must be valid JSON.") % self.verbose_name, code="invalid"
            )

    def get_prep_value(self, value):
        value = super().get_prep_value(value)
        return json.dumps(value, cls=JSONEncoder)

5. Create a form field for your JSONPGPPublicKeyField and JSONPGPSymmetricKeyField:

class JSONEncryptedFormField(forms.JSONField):
    widget = EncryptedJSONWidget

    def prepare_value(self, value):
        if isinstance(value, (dict, list)):
            return json.dumps(value, cls=JSONEncoder)
        return value

6. Update the formfield method for each of the custom classes defined in step 3:

class JSONPGPPublicKeyField(...):

    def formfield(self, kwargs):
        # Sets the default form field for the model field
        defaults = {"form_class": JSONEncryptedFormField}
        defaults.update(kwargs)
        return super().formfield(defaults)


class JSONPGPSymmetricKeyField(...):

    def formfield(self, kwargs):
        # Sets the default form field for the model field
        defaults = {"form_class": JSONEncryptedFormField}
        defaults.update(kwargs)
        return super().formfield(defaults)

7. Finally, create the EncryptedJSONWidget in the widgets.py file:

from django.forms.widgets import Textarea


class EncryptedJSONWidget(Textarea):
    def value_from_datadict(self, data, files, name):
        value = data.get(name)
        if not value:
            return None
        try:
            return json.loads(value)
        except json.JSONDecodeError:
            return value

[peterfarrell]

@some1ataplace PRs for a new JSON field are welcome.