Django REST Framework: File Tales

Here are the list of file related cases

1. Database Structure:

Typically it's best to have a base class that abstracts file properties:

class Document(models.Model):
    name = models.CharField(max_length=255, null=True, blank=True)
    path = models.CharField(max_length=255)

    class Meta:
        abstract = True


class Invoice(Document):
    pass

The above implies that a document may have a name and a path to the actual file.

But in development, over time, may arrive a situation where a document may require multiple files. So proactively, it is better to restructure as follows:

class Document(models.Model):
    name = models.CharField(max_length=255, null=True, blank=True)

    class Meta:
        abstract = True

class File(models.Model):
    path = models.CharField(max_length=255)

    class Meta:
        abstract = True


class Invoice(Document):
    pass

class InvoiceFile(File):
    invoice = models.ForeignKey(Invoice, related_name='files',
                                on_delete=models.CASCADE)

2. Serialization:

Let's set context where we have sellers and buyers, and all have invoices.

class Product(models.Model):
    name = models.CharField(max_length=255)


# seller
class SellerInvoice(Invoice):
    product = models.OneToOneField(Product, on_delete=models.CASCADE)

class SellerInvoiceFile(File):
    invoice = models.ForeignKey(SellerInvoice, related_name='seller_files',
                                on_delete=models.CASCADE)

# buyer
class BuyerInvoice(Invoice):
    product = models.OneToOneField(Product, on_delete=models.CASCADE)

class BuyerInvoiceFile(File):
    invoice = models.ForeignKey(BuyerInvoice, related_name='buyer_files',
                                on_delete=models.CASCADE)

Under certain business circumstances, a buyer invoice may need to be the same seller invoice. ie, in code terms, the buyer invoice is a replica of the seller invoice.

class BuyerInvoiceFileSerializer(BaseSerializer):

    class Meta:
        model = BuyerInvoiceFile
        fields = '__all__'
        extra_kwargs = {
            'invoice': {
                'write_only': True,
                'required': False
            }
        }

class BuyerInvoiceSerializer(BaseSerializer):
    is_same_as_seller = serializers.BooleanField(write_only=True,
                                                   default=False)
    buyer_files = BuyerInvoiceFileSerializer(many=True)

    class Meta:
        model = BuyerInvoice
        fields = '__all__'

    def create(self, validated_data):
        product = validated_data['product']

        if validated_data['is_same_as_seller']:
            validated_data['name'] = product.sellerinvoice.name

        return super().create(validated_data)

The easy part.

Now comes the thinking part: How to copy the files? It is evident that we must replicate the files rather than refer to the same.

We can add something like this:

        if validated_data['is_same_as_seller']:
            validated_data['name'] = product.sellerinvoice.name
            file_data = BuyerInvoiceFileSerializer(
                data=SellerInvoiceFileSerializer(
                    product.sellerinvoice.seller_files.all(),
                    fields=["path"], many=True).data, 
                many=True)
            file_data.is_valid(raise_exception=True)
            validated_data['buyer_files'] = file_data.data

This successfully does get seller invoice files, validate, but will not create the files for buyer invoice.

One way to overcome this to is to add files to initial_data as follows:

            # validated_data['buyer_files'] = file_data.data
            self.initial_data.update({
                'buyer_files': product.copy_seller_invoice_files(
                    list(file_data.data))
            })

This will create the document and followed by its files.

3. Storage Upload:

• Need to download the source file and upload it back to a different destination path.

• Always need to add a random string to file names. Typically to avoid file overwrite, the case when 2 different devices logged in as the same user and at the same time trying to upload a file. The timestamp is not a good option for the random string.

    def copy_seller_invoice_files(self, data):
        result = []

        for index, file in enumerate(data):
            file_name = f'{self.id}_buyer_invoice_{index}_{str(uuid.uuid4()).upper()[:8]}.pdf'
            new_path = f'/tmp/{file_name}'
            aws_path = f'{self.name}/{file_name}'
            s3.download_object(
                new_path,
                settings.AWS_STORAGE_BUCKET_NAME,
                file['path'])

            try:
                s3.upload_object(
                    new_path,
                    settings.AWS_STORAGE_BUCKET_NAME,
                    aws_path)
            except Exception as e:
                raise APIValidationError(detail='Unable to Upload Files')

            result.append({'document': aws_path})
            os.remove(new_path)
        return result

Repository:

Thanks.