Binary Encryption Issue

I have added below lines in nuxeo.conf nuxeo.core.binarymanager=org.nuxeo.ecm.core.blob.binary.AESBinaryManager nuxeo.core.binarymanager_key=password=mypassword

I was already having some files already available in binaries folder which were in plain format (not encrypted). So i tried to create document with encryption then i am getting Invalid Type (Bad Magic) exception . it is because the MD5 which is getting generated with encryption technique is already available in plain format (with out encryption) in binary folder.

and it is giving issue in below line:- *protected void decrypt(InputStream in, OutputStream out) throws IOException { byte[] magic = new byte[FILE_MAGIC.length]; IOUtils.read(in, magic); if (!Arrays.equals(magic, FILE_MAGIC)) { throw new IOException(“Invalid file (bad magic)“); } *

Resolution on my end:-

I have deleted content of binary/data folder and deleted the folder structure from webUi ex: XXX/workspaces/YYY and created doc with encryption it is working fine.

PROBLEM:

So I want to know is there any possibility that we can bring AESEncryption on demand . so it will not hamper already generated file in plain format. If i want to remove encryption i can remove it. and is there any possibility i can apply encryption on tenant specific.

0 votes

1 answers

1428 views

ANSWER



Hello,

adding the two lines to nuxeo.conf is the generic way to implement binary encryption, but it is the least flexible, since encryption is applied to all binaries without exception. This causes problems if there are already “plain” binaries, or if you don't want to encrypt all future binaries.

In this case, you will need to define BlobDispatchers and BlobProviders. I recommend you to read the following: https://doc.nuxeo.com/nxdoc/file-storage/

When you do something with a document (like creating it or download it), the BlobManager will decide what to do. The BlobManager will evaluate the conditions over all the BlobDispatchers, and it will select the desired BlobProvider. For example, you can use a custom property to encrypt binaries:

<extension target="org.nuxeo.ecm.core.blob.DocumentBlobManager" point="configuration">
    <blobdispatcher>
        <class>org.nuxeo.ecm.core.blob.DefaultBlobDispatcher</class>
        <property name="custom:encrypted=true">encrypted</property>
        <property name="default">default</property>
    </blobdispatcher>
</extension>

In the “name” inside the “property” tag you put the condition that must be true in order to use that BlobProvider. Remember not to forget the “default” BlobDispatcher pointing to the “default” BlobProvider! And then just define the BlobProvider:

<extension target="org.nuxeo.ecm.core.blob.BlobManager" point="configuration">
    <blobprovider name="encrypted">
        <class>org.nuxeo.ecm.core.blob.binary.AESBinaryManager</class>
        <property name="key">password=secret</property>
    </blobprovider>
</extension>

I hope it helps!

Regards.

1 votes



Rodri I created one plugin and installed it on my local instance still getting same error. Below is the extension code.

&lt;?xml version=&quot;1.0&quot;?&gt;
&lt;component name=&quot;com.softcell.dms.encryption&quot;&gt;

  &lt;extension target=&quot;org.nuxeo.ecm.core.blob.DocumentBlobManager&quot; point=&quot;configuration&quot;&gt;
    &lt;!-- 
    You might find some help here:
    https://explorer.nuxeo.com/nuxeo/site/distribution/latest/viewExtensionPoint/org.nuxeo.ecm.core.blob.DocumentBlobManager%2d%2dconfiguration
    --&gt;

    &lt;blobdispatcher&gt;
      &lt;class&gt;org.nuxeo.ecm.core.blob.DefaultBlobDispatcher&lt;/class&gt;
      &lt;property name=&quot;dc:source=secret&quot;&gt;encrypted&lt;/property&gt;
      &lt;property name=&quot;default&quot;&gt;default&lt;/property&gt;
    &lt;/blobdispatcher&gt;
  &lt;/extension&gt;

  &lt;extension target=&quot;org.nuxeo.ecm.core.blob.BlobManager&quot; point=&quot;configuration&quot;&gt;
    &lt;blobprovider name=&quot;encrypted&quot;&gt;
      &lt;class&gt;org.nuxeo.ecm.core.blob.binary.AESBinaryManager&lt;/class&gt;
      &lt;property name=&quot;key&quot;&gt;password=password&lt;/property&gt;
    &lt;/blobprovider&gt;
  &lt;/extension&gt;

&lt;/component&gt;
04/29/2020

Rodri I check the source code of Nuxeo and I came to know that it is throwing error on below code

protected void decrypt(InputStream in, OutputStream out) throws IOException {
        byte[] magic = new byte[FILE_MAGIC.length];
        IOUtils.read(in, magic);
        if (!Arrays.equals(magic, FILE_MAGIC)) {
            throw new IOException(&quot;Invalid file (bad magic)&quot;);
        }

where, protected static final byte[] FILE_MAGIC = new byte[] { 'N', 'U', 'X', 'E', 'O', 'C', 'R', 'Y', 'P', 'T' };

04/29/2020

You have to take in mind that, with the code above, you are saying that only documents with the property "dc:source" equals to "secret" are encrypted. This means that, if you have some documents with this dc:source=secret but the binary it is not encrypted, nuxeo will throw an error (the piece of code you pasted in the last message is where nuxeo decrypt files. It throws that error when the file is not encrypted). Make sure all the files with dc:source=secret are actually encrypted.
04/29/2020

Hi Rodri ,

Let's say i have created a new property or metadata as dms:encrypted. and configured as <property name="dms:encrypted=true">encrypted</property>

so if i am creating a document { "entity-type":"document", "name":"monkey", "type":"Picture", "path":"/FVSG/SME/LAP", "properties":{

  &quot;dms:encrypted&quot;:true,

"file:content":{

     &quot;upload-batch&quot;:&quot;batchId-39c3877e-f435-4394-a338-036ee40e7456&quot;,
     &quot;upload-fileId&quot;:&quot;0&quot;
  }

} }

So by this means i understand that this document should be encrypted while created and decrypted the same way while downloading.

so if i am creating the doc it gives invalid type(Bad Magic) as if it a new property it does not having any doc which is in plain format

04/30/2020

Hello.

If you define the "encrypted" BlobProvider with the "dms:encrypted=true" condition in the BlobDispatcher, then the document you are trying to create (with "dms:encrypted":true) should be encrypted. If the document is not being encrypted, it will be because there is something wrong with the BlobProvider or BlobDispatcher definition. If you are not working with Nuxeo Studio, did you add the xml file contribution to the MANIFEST file?

If the document is created correctly but it is giving errors while visualizing it, get the digest of the document (you can check if with the "file:content:digest" property) and search for it in the filesystem. Open it with a text editor like Notepad++ and check if the first words are "NUXEOCRYPT". If they are, the binary is encrypted. If not, it will confirm us there is something wrong with the BlobProvider/BlobDispatcher.

Anyway, I have seen you are using BatchUpload. I am not sure if this configuration is working with BatchUploading, as you are storing the file in Nuxeo before knowing if it should be encrypted or not.

Regards.

04/30/2020

Hi Rodri, Can you please reproduce this issue on your end. I have checked when i create doc it create tmp file named create_312334y34 which have NUXEOCRPYT in it . and put the file /data/binaries/be/f7 which is not having NUXEOCYPT. It iterates in getBinary method around 3 times may be because of thumbnail and small image creation but . when the decrypt method calls i am not having any while in temp folder which is having NUXECRYPT. that means it is not putting the same encrypted file in to binaries/data folder.
04/30/2020

Hi Rodri , I am able to encrypt the documents by creating a different repository for encrypted documents only. But I am still getting 'default' repository in the URLs created by Nuxeo for uploaded document.

You can get more details on below question. https://answers.nuxeo.com/general/q/ec0c1d8451d740b1bac9b228f5826ffa/Not-getting-correct-repository-name-in-document-URLs

05/06/2020

It seems there's a confusion in these comments about Repository vs Blob Provider. A Repository is the toplevel entity in which documents (and their metadata) are stored. A Blob Provider is an entity that knows how to store and retrieve blobs. By default there is one Blob Provider per Repository, but using a Blob Dispatcher you can associate rules to a Repository to dispatch blobs to one of several Blob Providers. See https://doc.nuxeo.com/nxdoc/file-storage-configuration/#blob-dispatcher for more.
05/06/2020