<h1>Odoo AWS S3 module</h1> <p>The basis of the module is to allow the exact same structure of the odoo filestore inside an S3 bucket. The default behaviour is always to consider the unavailabily of an S3 connection. In all cases it will default to the flesystem. This makes migrating from filestore to S3 seamless. Just install as a server wide module or even in one only database.</p> <p>After the propper key is loaded then the server will start witing new files to S3, if available. To migrate just use the aws cli tool and copy all files in filestore to the respective bucket, odoo will give priority to S3 bucket.</p> <h2>Requirements</h2> <ul> <li>boto3 - v1.8.9</li> <li>botocore - v1.11.9</li> </ul> <p><code>bash $&gt; pip install -r requirements.txt $&gt; aws configure </code></p> <p>This last command will ask you for the ASW credentials and secret and will generate a <strong>default</strong> profile.</p> <h2>Installation</h2> <p>Start the Odoo server or use an existing instance. Update modules list and install <em>odoo_s3</em> module. Activate the developer model and find in:</p> <ul> <li>Settings &gt;&gt; Technical &gt;&gt; Parameters &gt;&gt; System Parameters (find <strong>ir_attachment.location</strong>)</li> </ul> <p>The default should be: <em>s3://profile:default@testodoofs1</em></p> <p>To move the filestore to S3 go to:</p> <ul> <li>Settings &gt;&gt; General Settings (find <strong>AWS S3 Storage</strong>)</li> </ul> <p>Set the S3 profile and bucket and check &#39;Load S3 with existing filestore?&#39; and Apply.</p> <p>Or if you prefer you can order this using xml-rpc lib:</p> <p>```python</p> <h1>-<em>- coding: utf-8 -</em>-</h1> <p>import xmlrpclib</p> <h1>Instance Credentials</h1> <p>dbname = &#39;v10<em>odoo</em>s3&#39; user = &#39;admin&#39; pwd = &#39;admin&#39; host = &#39;localhost&#39; port = 8069 protocol = &#39;http&#39;</p> <p>com = xmlrpclib.ServerProxy(&quot;%s://%s:%s/xmlrpc/common&quot; % (protocol, host, port)) uid = com.login(dbname, user, pwd) sock = xmlrpclib.ServerProxy(&quot;%s://%s:%s/xmlrpc/object&quot; % (protocol, host, port))</p> <p>config<em>id = sock.execute(dbname, uid, pwd, &#39;base.config.settings&#39;, &#39;create&#39;, [], { &#39;s3</em>profile&#39;: &#39;default&#39;, &#39;s3<em>bucket&#39;: &#39;testodoofs1&#39;, &#39;s3</em>load&#39;: True }) res = sock.execute(dbname, uid, pwd, &#39;base.config.settings&#39;, &#39;execute&#39;, [config_id]) ```</p> <p>Also recommend using on your odoo.conf file on server wide modules:</p> <p><code>bash server_wide_modules = odoo_s3,web,web_kanban </code></p> <h3>What will happen</h3> <p>Odoo will install without the s3 filestore and after initialization will move all files to the bucket. This will happen using the bucket<em>name in odoo</em>s3/data/filestore_data.xml</p> <h2>Changelog</h2> <ul> <li>added recovery of trashed files if there is a failed read</li> <li>changed the dependency from awscli for initial move of the fs</li> </ul> <h2>Maintenance</h2> <p>To check what is the status of the filestore we can run a toll in Odoo shell environment:</p> <p>```bash IPython 5.6.0 -- An enhanced Interactive Python. ? -&gt; Introduction and overview of IPython&#39;s features. %quickref -&gt; Quick reference. help -&gt; Python&#39;s own help system. object? -&gt; Details about &#39;object&#39;, use &#39;object??&#39; for extra details.</p> <p>In [1]: res<em>list, totals = env[&#39;ir.attachment&#39;].search([]).check</em>s3<em>filestore() 2018-09-12 14:41:10,962 3522 INFO v10</em>odoo<em>s3 botocore.credentials: Found credentials in shared credentials file: ~/.aws/credentials 2018-09-12 14:41:17,167 3522 ERROR v10</em>odoo<em>s3 odoo.addons.odoo</em>s3.models.ir<em>attachment: S3: _file</em>read was not able to read from S3 or other filestore key:v10<em>odoo</em>s3/91/91c1ab69ca4a6c3c5e9c32187cb975d19b194b93 2018-09-12 14:41:29,978 3522 ERROR v10<em>odoo</em>s3 odoo.addons.odoo<em>s3.models.ir</em>attachment: S3: <em>file</em>read was not able to read from S3 or other filestore key:v10<em>odoo</em>s3/6b/6bc87910870d5ec831c3adbf4df22c5bbed1fe31 2018-09-12 14:41:30,375 3522 ERROR v10<em>odoo</em>s3 odoo.addons.odoo<em>s3.models.ir</em>attachment: S3: <em>file</em>read was not able to read from S3 or other filestore key:v10<em>odoo</em>s3/cc/ccd41799069ae21b845f931369eca4f9fdc76ba5 2018-09-12 14:41:30,682 3522 ERROR v10<em>odoo</em>s3 odoo.addons.odoo<em>s3.models.ir</em>attachment: S3: <em>file</em>read was not able to read from S3 or other filestore key:v10<em>odoo</em>s3/4f/4f21892e89f55600f30723c55094ac8658a48bab</p> <p>In [2]: totals Out[2]: {&#39;lost_count&#39;: 4}</p> <p>In [3]: filter(lambda x: x[&#39;s3<em>lost&#39;]==True, res</em>list) Out[3]: [{&#39;error&#39;: &#39;Not Found&#39;, &#39;fname&#39;: u&#39;91/91c1ab69ca4a6c3c5e9c32187cb975d19b194b93&#39;, &#39;name&#39;: u&#39;/mail/static/src/less/web.assets<em>backend/followers.less.css&#39;, &#39;s3</em>lost&#39;: True}, {&#39;error&#39;: &#39;Not Found&#39;, &#39;fname&#39;: u&#39;6b/6bc87910870d5ec831c3adbf4df22c5bbed1fe31&#39;, &#39;name&#39;: u&#39;Screenshot-20180904222459-481x628.png&#39;, &#39;s3<em>lost&#39;: True}, {&#39;error&#39;: &#39;Not Found&#39;, &#39;fname&#39;: u&#39;cc/ccd41799069ae21b845f931369eca4f9fdc76ba5&#39;, &#39;name&#39;: u&#39;Menu</em>009.png&#39;, &#39;s3<em>lost&#39;: True}, {&#39;error&#39;: &#39;Not Found&#39;, &#39;fname&#39;: u&#39;4f/4f21892e89f55600f30723c55094ac8658a48bab&#39;, &#39;name&#39;: u&#39;Screenshot-20180903121333-1348x741.png&#39;, &#39;s3</em>lost&#39;: True}]</p> <p>In [4]: env.cr.commit() # to make sure that field s3_lost gets updated</p> <p>```</p> <p>In the described scenario we will have the s3_lost field updated in the database and that allows us to drop and recreate the missing assets, if that&#39;s the situation.</p> <p>You can also find other results using the filter expression or sorting and grouping this dictionary.</p>