[Mayan EDMS: 2006] mayan stateless cluster / docker swarm

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[Mayan EDMS: 2006] mayan stateless cluster / docker swarm

Tony Nys
I want to set up 2 mayan servers running the docker image
Each server will be connected to the same database server (postgresql) and the same external blob storage (eg. Azure)

WIll there be any issue/concern setting up such a cluster / docker-swarm ?

--

---
You received this message because you are subscribed to the Google Groups "Mayan EDMS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

[Mayan EDMS: 2079] Re: mayan stateless cluster / docker swarm

rosarior
Administrator
Once concern that comes to mind is making sure both instances have access to the same locks. For this you need to switch to a shared lock backend. Mayan includes one such backend using the database. In your local.py change (both instances) the config option LOCK_MANAGER_DEFAULT_BACKEND to 'lock_manager.backends.model_lock.ModelLock'. This will cause the instances to write to a common database model to reserve and release resources so that both don't try to do OCR to the same document at the same time for example.

The other concern is data propagation delays. You can increase the default of two seconds to a higher number with COMMON_DB_SYNC_TASK_DELAY=3 or higher number. This will cause background tasks to wait the specified number of seconds before trying to access data that is still being propagated from one host to the other. Increase this number if you background tasks logs show that they are not finding a document that you know you uploaded.

The final concern is load balancing at the frontend to make sure requests are distributed in a fair manner between the two instances.

I look forward to hear from your experiences and any setup you used.

On Friday, August 11, 2017 at 2:39:22 AM UTC-4, Tony Nys wrote:
I want to set up 2 mayan servers running the docker image
Each server will be connected to the same database server (postgresql) and the same external blob storage (eg. Azure)

WIll there be any issue/concern setting up such a cluster / docker-swarm ?

--

---
You received this message because you are subscribed to the Google Groups "Mayan EDMS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

[Mayan EDMS: 2096] Re: mayan stateless cluster / docker swarm

Tony Nys
In reply to this post by Tony Nys
thanks for the info, as soon as I have time to test a swarm , I will let you know
I still need to figure out a way to put the log files on the Host somehow, so they can be linked into eg. the blob storage, etc...


PS. still struggling with local.py on docker (OSX): once it is created, the log file shows that it cannot find media settings:

mayan.settings <544> [ERROR] "<module>() Error importing user's local.py; No module named media.settings.local"




On Friday, August 11, 2017 at 8:39:22 AM UTC+2, Tony Nys wrote:
I want to set up 2 mayan servers running the docker image
Each server will be connected to the same database server (postgresql) and the same external blob storage (eg. Azure)

WIll there be any issue/concern setting up such a cluster / docker-swarm ?

--

---
You received this message because you are subscribed to the Google Groups "Mayan EDMS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.