[Mayan EDMS: 2052] worker and beat can't write to /tmp/mayan_locks.tmp using the advanced deployment instructions

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[Mayan EDMS: 2052] worker and beat can't write to /tmp/mayan_locks.tmp using the advanced deployment instructions

Hans Fritz
In the advanced deployment instructions (https://mayan.readthedocs.io/en/2.7/topics/deploying.html), the current setup starts UWSGI as root, and beat + worker as the www-data user.

This seems to be problematic, at least on Ubuntu 16.04. The lockfile at /tmp/mayan_locks.tmp is mode 0644 and belongs to root:root. Because of that, the worker and beat fail to start since they run as www-data and can't access the lock.

I'm seeing this in the wroker-stderr log:
Traceback (most recent call last):
  File "/usr/share/mayan-edms/bin/mayan-edms.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/share/mayan-edms/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
    utility.execute()
  File "/usr/share/mayan-edms/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 341, in execute
    django.setup()
  File "/usr/share/mayan-edms/local/lib/python2.7/site-packages/django/__init__.py", line 27, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/share/mayan-edms/local/lib/python2.7/site-packages/django/apps/registry.py", line 85, in populate
    app_config = AppConfig.create(entry)
  File "/usr/share/mayan-edms/local/lib/python2.7/site-packages/django/apps/config.py", line 116, in create
    mod = import_module(mod_path)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/share/mayan-edms/local/lib/python2.7/site-packages/mayan/apps/checkouts/apps.py", line 29, in <module>
    from .tasks import task_check_expired_check_outs  # NOQA
  File "/usr/share/mayan-edms/local/lib/python2.7/site-packages/mayan/apps/checkouts/tasks.py", line 8, in <module>
    from lock_manager.runtime import locking_backend
  File "/usr/share/mayan-edms/local/lib/python2.7/site-packages/mayan/apps/lock_manager/runtime.py", line 5, in <module>
    locking_backend = import_string(setting_backend.value)
  File "/usr/share/mayan-edms/local/lib/python2.7/site-packages/django/utils/module_loading.py", line 20, in import_string
    module = import_module(module_path)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/usr/share/mayan-edms/local/lib/python2.7/site-packages/mayan/apps/lock_manager/backends/file_lock.py", line 25, in <module>
    open(lock_file, 'a').close()
IOError: [Errno 13] Permission denied: u'/tmp/mayan_locks.tmp'

A workaround is to run beat and worker under the root user (by changing the user in the config files), but that probably isn't a good thing since they're user facing.

Am I overlooking something, or is it an error in the instructions?

--

---
You received this message because you are subscribed to the Google Groups "Mayan EDMS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

[Mayan EDMS: 2053] Re: worker and beat can't write to /tmp/mayan_locks.tmp using the advanced deployment instructions

Hans Fritz
This was my mistake! I forgot one line in a config file that launched uwsgi as root instead of www-data, thus the lockfile was created by root instead of www-data. Once I fixed the config file and deleted the logfile, all was well.

--

---
You received this message because you are subscribed to the Google Groups "Mayan EDMS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

[Mayan EDMS: 2056] Re: worker and beat can't write to /tmp/mayan_locks.tmp using the advanced deployment instructions

Hans Fritz
In reply to this post by Hans Fritz
No this is still a problem after all. The lockfile is created by root and is not readable to www-data. I'm guessing program:mayan-uwsgi creates it (runs as root) and the worker + beat try to write to it but can't.

--

---
You received this message because you are subscribed to the Google Groups "Mayan EDMS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

[Mayan EDMS: 2063] Re: worker and beat can't write to /tmp/mayan_locks.tmp using the advanced deployment instructions

rosarior
Administrator
You are correct. It is an error in the documentation. Under the Docker image everything runs as root to be able to access staging and watch folder files (due to user ownership issues with Docker). For advanced deployments where users have more control makes sense to have everything running under a non privileged user like www-data. Try running the uwsgi process as that user. 

On Thursday, August 31, 2017 at 9:43:07 AM UTC-4, Hans Fritz wrote:
No this is still a problem after all. The lockfile is created by root and is not readable to www-data. I'm guessing program:mayan-uwsgi creates it (runs as root) and the worker + beat try to write to it but can't.

--

---
You received this message because you are subscribed to the Google Groups "Mayan EDMS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

[Mayan EDMS: 2065] Re: worker and beat can't write to /tmp/mayan_locks.tmp using the advanced deployment instructions

Hans Fritz
I did but it fails because /tmp/mayan_locks.tmp is still owned by root. I'm not sure which part of the application generates it. The only workaround I found is to create /tmp/mayan_locks.tmp ahead of time and chown www-data:www-data ahead of time. I'll open an issue on GitLab.

On Thursday, 31 August 2017 16:43:44 UTC-4, Roberto Rosario wrote:
You are correct. It is an error in the documentation. Under the Docker image everything runs as root to be able to access staging and watch folder files (due to user ownership issues with Docker). For advanced deployments where users have more control makes sense to have everything running under a non privileged user like www-data. Try running the uwsgi process as that user. 

On Thursday, August 31, 2017 at 9:43:07 AM UTC-4, Hans Fritz wrote:
No this is still a problem after all. The lockfile is created by root and is not readable to www-data. I'm guessing program:mayan-uwsgi creates it (runs as root) and the worker + beat try to write to it but can't.

--

---
You received this message because you are subscribed to the Google Groups "Mayan EDMS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

[Mayan EDMS: 2066] Re: worker and beat can't write to /tmp/mayan_locks.tmp using the advanced deployment instructions

rosarior
Administrator
Thanks for testing. Please do so to track the issue. Thanks.

On Thursday, August 31, 2017 at 5:58:42 PM UTC-4, Hans Fritz wrote:
I did but it fails because /tmp/mayan_locks.tmp is still owned by root. I'm not sure which part of the application generates it. The only workaround I found is to create /tmp/mayan_locks.tmp ahead of time and chown www-data:www-data ahead of time. I'll open an issue on GitLab.

On Thursday, 31 August 2017 16:43:44 UTC-4, Roberto Rosario wrote:
You are correct. It is an error in the documentation. Under the Docker image everything runs as root to be able to access staging and watch folder files (due to user ownership issues with Docker). For advanced deployments where users have more control makes sense to have everything running under a non privileged user like www-data. Try running the uwsgi process as that user. 

On Thursday, August 31, 2017 at 9:43:07 AM UTC-4, Hans Fritz wrote:
No this is still a problem after all. The lockfile is created by root and is not readable to www-data. I'm guessing program:mayan-uwsgi creates it (runs as root) and the worker + beat try to write to it but can't.

--

---
You received this message because you are subscribed to the Google Groups "Mayan EDMS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

[Mayan EDMS: 2067] Re: worker and beat can't write to /tmp/mayan_locks.tmp using the advanced deployment instructions

Hans Fritz
Opened an issue: https://gitlab.com/mayan-edms/mayan-edms/issues/427

Thanks!

--

---
You received this message because you are subscribed to the Google Groups "Mayan EDMS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

[Mayan EDMS: 2074] Re: worker and beat can't write to /tmp/mayan_locks.tmp using the advanced deployment instructions

rosarior
Administrator
Started a new repository for testing the deployment here: https://gitlab.com/mayan-edms/mayan-edms-vagrant

It is using Vagrant to test a bash script that does the advanced deployment. The bash script that could be run almost verbatim on a production system.

On a side note, if this bash script if made more intelligent by detecting the host OS, the same script could be updated to do the deployment on different OS (Ubuntu, Fedora, Arch, MacOS). 

A further advantage is that it could be executed from the web directly, something like:

wget -O - https://bootstrap.mayan-edms.com | sudo sh


On Thursday, August 31, 2017 at 8:59:09 PM UTC-4, Hans Fritz wrote:
Opened an issue: <a href="https://gitlab.com/mayan-edms/mayan-edms/issues/427" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgitlab.com%2Fmayan-edms%2Fmayan-edms%2Fissues%2F427\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHVolj2LE92BkW1UhoNGAZI13pmaA&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgitlab.com%2Fmayan-edms%2Fmayan-edms%2Fissues%2F427\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHVolj2LE92BkW1UhoNGAZI13pmaA&#39;;return true;">https://gitlab.com/mayan-edms/mayan-edms/issues/427

Thanks!

--

---
You received this message because you are subscribed to the Google Groups "Mayan EDMS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.