[Mayan EDMS: 1958] Server error when viewing indexes

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[Mayan EDMS: 1958] Server error when viewing indexes

Dave Kimmel
Hey there,

I'm using Mayan to store a bunch of my personal documents (bills, etc) and using indexes to organize them.  So far everything works really well, except for the occasional "Server error" when viewing the indexes.  Most of the time I can resolve the error by refreshing the page, but ideally I'd like it to stop happening in the first place.  The problem doesn't occur all the time - sometimes I'll see it once and a refresh makes it disappear, other times I can refresh dozens of times and still see the error.  The problem only occurs on index nodes that show documents, not index nodes that only show child nodes.

When the error occurs, I see the following in the error log:

2017-07-31 08:50:15,983 common.middleware.error_logging <3752> [ERROR] "process_exception() line 17 Exception caught by request middleware; <WSGIRequest: GET '/indexing/instance/node/363/'>, maximum recursion depth exceeded while calling a Python object"
Traceback (most recent call last):
  File "/usr/local/mayan-edms/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/mayan-edms/lib/python2.7/site-packages/django/views/generic/base.py", line 68, in view
    return self.dispatch(request, *args, **kwargs)
  File "/usr/local/mayan-edms/lib/python2.7/site-packages/mayan/apps/document_indexing/views.py", line 240, in dispatch
    self, request, *args, **kwargs
  File "/usr/local/mayan-edms/lib/python2.7/site-packages/mayan/apps/common/mixins.py", line 350, in dispatch
    ).dispatch(request, *args, **kwargs)
  File "/usr/local/mayan-edms/lib/python2.7/site-packages/mayan/apps/common/mixins.py", line 315, in dispatch
    ).dispatch(request, *args, **kwargs)
  File "/usr/local/mayan-edms/lib/python2.7/site-packages/django/views/generic/base.py", line 88, in dispatch
    return handler(request, *args, **kwargs)
  File "/usr/local/mayan-edms/lib/python2.7/site-packages/django/views/generic/list.py", line 159, in get
    self.object_list = self.get_queryset()
  File "/usr/local/mayan-edms/lib/python2.7/site-packages/mayan/apps/common/mixins.py", line 285, in get_queryset
    self.queryset = self.__class__.original_get_queryset(self, *args, **kwargs)

The last two entries repeat for a long time and it ends with:

  File "/usr/local/mayan-edms/lib/python2.7/site-packages/mayan/apps/document_indexing/views.py", line 248, in get_queryset
    return DocumentListView.get_queryset(self)
  File "/usr/local/mayan-edms/lib/python2.7/site-packages/mayan/apps/common/mixins.py", line 287, in get_queryset
    return super(PreserveGetQuerysetMixin, self).get_queryset(*args, **kwargs)
RuntimeError: maximum recursion depth exceeded while calling a Python object

I'm not sure what to do to resolve this.

My Mayan install is on CentOS 7.3 VM, with Python 2.7.5, and the DB is on a separate server running PostgreSQL 9.6.1 (on FreeBSD).  I installed Mayan using the "Advanced Deployment" instructions.  I'm on the latest version of Mayan - 2.6.4.  I've also gone through a few upgrades (pip install -U mayan-edms) on this install - I think I started at 2.1 or 2.2, but I can't recall the exact version.

Most of my indexes are simply the Root node with one child level that has a form similar to "{{ document.metadata_value_of.zFrom }}", there's nothing terribly complex about them.  I've seen this problem on all of the indexes.

So far I've tried rebuilding the indexes and deleting all of the .pyc files (in case something old was left over from one of the upgrades).  Neither really helped, though I have noticed that restarting the UWSGI process seems to make it behave for a while.

Is there enough here to help me figure out what's happening?  If not, what can I do to get closer to resolving this?

Thanks,
-- Dave

--

---
You received this message because you are subscribed to the Google Groups "Mayan EDMS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

[Mayan EDMS: 1964] Re: Server error when viewing indexes

rosarior
Administrator
Thanks for reporting this and for the debug information. The first step is to try to make it repeatable to find the cause, things that happen sometimes and sometimes not are very hard to track down. I'll try to write a test to replicate this behavior in development. Thank you. 

On Monday, July 31, 2017 at 11:10:17 AM UTC-4, David Kimmel wrote:
Hey there,

I'm using Mayan to store a bunch of my personal documents (bills, etc) and using indexes to organize them.  So far everything works really well, except for the occasional "Server error" when viewing the indexes.  Most of the time I can resolve the error by refreshing the page, but ideally I'd like it to stop happening in the first place.  The problem doesn't occur all the time - sometimes I'll see it once and a refresh makes it disappear, other times I can refresh dozens of times and still see the error.  The problem only occurs on index nodes that show documents, not index nodes that only show child nodes.

When the error occurs, I see the following in the error log:

2017-07-31 08:50:15,983 common.middleware.error_logging <3752> [ERROR] "process_exception() line 17 Exception caught by request middleware; <WSGIRequest: GET '/indexing/instance/node/363/'>, maximum recursion depth exceeded while calling a Python object"
Traceback (most recent call last):
  File "/usr/local/mayan-edms/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/mayan-edms/lib/python2.7/site-packages/django/views/generic/base.py", line 68, in view
    return self.dispatch(request, *args, **kwargs)
  File "/usr/local/mayan-edms/lib/python2.7/site-packages/mayan/apps/document_indexing/views.py", line 240, in dispatch
    self, request, *args, **kwargs
  File "/usr/local/mayan-edms/lib/python2.7/site-packages/mayan/apps/common/mixins.py", line 350, in dispatch
    ).dispatch(request, *args, **kwargs)
  File "/usr/local/mayan-edms/lib/python2.7/site-packages/mayan/apps/common/mixins.py", line 315, in dispatch
    ).dispatch(request, *args, **kwargs)
  File "/usr/local/mayan-edms/lib/python2.7/site-packages/django/views/generic/base.py", line 88, in dispatch
    return handler(request, *args, **kwargs)
  File "/usr/local/mayan-edms/lib/python2.7/site-packages/django/views/generic/list.py", line 159, in get
    self.object_list = self.get_queryset()
  File "/usr/local/mayan-edms/lib/python2.7/site-packages/mayan/apps/common/mixins.py", line 285, in get_queryset
    self.queryset = self.__class__.original_get_queryset(self, *args, **kwargs)

The last two entries repeat for a long time and it ends with:

  File "/usr/local/mayan-edms/lib/python2.7/site-packages/mayan/apps/document_indexing/views.py", line 248, in get_queryset
    return DocumentListView.get_queryset(self)
  File "/usr/local/mayan-edms/lib/python2.7/site-packages/mayan/apps/common/mixins.py", line 287, in get_queryset
    return super(PreserveGetQuerysetMixin, self).get_queryset(*args, **kwargs)
RuntimeError: maximum recursion depth exceeded while calling a Python object

I'm not sure what to do to resolve this.

My Mayan install is on CentOS 7.3 VM, with Python 2.7.5, and the DB is on a separate server running PostgreSQL 9.6.1 (on FreeBSD).  I installed Mayan using the "Advanced Deployment" instructions.  I'm on the latest version of Mayan - 2.6.4.  I've also gone through a few upgrades (pip install -U mayan-edms) on this install - I think I started at 2.1 or 2.2, but I can't recall the exact version.

Most of my indexes are simply the Root node with one child level that has a form similar to "{{ document.metadata_value_of.zFrom }}", there's nothing terribly complex about them.  I've seen this problem on all of the indexes.

So far I've tried rebuilding the indexes and deleting all of the .pyc files (in case something old was left over from one of the upgrades).  Neither really helped, though I have noticed that restarting the UWSGI process seems to make it behave for a while.

Is there enough here to help me figure out what's happening?  If not, what can I do to get closer to resolving this?

Thanks,
-- Dave

--

---
You received this message because you are subscribed to the Google Groups "Mayan EDMS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Mayan EDMS: 1977] Server error when viewing indexes

Dave Kimmel
Hey Roberto,

Since it’s intermittent, it’s pretty tough to write detailed replication steps.  I’ve been keeping an eye on it to see if I could tie a specific action to triggering the problem.

Today the indexes broke again and I noticed that it didn’t happen until after I moved a document to the trash.  As soon as I did this, all of my indexes started showing the server error page.  I’m going to continue monitoring and see if any other actions trigger this.  Interestingly enough, I’ve been doing a bunch of updates to the metadata today and did not get the error at all.  

Are index updates done differently when moving documents to the trash vs updating metadata?

This really feels like a caching problem to me, but I honestly don’t know enough about Mayan and Django to know if this is a reasonable assumption.

Thanks,
-- Dave Kimmel
   [hidden email]



> On Aug 2, 2017, at 13:22, Roberto Rosario <[hidden email]> wrote:
>
> Thanks for reporting this and for the debug information. The first step is to try to make it repeatable to find the cause, things that happen sometimes and sometimes not are very hard to track down. I'll try to write a test to replicate this behavior in development. Thank you.

--

---
You received this message because you are subscribed to the Google Groups "Mayan EDMS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Mayan EDMS: 2009] Server error when viewing indexes

rosarior
Administrator
Made some progress on this. The problem is not really with the indexing but with this: https://gitlab.com/mayan-edms/mayan-edms/blob/master/mayan/apps/common/mixins.py#L269

It is a solution to allow subclass views to have a method called get_queryset while also allowing their super class to have the same method without causing an method override. It manipulates Python class MRO (method resolution order). For some reason in your instance, this code doesn't execute correctly and the call to the get_queryset method ends up calling itself until Python detects the recursion and stops it via the RunTimeError exception. This is the error manifests in document_indexing/views.py by really starts in common/mixins.py. So in a sense your perception of how it behaved liked a caching problem was not far from the real cause.

The solution would be to find the cause and fix the PreserveGetQuerysetMixin mixin class (a bit experimental) or remove it and do proper method overloading on all classes that use the get_queryset method. The mixin class was created to prevent security issues where a view would bypass the ACL check from the get_queryset method, on purpose or by accident. Will have to choose the lesser of two evils to fix this.

On Monday, August 7, 2017 at 1:48:45 PM UTC-4, David Kimmel wrote:
Hey Roberto,

Since it’s intermittent, it’s pretty tough to write detailed replication steps.  I’ve been keeping an eye on it to see if I could tie a specific action to triggering the problem.

Today the indexes broke again and I noticed that it didn’t happen until after I moved a document to the trash.  As soon as I did this, all of my indexes started showing the server error page.  I’m going to continue monitoring and see if any other actions trigger this.  Interestingly enough, I’ve been doing a bunch of updates to the metadata today and did not get the error at all.  

Are index updates done differently when moving documents to the trash vs updating metadata?

This really feels like a caching problem to me, but I honestly don’t know enough about Mayan and Django to know if this is a reasonable assumption.

Thanks,
-- Dave Kimmel
   [hidden email]



> On Aug 2, 2017, at 13:22, Roberto Rosario <[hidden email]> wrote:
>
> Thanks for reporting this and for the debug information. The first step is to try to make it repeatable to find the cause, things that happen sometimes and sometimes not are very hard to track down. I'll try to write a test to replicate this behavior in development. Thank you.

--

---
You received this message because you are subscribed to the Google Groups "Mayan EDMS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.