Skip to content

Frequent error 502 and high server load since update to 16.0.1 #16020

@djermann

Description

@djermann

Steps to reproduce

  1. Upgrade to 16.0.1

Expected behaviour

Load should be normal and no 502 error

Actual behaviour

From time to time Nextcloud becomes unavailable with a 502 - Bad Gateway error and the load is between 10 - 15.
I checked the processlist of MariaDB/MySQL and there seem to be a lot of queries going on for many hours:

MySQL processlist
+--------+--------------+-----------+--------------+---------+------+----------------+------------------------------------------------------------------------------------------------------+----------+
| Id     | User         | Host      | db           | Command | Time | State          | Info                                                                                                 | Progress |
+--------+--------------+-----------+--------------+---------+------+----------------+------------------------------------------------------------------------------------------------------+----------+
| 324404 | <REMOVED> | localhost | <REMOVED> | Query   |  187 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (1, 23, 1, 8, 1, 1, 30)) AND ((`f`.`mim |    0.000 |
| 326496 | <REMOVED> | localhost | <REMOVED> | Query   |  200 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (1, 23, 1, 8, 1, 1, 30)) AND ((`f`.`mim |    0.000 |
| 334566 | <REMOVED> | localhost | <REMOVED> | Query   |   99 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (1, 165)) AND ((`f`.`mimetype` <> 2) OR |    0.000 |
| 334646 | <REMOVED> | localhost | <REMOVED> | Query   |   99 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (1, 165)) AND ((`f`.`mimetype` <> 2) OR |    0.000 |
| 334847 | <REMOVED> | localhost | <REMOVED> | Query   |   67 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (1, 20, 1, 1, 55)) AND ((`f`.`mimetype` |    0.000 |
| 339223 | <REMOVED> | localhost | <REMOVED> | Query   |   53 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (1, 88)) AND ((`f`.`mimetype` <> 2) OR  |    0.000 |
| 341354 | <REMOVED> | localhost | <REMOVED> | Query   |   20 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 341418 | <REMOVED> | localhost | <REMOVED> | Query   |   51 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 341527 | <REMOVED> | localhost | <REMOVED> | Query   |   51 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 341617 | <REMOVED> | localhost | <REMOVED> | Query   |   20 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 341643 | <REMOVED> | localhost | <REMOVED> | Query   |   20 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 341685 | <REMOVED> | localhost | <REMOVED> | Query   |   20 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 341714 | <REMOVED> | localhost | <REMOVED> | Query   |   54 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 341730 | <REMOVED> | localhost | <REMOVED> | Query   |   20 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 341835 | <REMOVED> | localhost | <REMOVED> | Query   |   20 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 341875 | <REMOVED> | localhost | <REMOVED> | Query   |   51 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 341938 | <REMOVED> | localhost | <REMOVED> | Query   |   20 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 342183 | <REMOVED> | localhost | <REMOVED> | Query   |   20 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
| 343104 | <REMOVED> | localhost | <REMOVED> | Query   |  204 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (1, 8, 1, 1, 224)) AND ((`f`.`mimetype` |    0.000 |
| 343386 | <REMOVED> | localhost | <REMOVED> | Query   |   20 | Sorting result | SELECT `f`.* FROM `oc_filecache` `f` WHERE (`f`.`storage` IN (20, 1, 8, 1, 1, 61)) AND ((`f`.`mimety |    0.000 |
Server configuration

Operating system:
Debian 9.9
Web server:
Nginx
Database:
MariaDB 10.1
PHP version:
7.2
Nextcloud version: (see Nextcloud admin page)
16.0.1
Updated from an older Nextcloud/ownCloud or fresh install:
Updated from 13.0.4
Where did you install Nextcloud from:
Website

List of activated apps:

App list
Enabled:
  - accessibility: 1.2.0
  - activity: 2.9.1
  - announcementcenter: 3.5.1
  - apporder: 0.7.1
  - bookmarks: 1.0.6
  - bruteforcesettings: 1.3.0
  - calendar: 1.7.0
  - circles: 0.17.3
  - cloud_federation_api: 0.2.0
  - comments: 1.6.0
  - contacts: 3.1.3
  - dav: 1.9.2
  - encryption: 2.4.0
  - external: 3.3.0
  - federatedfilesharing: 1.6.0
  - federation: 1.6.0
  - files: 1.11.0
  - files_pdfviewer: 1.5.0
  - files_rightclick: 0.13.0
  - files_sharing: 1.8.0
  - files_texteditor: 2.8.0
  - files_trashbin: 1.6.0
  - files_versions: 1.9.0
  - files_videoplayer: 1.5.0
  - firstrunwizard: 2.5.0
  - gallery: 18.3.0
  - logreader: 2.1.0
  - lookup_server_connector: 1.4.0
  - music: 0.9.5
  - nextcloud_announcements: 1.5.0
  - notifications: 2.4.1
  - oauth2: 1.4.2
  - password_policy: 1.6.0
  - privacy: 1.0.0
  - provisioning_api: 1.6.0
  - quota_warning: 1.5.0
  - radio: 0.6.5
  - recommendations: 0.4.0
  - serverinfo: 1.6.0
  - sharebymail: 1.6.0
  - support: 1.0.0
  - survey_client: 1.4.0
  - systemtags: 1.6.0
  - theming: 1.7.0
  - twofactor_backupcodes: 1.5.0
  - updatenotification: 1.6.0
  - viewer: 1.0.0
  - workflowengine: 1.6.0
Disabled:
  - admin_audit
  - files_external
  - user_ldapr

Nextcloud configuration:

Config report
{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "16.0.1.1",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "logtimezone": "Europe\/Zurich",
        "installed": true,
        "activity_expire_days": 180,
        "trashbin_retention_obligation": "auto,90",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "loglevel": 2,
        "log_type": "errorlog",
        "mail_smtpmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauth": 1,
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "25",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "updater.release.channel": "stable",
        "maintenance": false,
        "theme": ""
    }
}

Are you using external storage, if yes which one: local/smb/sftp/...
no
Are you using encryption: yes/no
yes
Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...
no

Logs

Nothing interesting in the logs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    0. Needs triagePending check for reproducibility or if it fits our roadmapbugneeds info

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions