vr: cleanup files in /var/cache/cloud/processed every day

[weizhouapache]

Description

This PR fixes #3757

This adds a daily cron job to clean up files in /var/cache/cloud/processed older than 180 days, 179 days, ... until the disk space usage does not exceed threshold (80%). the logs in last 5 days will not be cleaned.

Types of changes

• Breaking change (fix or feature that would cause existing functionality to change)
• New feature (non-breaking change which adds functionality)
• Bug fix (non-breaking change which fixes an issue)
• Enhancement (improves an existing feature and functionality)
• Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

• Major
• Minor

Bug Severity

• BLOCKER
• Critical
• Major
• Minor
• Trivial

Screenshots (if appropriate):

How Has This Been Tested?

[nvazquez]

@blueorangutan package

[blueorangutan]

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

[nvazquez]

Code LGTM

[blueorangutan]

Packaging result: ✔️ el7 ✖️ el8 ✔️ debian. SL-JID 917

[nvazquez]

@blueorangutan package

[blueorangutan]

@nvazquez a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

[blueorangutan]

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian. SL-JID 924

[nvazquez]

@blueorangutan test

[blueorangutan]

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[blueorangutan]

Trillian Build Failed (tid-1711)

[ravening]

@weizhouapache the logrotate service should be able to take care of this right rather than adding a new shell script? we can specify the duration to keep the logs also

[rohityadavcloud]

@blueorangutan test

[blueorangutan]

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[weizhouapache]

@weizhouapache the logrotate service should be able to take care of this right rather than adding a new shell script? we can specify the duration to keep the logs also

@ravening no. logrorate will rotate the log files in /var/log

[ravening]

@weizhouapache the logrotate service should be able to take care of this right rather than adding a new shell script? we can specify the duration to keep the logs also

@ravening no. logrorate will rotate the log files in /var/log

@weizhouapache but we can add an extra rule to monitor /var/cache also right?

[blueorangutan]

Trillian test result (tid-1714)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 34754 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr5331-t1714-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_iso.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 87 look OK, 0 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File

[weizhouapache]

@weizhouapache the logrotate service should be able to take care of this right rather than adding a new shell script? we can specify the duration to keep the logs also

@ravening no. logrorate will rotate the log files in /var/log

@weizhouapache but we can add an extra rule to monitor /var/cache also right?

@ravening the problem is, we do not know what the filenames are.
here is an example

root@r-88-VM:~# ls -l /var/cache/cloud/processed/
total 24
-rw-r--r-- 1 root root 239 Aug 19 07:44 vm_dhcp_entry.json.82cf8311-f5d4-4a10-9b50-001e0558082a.gz
-rw-r--r-- 1 root root 239 Aug 19 07:42 vm_dhcp_entry.json.ef071140-60d8-4a20-970d-ca4979aee9f5.gz
-rw-r--r-- 1 root root 406 Aug 19 07:44 vm_metadata.json.26853b78-280b-4ea5-94b2-55e83bb332e4.gz
-rw-r--r-- 1 root root 406 Aug 19 07:42 vm_metadata.json.27cbd5c6-43b8-417f-8c1d-e51fc697ce10.gz
-rw-r--r-- 1 root root 160 Aug 19 07:42 vm_password.json.08296973-3f9a-4fb5-ac75-d9c9c9c9c8fb.gz
-rw-r--r-- 1 root root 160 Aug 19 07:44 vm_password.json.dea5c53d-043a-43f1-980c-f904a1e7b5b7.gz

[davidjumani]

CLGTM

[DaanHoogland]

license header needed?

[weizhouapache]

@DaanHoogland good question...

[weizhouapache]

@DaanHoogland done.

[DaanHoogland]

clgtm