ACL Rule works intermittently, requires reboot or cleanup

[btzq]

ISSUE TYPE

• Bug Report

COMPONENT NAME

VR

CLOUDSTACK VERSION

4.19.0

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

This happens on and off, but often enough to know its a bug.

When a ACL Rule has been updated, it doesnt always take effect. Sometimes it works, sometime it does not.

When it does not work, we need to reboot the router, which causes downtime, or, clean up the VPC.

STEPS TO REPRODUCE

- Create VPC (2 Subnets or More)
- Create custom ACL Rule (we have around 30 ACL Rules per Tier)
- Everything works fine
- Update ACL Rule (maybe add a new port to allow access)

Result: Hit or miss. But when it is a miss, we need to reboot or cleanup the router to take effect.

EXPECTED RESULTS

When an ACL Rule is changed, it should take effect immediately and reliably

ACTUAL RESULTS

Does not always work. 

[DaanHoogland]

@btzq sounds bad, but as you say hit or miss. This will be also true for anybody fixing it, so please add as much details as you can. errors from logs on the router host and MS, cidrs, port numbers and maybe even ips.

Things that could help

• is the command getting to the router
• is the command file being processed
• is the service being restarted
• etc

[btzq]

@DaanHoogland

We encountered the situation again on one of our VPCs.

We found that the VR iptables shows that the rule was indeed added, but changes did not take effect.

Note, this VPC has a few hundred ACL Rules, and 15 Private Gateways.

[DaanHoogland]

We found that the VR iptables shows that the rule was indeed added, but changes did not take effect.

Note, this VPC has a few hundred ACL Rules, and 15 Private Gateways.

wouw,

Can you look in the VR if the command for the rules is being processed? I.E. if the file with the new rule is being moved to /var/cache/cloud/processed/ on the router?

As this seems to be quite a busy VM can you also check if there is any disk problem on it?

[harikrishna-patnala]

@btzq are you still facing this issue, if so please move in the direction of what @DaanHoogland has mentioned and feel free to reopen this issue with more information helping the investigation of this issue.