Error when querying VLANs that are duplicated across datacenters | ValueError: get() returned more than one result #276

New issue

Open

opened 2023-08-10 22:46:06 +02:00 by eliezerlp · 1 comment

eliezerlp commented

2023-08-10 22:46:06 +02:00

(Migrated from github.com)

Describe the bug
We hit the issue noted by the FIXME in the code that results from the same VLAN ID existing in Netbox across datacenters:

    def get_or_create_vlan(self, vlan_id):
        # FIXME: we may need to specify the datacenter
        # since users may have same vlan id in multiple dc
        vlan = nb.ipam.vlans.get(
            vid=vlan_id,
        )
        if vlan is None:
            vlan = nb.ipam.vlans.create(
                name='VLAN {}'.format(vlan_id),
                vid=vlan_id,
            )
        return vlan

Error raised:

INFO:root:Resetting tagged VLAN(s) on interface eno1.1234
DEBUG:urllib3.connectionpool:http://***:****/ "GET /api/ipam/vlans/?vid=1234&limit=0 HTTP/1.1" 200 2790
Traceback (most recent call last):
  File "/root/nb/venv/bin/netbox_agent", line 11, in <module>
    load_entry_point('netbox-agent==0.7.1', 'console_scripts', 'netbox_agent')()
  File "/root/nb/venv/lib/python3.7/site-packages/netbox_agent/cli.py", line 50, in main
    return run(config)
  File "/root/nb/venv/lib/python3.7/site-packages/netbox_agent/cli.py", line 43, in run
    server.netbox_create_or_update(config)
  File "/root/nb/venv/lib/python3.7/site-packages/netbox_agent/server.py", line 413, in netbox_create_or_update
    self.network.create_or_update_netbox_network_cards()
  File "/root/nb/venv/lib/python3.7/site-packages/netbox_agent/network.py", line 448, in create_or_update_netbox_network_cards
    ret, interface = self.reset_vlan_on_interface(nic, interface)
  File "/root/nb/venv/lib/python3.7/site-packages/netbox_agent/network.py", line 244, in reset_vlan_on_interface
    nb_vlan = self.get_or_create_vlan(vlan_id)
  File "/root/nb/venv/lib/python3.7/site-packages/netbox_agent/network.py", line 204, in get_or_create_vlan
    vid=vlan_id,
  File "/root/nb/venv/lib/python3.7/site-packages/pynetbox/core/endpoint.py", line 148, in get
    "get() returned more than one result. "
ValueError: get() returned more than one result. Check that the kwarg(s) passed are valid for this endpoint or use filter() or all() instead.

Expected behavior
When querying for the VLANs netbox-agent should include the datacenter as a filter.

We were able to work around this issue with the following edit:

diff --git a/netbox_agent/network.py b/netbox_agent/network.py
index 673dfc1..0793454 100644
--- a/netbox_agent/network.py
+++ b/netbox_agent/network.py
@@ -202,6 +202,7 @@ class Network(object):
         # since users may have same vlan id in multiple dc
         vlan = nb.ipam.vlans.get(
             vid=vlan_id,
+            site=self.server.get_datacenter(),
         )
         if vlan is None:
             vlan = nb.ipam.vlans.create(

If this approach is a reasonable one, I'd be happy to open a PR for the specific change.

Configuration file

netbox:
 url: 'http://****:****'
 token: *****

network:
  ignore_interfaces: "(dummy.*|docker.*|veth.*|br-.*)"
  ignore_ips: (127\.0\.0\..*)
  lldp: false

datacenter_location:
  driver: "cmd:echo 'dc-abc'"
  regex: "(.*)"

rack_location:

inventory: true

Environment:

OS: Fedora
Netbox agent version = master

Additional context
Thank you for sharing and helping maintaining this wonderful tool!

**Describe the bug** We hit the issue noted by the `FIXME` in the [code](https://github.com/Solvik/netbox-agent/blob/19a158ec823bea04d4473e9f829f6d277a1e39a9/netbox_agent/network.py#L200) that results from the same VLAN ID existing in Netbox across datacenters: ``` def get_or_create_vlan(self, vlan_id): # FIXME: we may need to specify the datacenter # since users may have same vlan id in multiple dc vlan = nb.ipam.vlans.get( vid=vlan_id, ) if vlan is None: vlan = nb.ipam.vlans.create( name='VLAN {}'.format(vlan_id), vid=vlan_id, ) return vlan ``` Error raised: ``` INFO:root:Resetting tagged VLAN(s) on interface eno1.1234 DEBUG:urllib3.connectionpool:http://***:****/ "GET /api/ipam/vlans/?vid=1234&limit=0 HTTP/1.1" 200 2790 Traceback (most recent call last): File "/root/nb/venv/bin/netbox_agent", line 11, in <module> load_entry_point('netbox-agent==0.7.1', 'console_scripts', 'netbox_agent')() File "/root/nb/venv/lib/python3.7/site-packages/netbox_agent/cli.py", line 50, in main return run(config) File "/root/nb/venv/lib/python3.7/site-packages/netbox_agent/cli.py", line 43, in run server.netbox_create_or_update(config) File "/root/nb/venv/lib/python3.7/site-packages/netbox_agent/server.py", line 413, in netbox_create_or_update self.network.create_or_update_netbox_network_cards() File "/root/nb/venv/lib/python3.7/site-packages/netbox_agent/network.py", line 448, in create_or_update_netbox_network_cards ret, interface = self.reset_vlan_on_interface(nic, interface) File "/root/nb/venv/lib/python3.7/site-packages/netbox_agent/network.py", line 244, in reset_vlan_on_interface nb_vlan = self.get_or_create_vlan(vlan_id) File "/root/nb/venv/lib/python3.7/site-packages/netbox_agent/network.py", line 204, in get_or_create_vlan vid=vlan_id, File "/root/nb/venv/lib/python3.7/site-packages/pynetbox/core/endpoint.py", line 148, in get "get() returned more than one result. " ValueError: get() returned more than one result. Check that the kwarg(s) passed are valid for this endpoint or use filter() or all() instead. ``` **Expected behavior** When querying for the VLANs netbox-agent should include the datacenter as a filter. We were able to work around this issue with the following edit: ``` diff --git a/netbox_agent/network.py b/netbox_agent/network.py index 673dfc1..0793454 100644 --- a/netbox_agent/network.py +++ b/netbox_agent/network.py @@ -202,6 +202,7 @@ class Network(object): # since users may have same vlan id in multiple dc vlan = nb.ipam.vlans.get( vid=vlan_id, + site=self.server.get_datacenter(), ) if vlan is None: vlan = nb.ipam.vlans.create( ``` If this approach is a reasonable one, I'd be happy to open a PR for the specific change. **Configuration file** ``` netbox: url: 'http://****:****' token: ***** network: ignore_interfaces: "(dummy.*|docker.*|veth.*|br-.*)" ignore_ips: (127\.0\.0\..*) lldp: false datacenter_location: driver: "cmd:echo 'dc-abc'" regex: "(.*)" rack_location: inventory: true ``` **Environment:** - OS: Fedora - Netbox agent version = master **Additional context** Thank you for sharing and helping maintaining this wonderful tool!

Solvik commented

2023-08-18 17:28:21 +02:00

(Migrated from github.com)

Hi @eliezerlp

This comment brings back memories!

Over the years I haven't had any network setup with similar VLAN IDs across datacenters, especially since modern networks use BGP/EVPN, so I guess we could finally add this parameter to filter out this.
Would you mind opening a PR ?

As for maintaining, I must confess I wasn't very active these past few months/years ; you should really thank @cyrinux who took over for quite some time :)

Hi @eliezerlp This comment brings back memories! Over the years I haven't had any network setup with similar VLAN IDs across datacenters, especially since modern networks use BGP/EVPN, so I guess we could finally add this parameter to filter out this. Would you mind opening a PR ? As for maintaining, I must confess I wasn't very active these past few months/years ; you should really thank @cyrinux who took over for quite some time :)