DGNum/netbox-agent
6
0
Fork 0
Code Issues 28 Pull requests 9 Projects Releases 14 Packages Wiki Activity Actions

support netbox >=2.9 #177

Merged
Solvik merged 22 commits from 176-netbox-2.9 into master 2022-03-07 16:03:05 +01:00
Conversation 10 Commits 22 Files changed 21 +795 -410
Solvik commented 2020-11-28 11:58:08 +01:00 (Migrated from github.com)
Copy link

Closes #176

Closes #176
Solvik commented 2020-11-28 12:48:18 +01:00 (Migrated from github.com)
Copy link

Still needs to fix tags for Inventory and custom tags

Still needs to fix tags for Inventory and custom tags
ecoutinho commented 2020-11-28 19:19:12 +01:00 (Migrated from github.com)
Copy link

Thanks for addressing this issue. After upgrading to latest pynetbox, the update-inventory resulted in:

INFO:root:Unassigning IP 10.1.1.1/32 from eth0
DEBUG:urllib3.connectionpool:https://x.x.x:443 "PATCH /api/ipam/ip-addresses/109/ HTTP/1.1" 400 94
Traceback (most recent call last):
  File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/src/netbox-agent-176-netbox-2.9/netbox_agent/cli.py", line 48, in <module>
    main()
  File "/usr/local/src/netbox-agent-176-netbox-2.9/netbox_agent/cli.py", line 44, in main
    return run(config)
  File "/usr/local/src/netbox-agent-176-netbox-2.9/netbox_agent/cli.py", line 39, in run
    server.netbox_create_or_update(config)
  File "/usr/local/src/netbox-agent-176-netbox-2.9/netbox_agent/virtualmachine.py", line 104, in netbox_create_or_update
    self.network.create_or_update_netbox_network_cards()
  File "/usr/local/src/netbox-agent-176-netbox-2.9/netbox_agent/network.py", line 436, in create_or_update_netbox_network_cards
    netbox_ip.save()
  File "/opt/netbox-2.9.10/venv/lib64/python3.6/site-packages/pynetbox/core/response.py", line 391, in save
    if req.patch({i: serialized[i] for i in diff}):
  File "/opt/netbox-2.9.10/venv/lib64/python3.6/site-packages/pynetbox/core/query.py", line 409, in patch
    return self._make_call(verb="patch", data=data)
  File "/opt/netbox-2.9.10/venv/lib64/python3.6/site-packages/pynetbox/core/query.py", line 274, in _make_call
    raise RequestError(req)
pynetbox.core.query.RequestError: The request failed with code 400 Bad Request: {'vminterface': ['IP address is primary for virtual machine xxxxx but not assigned to it!']}

It seems that the IP must be removed from 'Primary IPv4' before removing from the interface.

Then I noticed the mask was wrong and changed it into '/24'. This time netbox_agent didn't try to delete the interface. It resulted in another error:

DEBUG:urllib3.connectionpool:https://x.x.x:443 "GET /api/ipam/ip-addresses/?address=10.1.1.1%2F24 HTTP/1.1" 200 1218
DEBUG:urllib3.connectionpool:https://x.x.x:443 "GET /api/ipam/ip-addresses/109/ HTTP/1.1" 200 1166
DEBUG:root:Finished updating NIC!
Traceback (most recent call last):
  File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/src/netbox-agent-176-netbox-2.9/netbox_agent/cli.py", line 48, in <module>
    main()
  File "/usr/local/src/netbox-agent-176-netbox-2.9/netbox_agent/cli.py", line 44, in main
    return run(config)
  File "/usr/local/src/netbox-agent-176-netbox-2.9/netbox_agent/cli.py", line 39, in run
    server.netbox_create_or_update(config)
  File "/usr/local/src/netbox-agent-176-netbox-2.9/netbox_agent/virtualmachine.py", line 113, in netbox_create_or_update
    if sorted(set(vm.tags)) != sorted(set(self.tags)):
TypeError: '<' not supported between instances of 'Record' and 'Record'

This VM already has 3 tags.

Thanks for addressing this issue. After upgrading to latest pynetbox, the update-inventory resulted in: ``` INFO:root:Unassigning IP 10.1.1.1/32 from eth0 DEBUG:urllib3.connectionpool:https://x.x.x:443 "PATCH /api/ipam/ip-addresses/109/ HTTP/1.1" 400 94 Traceback (most recent call last): File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/usr/local/src/netbox-agent-176-netbox-2.9/netbox_agent/cli.py", line 48, in <module> main() File "/usr/local/src/netbox-agent-176-netbox-2.9/netbox_agent/cli.py", line 44, in main return run(config) File "/usr/local/src/netbox-agent-176-netbox-2.9/netbox_agent/cli.py", line 39, in run server.netbox_create_or_update(config) File "/usr/local/src/netbox-agent-176-netbox-2.9/netbox_agent/virtualmachine.py", line 104, in netbox_create_or_update self.network.create_or_update_netbox_network_cards() File "/usr/local/src/netbox-agent-176-netbox-2.9/netbox_agent/network.py", line 436, in create_or_update_netbox_network_cards netbox_ip.save() File "/opt/netbox-2.9.10/venv/lib64/python3.6/site-packages/pynetbox/core/response.py", line 391, in save if req.patch({i: serialized[i] for i in diff}): File "/opt/netbox-2.9.10/venv/lib64/python3.6/site-packages/pynetbox/core/query.py", line 409, in patch return self._make_call(verb="patch", data=data) File "/opt/netbox-2.9.10/venv/lib64/python3.6/site-packages/pynetbox/core/query.py", line 274, in _make_call raise RequestError(req) pynetbox.core.query.RequestError: The request failed with code 400 Bad Request: {'vminterface': ['IP address is primary for virtual machine xxxxx but not assigned to it!']} ``` It seems that the IP must be removed from 'Primary IPv4' before removing from the interface. Then I noticed the mask was wrong and changed it into '/24'. This time netbox_agent didn't try to delete the interface. It resulted in another error: ``` DEBUG:urllib3.connectionpool:https://x.x.x:443 "GET /api/ipam/ip-addresses/?address=10.1.1.1%2F24 HTTP/1.1" 200 1218 DEBUG:urllib3.connectionpool:https://x.x.x:443 "GET /api/ipam/ip-addresses/109/ HTTP/1.1" 200 1166 DEBUG:root:Finished updating NIC! Traceback (most recent call last): File "/usr/lib64/python3.6/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/usr/lib64/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/usr/local/src/netbox-agent-176-netbox-2.9/netbox_agent/cli.py", line 48, in <module> main() File "/usr/local/src/netbox-agent-176-netbox-2.9/netbox_agent/cli.py", line 44, in main return run(config) File "/usr/local/src/netbox-agent-176-netbox-2.9/netbox_agent/cli.py", line 39, in run server.netbox_create_or_update(config) File "/usr/local/src/netbox-agent-176-netbox-2.9/netbox_agent/virtualmachine.py", line 113, in netbox_create_or_update if sorted(set(vm.tags)) != sorted(set(self.tags)): TypeError: '<' not supported between instances of 'Record' and 'Record' ``` This VM already has 3 tags.
Solvik commented 2020-11-30 10:23:26 +01:00 (Migrated from github.com)
Copy link

I'm yet to find a solution with the breaking changes for tags
For network, I'll try to reproduce the primary IP and handle this case

I'm yet to find a solution with the breaking changes for tags For network, I'll try to reproduce the primary IP and handle this case
Solvik commented 2020-12-15 12:20:15 +01:00 (Migrated from github.com)
Copy link

For tags I'm blocked by https://github.com/digitalocean/pynetbox/pull/307 or https://github.com/digitalocean/pynetbox/pull/291

Still trying to figure out something clean not depending on these 2 issues

For tags I'm blocked by https://github.com/digitalocean/pynetbox/pull/307 or https://github.com/digitalocean/pynetbox/pull/291 Still trying to figure out something clean not depending on these 2 issues
jbe-dw commented 2020-12-15 18:41:52 +01:00 (Migrated from github.com)
Copy link

Hello,

I have seen you mentioned my PR here. There is no way for you to make it work with 2.9 unless pynetbox is patched, because tags (and ip adresses interface assignment as well) has been changed.

Pre 2.9 tags was a list of tag names, and processed as a simple list by pynetbox. Updating this list with a new tag resulted in creating the tag if it did not exist.

Since 2.9 tags are objects like others, and you need to provide either the list of objects, or their ids, or a list of dicts with a least the "name" key (or a mix). Since pynetbox serialize inputs into json, it doesn't do it correctly. You can try to change your calls with the good format, pynetbox would send the wrong format anyway.

Regards

Hello, I have seen you mentioned my PR here. There is no way for you to make it work with 2.9 unless pynetbox is patched, because tags (and ip adresses interface assignment as well) has been changed. Pre 2.9 tags was a list of tag names, and processed as a simple list by pynetbox. Updating this list with a new tag resulted in creating the tag if it did not exist. Since 2.9 tags are objects like others, and you need to provide either the list of objects, or their ids, or a list of dicts with a least the "name" key (or a mix). Since pynetbox serialize inputs into json, it doesn't do it correctly. You can try to change your calls with the good format, pynetbox would send the wrong format anyway. Regards
Solvik commented 2021-01-06 09:41:25 +01:00 (Migrated from github.com)
Copy link

Seems like pynetbox merged the tags support 13 days ago. I'll go ahead and test this so I can do a proper release of netbox-agent

Seems like pynetbox merged the tags support 13 days ago. I'll go ahead and test this so I can do a proper release of netbox-agent
Solvik commented 2021-05-11 21:57:00 +02:00 (Migrated from github.com)
Copy link

I've finally had the time to continue this PR.
It's working pretty well with my Netbox 2.10 install

I need to do more tests on, Netbox 2.11

I also think I'll remove all the if with versions <2.9 as this release of netbox_agent should only be compatible with newer Netbox versions

@jbe-dw @ecoutinho if you're still using the agent, I'd be happy to get your feeback.

I've finally had the time to continue this PR. It's working pretty well with my Netbox 2.10 install I need to do more tests on, Netbox 2.11 I also think I'll remove all the if with versions <2.9 as this release of `netbox_agent` should only be compatible with newer Netbox versions @jbe-dw @ecoutinho if you're still using the agent, I'd be happy to get your feeback.
jbe-dw commented 2021-05-12 06:58:15 +02:00 (Migrated from github.com)
Copy link

Hello,

I've posted here because you mentioned a discussion in pynetbox but i'm not using the agent.

Regards

Hello, I've posted here because you mentioned a discussion in pynetbox but i'm not using the agent. Regards
geektophe commented 2022-02-21 18:22:01 +01:00 (Migrated from github.com)
Copy link

Here's a short description of the changes we made on your branch after running the agent against a Netbox server in version 3.1.7 with pynetbox in version 6.1.2:

  • Added an option to specify an SSL CA certificates file to talk to the Netbox API
  • Enhanced GPU expansion bays inventory: internal GPU (VGA) goes into the blade, external GPU (3D) goes into the expansion bay
  • Unified the way expansion bays are managed (GPU and drive exansion bays)
  • Started to refactor network module to make it more readable
  • Dependencies in setup.py now reads its requirements from requirements.txt to avoid double maintenance

Currently, we managed to run the agent successfully under Ubuntu trusty, xenial and bionic (focal and CentOS 7 will follow soon, but I don't see why it would work differently).

Here's a short description of the changes we made on your branch after running the agent against a Netbox server in version `3.1.7` with `pynetbox` in version `6.1.2`: - Added an option to specify an SSL CA certificates file to talk to the Netbox API - Enhanced GPU expansion bays inventory: internal GPU (VGA) goes into the blade, external GPU (3D) goes into the expansion bay - Unified the way expansion bays are managed (GPU and drive exansion bays) - Started to refactor `network` module to make it more readable - Dependencies in `setup.py` now reads its requirements from `requirements.txt` to avoid double maintenance Currently, we managed to run the agent successfully under Ubuntu `trusty`, `xenial` and `bionic` (`focal` and `CentOS 7` will follow soon, but I don't see why it would work differently).
🎉 2
geektophe commented 2022-03-07 16:02:57 +01:00 (Migrated from github.com)
Copy link

We've successfully tested this branch against Netbox 3.1.7 using pynetbox version 6.1.2. Perhaps there will some minor bugs to fix du to hardware specifics, but we think it's time to merge that work.

We've successfully tested this branch against Netbox `3.1.7` using `pynetbox` version `6.1.2`. Perhaps there will some minor bugs to fix du to hardware specifics, but we think it's time to merge that work.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
help wanted

Extra attention is needed

  
need testing

   
status: accepted

   
status: discussing

   
type: bug

Something isn't working

  
type: documentation

Improvements or additions to documentation

  
type: duplicate

This issue or pull request already exists

  
type: enhancement

New feature or request

  
type:hardware

Mostly new hardware

  
type: invalid

This doesn't seem right

  
type: major feature

   
type: minor feature
  
bug

Something is not working

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
help wanted

Need some help

  
invalid

Something is wrong

  
question

More information is needed

  
wontfix

This won't be fixed

No labels
help wanted
need testing
status: accepted
status: discussing
type: bug
type: documentation
type: duplicate
type: enhancement
type:hardware
type: invalid
type: major feature
type: minor feature
bug
duplicate
enhancement
help wanted
invalid
question
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: DGNum/netbox-agent#177
No description provided.
Delete branch "176-netbox-2.9"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?