Compare commits

...

103 commits
v0.7.1 ... main

Author SHA1 Message Date
4242832396 Merge pull request 'Fix disk size inventory' (#310) from ribetm/disk_size into main
Reviewed-on: #310
2024-10-21 13:35:29 +02:00
13b84b4da1
feat: Run formatters 2024-10-21 12:55:54 +02:00
837860e31a
feat: Add nix tooling 2024-10-21 12:55:40 +02:00
c1a7f661a3 Merge pull request 'Virtualmachine corrections' (#279) from obeone/virtualmachine_corrections into master
Reviewed-on: #279
2024-10-21 12:32:30 +02:00
Grégoire Compagnon
7d268ea0e8
Return 0 if everything ok as excepted in a shell 2024-10-21 12:30:09 +02:00
Grégoire Compagnon
9d496c6854
Check if it's a VM before running lldp related actions 2024-10-21 12:30:09 +02:00
Grégoire Compagnon
6ef23eae4d
Add missing prtint debug 2024-10-21 12:30:09 +02:00
12ceea413c Merge pull request 'Replaced deprecated module pkg_resources' (#311) from ribetm/py312 into master
Reviewed-on: #311
2024-10-21 10:18:18 +02:00
7a968deee9 Merge pull request 'Make lshw scrpping more resilient' (#292) from sinavir/fix_gpu_without_vendor into master
Reviewed-on: #292
2024-10-21 10:15:27 +02:00
d55cbd62fc Merge pull request 'Fix not working tag with VM' (#293) from sinavir/fix_vm_tags into master
Reviewed-on: #293
2024-10-21 10:15:11 +02:00
59ce76fc29 Merge pull request 'feat(network): Batch requests when filtering on interfaces' (#297) from Tom-Hubrecht/master into master
Reviewed-on: #297
2024-10-21 10:01:35 +02:00
Mathis Ribet
c3d3e6857a
Replaced deprecated module pkg_resources
Removed in py3.12
2024-10-17 20:52:28 +02:00
Mathis Ribet
56627c1aa9
Fix disk size 2024-10-17 20:50:23 +02:00
CllaudiaB
8dde35dd31
Merge pull request #306 from CllaudiaB/netifaces2
netifaces2
2024-10-17 11:01:22 +02:00
clbu
de88ca85b9
fix(network): use netifaces2 2024-10-17 10:46:05 +02:00
CllaudiaB
514627aa72
Merge pull request #305 from CllaudiaB/version
Support Netbox 3
2024-10-17 10:37:09 +02:00
CllaudiaB
4b54a0a3db
fix: typo
Co-authored-by: n1nj4- <39305378+n1nj444@users.noreply.github.com>
2024-10-17 10:26:41 +02:00
clbu
e44fd2fe78
fix(network): retrieve switch interface using id instead of name 2024-10-16 10:44:08 +02:00
clbu
1429fedb9d
fix(network): cable
https://github.com/netbox-community/netbox/issues/9102
2024-10-10 23:15:55 +02:00
clbu
818c835711
fix(requirements): update dependencies version 2024-10-09 15:45:07 +02:00
clbu
40af19e801
fix(readme): required Netbox version 2024-10-09 15:32:25 +02:00
clbu
1d69f4e2f0
python version compatible with dependencies 2024-10-09 15:12:39 +02:00
clbu
a7104b6b94
updating dependencies 2024-08-07 12:29:19 +02:00
clbu
ee41fb4fc2
replace device_role with role 2024-08-07 12:24:49 +02:00
Tom Hubrecht
e04d0c6d59 feat(network): Batch requests when filtering on interfaces
This avoids an issue where the requested URI is too long when many
interfaces are present
2024-05-07 13:54:27 +02:00
sinavir
c9a57de843 fix: vm tags 2024-03-28 12:07:17 +01:00
sinavir
f512e7a0a9 fix: replace list.push by list.append 2024-03-27 18:49:05 +01:00
sinavir
116334be2f fix: make lshw props finding more resilient 2024-03-27 17:12:48 +01:00
Oleg Zagrebelsky
ba4cdb217b
fix lldp.py (#280)
Co-authored-by: kszd487 <kszd487@ukc02dd0ylp3ywui.lan>
2023-08-28 18:54:53 +02:00
renovate[bot]
7ab7bbb9e1
chore(deps): update dependency distro to v1.8.0 (#244)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-08-18 17:24:00 +02:00
renovate[bot]
b6a3acd6b4
chore(deps): update dependency netifaces to v0.11.0 (#188)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-08-18 17:22:58 +02:00
renovate[bot]
bb05e12f6e
chore(deps): update dependency packaging to v23 (#261)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-08-18 17:22:25 +02:00
renovate[bot]
fa51ca31ca
chore(deps): update dependency python-slugify to v8 (#263)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-08-18 17:13:35 +02:00
renovate[bot]
0fdb56e01d
chore(deps): update dependency pyyaml to v6 (#196)
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
2023-08-18 15:44:16 +02:00
Cyril Levis
19a158ec82
Merge pull request #271 from AlexDaichendt/fix-227-lshw 2023-04-08 12:42:53 +02:00
Alexander Daichendt
08360bafbb fix: lshw crashes due to duplicate logical name 2023-04-08 11:17:23 +02:00
Cyril Levis
e0d734d0ca
Merge pull request #268 from Varriount/make-ipmi-optional 2023-02-28 09:24:33 +01:00
Clay Sweetser
dc224e209b Make IPMI information gathering optional. 2023-02-27 19:06:14 -05:00
Cyril Levis
7259c7602d
Merge pull request #266 from Varriount/patch-2
Simplify `is_vm` function
2023-02-27 14:37:10 +01:00
Clay Sweetser
8cba98ec43
Simplify is_vm function 2023-02-24 19:15:23 -05:00
Cyril Levis
ab96965767
Merge pull request #264 from Farfaday/feat/manage_mtu
feat: Retrieve and manage MTU for the interfaces
2023-02-15 11:55:02 +01:00
François Blondel
221ac16e87 feat: Retrieve and manage MTU for the interfaces 2023-02-14 23:32:26 +01:00
Cyril Levis
9204ae2187
Merge pull request #262 from Solvik/fix/cle/hp-inventory-fix-3
fix: allow the inventory of several hp controller of the same model
2023-01-16 15:06:54 +01:00
Cyril Levis
528ecc09b0
fix: allow the inventory of several hp controller of the same model 2023-01-13 17:12:25 +01:00
Cyril Levis
117b39350e
Merge pull request #260 from Solvik/fix/cle/hp-inventory-fix-2
fix: prevent crash if disk fields attrs missing
2022-12-14 09:02:16 +01:00
Cyril Levis
4cf054278f
fix: prevent crash if disk fields attrs missing 2022-12-14 09:00:37 +01:00
Cyril Levis
b82dc80fe3
Merge pull request #259 from Solvik/fix/cle/hp-inventory-fix-wait
disk inventory fix and hp improvement
2022-12-13 17:28:57 +01:00
Cyril Levis
49b269efa6
chore: fetch more data as custom_fields for hp inventory 2022-12-13 17:27:35 +01:00
Cyril Levis
cedb6818a3
fix: handle big list of hp disk
we cant use p.wait() if too much data.
2022-12-13 17:26:45 +01:00
Cyril Levis
a9af96bba2
fix: get disk serial number 2022-12-13 17:26:14 +01:00
Cyril Levis
282f914665
Merge pull request #256 from Solvik/fix/cle/hpe-controller
fix: enable disk inventory for HPE
2022-12-09 14:22:07 +01:00
Cyril Levis
ca2a69b66f
fix: enable disk inventory for HPE 2022-12-09 14:19:26 +01:00
Cyril Levis
778814b0a1
Merge pull request #254 from Solvik/fix/cle/hp-controller
fix: add space in ignore_patterns
2022-12-02 12:30:40 +01:00
Cyril Levis
e0685e7167
fix: add space in ignore_patterns
to prevent false positive like `/data/cache/temp` for eg
2022-12-02 12:28:47 +01:00
Cyril Levis
ba0be73d3c
Merge pull request #253 from Solvik/fix/cle/storcli-inventory
fix: deadlock storcli inventory
2022-11-30 09:52:03 +01:00
Cyril Levis
bf65da0c58
fix: deadlock storcli inventory
When there is a lot of data return by storcli due to a lot of disk, the inventory was in deadlock
Its a known problem, see:
https://docs.python.org/3/library/subprocess.html#subprocess.Popen.wait
2022-11-30 09:49:51 +01:00
Cyril Levis
5b0df6ca05
Merge pull request #251 from Solvik/fix/dvdrom-inventory
fix: dvdrom inventory crash
2022-11-19 09:38:51 +01:00
Cyril Levis
2d8ec831e6
fix: dvd rom inventory crash
dvdrom dont have serial number, we fallback on a 'unknown' value now to prevent crash
2022-11-17 11:00:48 +01:00
Cyril Levis
fde6211f5b
Merge pull request #250 from Solvik/fix/hp-raid-ignore-errors-3
refactor hp raid errors handling
2022-11-16 15:48:28 +01:00
Cyril Levis
e96a50379b
chore: refactor hp raid ignore errors handling 2022-11-16 15:46:57 +01:00
Cyril Levis
633b6d3851
fix: ignore more cache errors 2022-11-16 14:43:35 +01:00
Cyril Levis
04c1f83e74
Merge pull request #249 from Solvik/fix/hp-raid-ignore-errors
fix: ignore some more hp controllers errors while parsing
2022-11-16 10:46:06 +01:00
Cyril Levis
9b06584fed
fix: ignore some more hp controllers errors while parsing 2022-11-16 10:45:25 +01:00
Cyril Levis
f7cdd92fa3
Merge pull request #232 from KivraChristoffer/fix-#226
This will solve issue #226
2022-11-10 15:53:46 +01:00
Cyril Levis
53769db3e4
Merge pull request #246 from ypid/fix/various
fix: TypeError: Object of type Record is not JSON serializable; fix: disk is also virtual if `product` is None
2022-11-10 15:52:39 +01:00
Cyril Levis
bce02a5e7c
Merge branch 'master' into fix/various 2022-11-10 15:51:32 +01:00
Cyril Levis
368d793704
Merge pull request #248 from Solvik/fix/cle/inventory-crash-2
fix: HP inventory crash is some warnings
2022-11-10 15:47:39 +01:00
Cyril Levis
776f951e3b
fix: hp inventory crash is controller return some warning
about the cache to be re-enable for example

```
DEBUG:urllib3.connectionpool:https://netbox.local:443 "GET /api/dcim/inventory-items/?device_id=9&tag=hw-disk&limit=0 HTTP/1.1" 200 52
('A cache backup failure has occurred. Please execute the "reenablecache" '
 'command')
Traceback (most recent call last):
  File "/usr/bin/netbox_agent", line 33, in <module>
    sys.exit(load_entry_point('netbox-agent==0.7.1', 'console_scripts', 'netbox_agent')())
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/cli.py", line 50, in main
    return run(config)
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/cli.py", line 43, in run
    server.netbox_create_or_update(config)
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/server.py", line 417, in netbox_create_or_update
    self.inventory.create_or_update()
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/inventory.py", line 547, in create_or_update
    self.do_netbox_disks()
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/inventory.py", line 417, in do_netbox_disks
    disks = self.get_hw_disks()
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/inventory.py", line 332, in get_hw_disks
    for raid_card in self.get_raid_cards(filter_cards=True):
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/inventory.py", line 242, in get_raid_cards
    self.raid = raid_class()
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/raid/hp.py", line 180, in __init__
    self.convert_to_dict()
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/raid/hp.py", line 185, in convert_to_dict
    controllers = _parse_ctrl_output(lines)
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/raid/hp.py", line 34, in _parse_ctrl_output
    attr, val = line.split(': ', 1)
```
2022-11-10 15:47:16 +01:00
Cyril Levis
84f1f00ffe
Merge pull request #247 from Solvik/fix/cle/inventory-crash-1
fix: disk inventory crash some values are None
2022-11-10 15:46:24 +01:00
Cyril Levis
8c12fa8e86
fix: crash if product or description None 2022-11-10 15:19:31 +01:00
Robin Schneider
5d0f0bf2fa
chore: Fix typo 2022-11-03 16:51:43 +01:00
Robin Schneider
4bd4b6bb94
fix: disk is also virtual if product is None
Example on QEMU/KVM:

```json
{'logicalname': '/dev/vda', 'product': None, 'serial': None, 'version': None, 'size': 10737418240, 'description': 'Virtual I/O device', 'type': 'Virtual I/O device'}
```
2022-11-03 16:51:43 +01:00
Robin Schneider
d286fde999
chore: Remove unused API parameter: device_platform 2022-11-03 16:51:42 +01:00
Robin Schneider
dfe937d54e
fix: TypeError: Object of type Record is not JSON serializable 2022-11-03 16:51:33 +01:00
Cyril Levis
042a6fcf35
Merge pull request #239 from Solvik/fix/inventory-disk-issue
Inventory disk issue
2022-08-25 08:43:36 +02:00
Cyril Levis
f4d7796094
fix: dvd-ram should be consider as virtual to be ignore 2022-08-23 09:00:31 +02:00
Cyril Levis
0b9087fa41
fix: inventory disk issue if size None
```
Traceback (most recent call last):
  File "/usr/bin/netbox_agent", line 33, in <module>
    sys.exit(load_entry_point('netbox-agent==0.7.1', 'console_scripts', 'netbox_agent')())
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/cli.py", line 50, in main
    return run(config)
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/cli.py", line 43, in run
    server.netbox_create_or_update(config)
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/server.py", line 417, in netbox_create_or_update
    self.inventory.create_or_update()
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/inventory.py", line 550, in create_or_update
    self.do_netbox_disks()
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/inventory.py", line 420, in do_netbox_disks
    disks = self.get_hw_disks()
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/inventory.py", line 344, in get_hw_disks
    size = int(disk.get('size', 0))
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'
```
2022-08-23 08:41:59 +02:00
Sylvain Rabot
79a08359ae
Add Amazon EC2 to virtual machines (#236) 2022-08-08 15:37:13 +02:00
bds-congnguyen
29d2f23986
Update requirements.txt (#233)
* Update requirements.txt

Fix `ModuleNotFoundError: No module named 'distro'` cause linux_distribution return None
2022-08-08 15:26:58 +02:00
ChrNic
838ffd8e41 using dynamic ref to NIC 2022-07-21 08:32:24 +02:00
DeathRabbit679
ea66becd3d
Use platform id instead of record (#231) 2022-07-18 21:08:03 +01:00
illes
0cff7d3477
Fix module platform has no attribute linux_distribution (#224) 2022-07-18 21:06:33 +01:00
illes
be770a947a
Fix for missing mgmt-ip in LLDP output (#225) 2022-06-20 13:13:31 +02:00
Cyril Levis
6181800cb3
Merge pull request #220 from cyrinux/fix/hp-raid
fix: hp raid, prevent us to miss some errors
2022-04-06 09:25:56 +02:00
Cyril Levis
2f23844dfd
fix: hp raid, prevent us to miss some errors when matching only returncode 1
Signed-off-by: Cyril Levis <git@levis.name>
2022-04-04 14:39:19 +02:00
Cyril Levis
5f0aae6c01
Merge pull request #218 from KivraChristoffer/Feature/fix-HP-raid
Bug: if HP raidcard have no discs
2022-04-04 09:26:51 +02:00
ChrNic
6717b43cc9 Solves if raidcard have no discs 2022-04-04 09:05:42 +02:00
Cyril Levis
05bfc69f26
Merge pull request #214 from morihaya/readme-patch
Fix netbox_agent.yml to netbox_agent.yaml
2022-04-01 08:58:23 +02:00
Cyril Levis
d06931852f
Merge pull request #216 from cyrinux/feature/platform
Added platform to the config with auto-detect as fallback
2022-04-01 08:57:24 +02:00
Cyril Levis
46354865d3
chore: update Help
Signed-off-by: Cyril Levis <git@levis.name>
2022-03-31 21:21:19 +02:00
Cyril Levis
3dbeb5b9de
chore: cleanup virtual_machines get_device_platform
Signed-off-by: Cyril Levis <git@levis.name>
2022-03-31 21:04:32 +02:00
Cyril Levis
dfb6b234ba
chore: cleanup, add device_platform in summary
Signed-off-by: Cyril Levis <git@levis.name>
2022-03-31 20:58:50 +02:00
Cyril Levis
9e797c376e
fix: always get_device_platform
Signed-off-by: Cyril Levis <git@levis.name>
2022-03-31 19:54:34 +02:00
Cyril Levis
cf2cc54da4
feat: auto-detect device platform
this platform module still works for python3.6 we use.

Signed-off-by: Cyril Levis <git@levis.name>
2022-03-31 17:33:35 +02:00
Fredrik Bergström
2a4f24f00a not failing if platform is not set 2022-03-31 11:14:26 +02:00
Fredrik Bergström
a7b965a8b5 Added platform to the config 2022-03-30 13:27:13 +02:00
Yukiya Hayashi A.k.a morihaya
ddf2e4d1cc
Fix netbox_agent.yml to netbox_agent.yaml
Unify the notation of netbox_agent.yaml in the README
2022-03-30 00:17:27 +09:00
Cyril Levis
7bf955d601
Merge pull request #213 from cyrinux/feat/add_tags_replace_params
fix(#212): prevent server tags to be override by default
2022-03-28 15:30:57 +02:00
Cyril Levis
dd4d935fb2
fix(#212): prevent server tags to be override.
Add a param `--replace-tags` to override all tags, default behavior is to append new tags.
2022-03-28 15:17:27 +02:00
Cyril Levis
ce83364ff9
Merge pull request #211 from cyrinux/feat/add_qemu_support
feat: add QEMU support
2022-03-12 10:42:28 +01:00
Cyril Levis
75f14fa895
feat: add QEMU support 2022-03-12 10:40:34 +01:00
Christophe Simon
bb08813700
Merge pull request #209 from Solvik/fix/chriss/raid_cmd_returncod_hp_jbod
Fixed HP raid parser when disks are in JBOD
2022-03-11 17:05:38 +01:00
Christophe Simon
ad951b9288 Fixed HP raid parser when disks are in JBOD
The function parsing the RAID logical volumes in the HP module did not
manage the case where disks were set in JBOD mode, thus having no RAID
array set.

This patch fixes this by checking if arrays are defined on pdisks before
parsing the logical disks.

Also added returncode read when executing the RAID related commands to
raise a more precise error.
2022-03-11 15:55:07 +01:00
40 changed files with 1854 additions and 1265 deletions

1
.envrc Normal file
View file

@ -0,0 +1 @@
use nix

3
.gitignore vendored
View file

@ -181,3 +181,6 @@ dmypy.json
# End of https://www.gitignore.io/api/emacs,python
netbox-docker
/.vscode
.direnv
.pre-commit-config.yaml

View file

@ -20,8 +20,8 @@ The goal is to generate an existing infrastructure on Netbox and have the abilit
# Requirements
- Netbox >= 2.6
- Python >= 3.4
- Netbox >= 3.7
- Python >= 3.7
- [pynetbox](https://github.com/digitalocean/pynetbox/)
- [python3-netaddr](https://github.com/drkjam/netaddr)
- [python3-netifaces](https://github.com/al45tair/netifaces)
@ -51,7 +51,7 @@ The agent can be run from a shell and get its configuration from either the conf
Configuration values are overridden based on the following precedence: command line arguments (might include config file) > environment variables > default config file > defaults.
```
# netbox_agent -c /etc/netbox_agent.yml --register
# netbox_agent -c /etc/netbox_agent.yaml --register
INFO:root:Creating chassis blade (serial: QTFCQ574502EF)
INFO:root:Creating blade (serial: QTFCQ574502D2) myserver on chassis QTFCQ574502EF
INFO:root:Setting device (QTFCQ574502D2) new slot on Slot 9 (Chassis QTFCQ574502EF)..

55
default.nix Normal file
View file

@ -0,0 +1,55 @@
{
sources ? import ./npins,
pkgs ? import sources.nixpkgs { },
}:
let
checks = (import sources.git-hooks).run {
src = ./.;
hooks =
{
commitizen.enable = true;
}
// (pkgs.lib.genAttrs
[
"black"
"isort"
"ruff"
]
(hook: {
enable = true;
stages = [ "pre-push" ];
})
);
};
python3 = pkgs.python3.override {
packageOverrides = self: _: {
netifaces2 = self.callPackage ./nix/netifaces2.nix { };
};
};
in
{
devShell = pkgs.mkShell {
name = "netbox-agent.dev";
packages = [
(python3.withPackages (ps: [
ps.pynetbox
ps.netaddr
ps.netifaces2
ps.pyyaml
ps.jsonargparse
ps.python-slugify
ps.packaging
ps.distro
]))
] ++ checks.enabledPackages;
shellHook = ''
${checks.shellHook}
'';
};
}

View file

@ -1,6 +1,7 @@
from pkg_resources import DistributionNotFound, get_distribution
from importlib.metadata import PackageNotFoundError
from importlib.metadata import version as _get_version
try:
__version__ = get_distribution(__name__).version
except DistributionNotFound:
__version__ = _get_version(__name__)
except PackageNotFoundError:
pass

View file

@ -1,4 +1,5 @@
from packaging import version
import netbox_agent.dmidecode as dmidecode
from netbox_agent.config import config
from netbox_agent.config import netbox_instance as nb
@ -11,12 +12,12 @@ from netbox_agent.vendors.supermicro import SupermicroHost
from netbox_agent.virtualmachine import VirtualMachine, is_vm
MANUFACTURERS = {
'Dell Inc.': DellHost,
'HP': HPHost,
'HPE': HPHost,
'Supermicro': SupermicroHost,
'Quanta Cloud Technology Inc.': QCTHost,
'Generic': GenericHost,
"Dell Inc.": DellHost,
"HP": HPHost,
"HPE": HPHost,
"Supermicro": SupermicroHost,
"Quanta Cloud Technology Inc.": QCTHost,
"Generic": GenericHost,
}
@ -25,21 +26,29 @@ def run(config):
if config.virtual.enabled or is_vm(dmi):
if not config.virtual.cluster_name:
raise Exception('virtual.cluster_name parameter is mandatory because it\'s a VM')
raise Exception(
"virtual.cluster_name parameter is mandatory because it's a VM"
)
server = VirtualMachine(dmi=dmi)
else:
manufacturer = dmidecode.get_by_type(dmi, 'Chassis')[0].get('Manufacturer')
manufacturer = dmidecode.get_by_type(dmi, "Chassis")[0].get("Manufacturer")
try:
server = MANUFACTURERS[manufacturer](dmi=dmi)
except KeyError:
server = GenericHost(dmi=dmi)
if version.parse(nb.version) < version.parse('2.9'):
print('netbox-agent is not compatible with Netbox prior to verison 2.9')
if version.parse(nb.version) < version.parse("3.7"):
print("netbox-agent is not compatible with Netbox prior to version 3.7")
return False
if config.register or config.update_all or config.update_network or \
config.update_location or config.update_inventory or config.update_psu:
if (
config.register
or config.update_all
or config.update_network
or config.update_location
or config.update_inventory
or config.update_psu
):
server.netbox_create_or_update(config)
if config.debug:
server.print_debug()
@ -47,8 +56,8 @@ def run(config):
def main():
return run(config)
return 0 if run(config) else 1
if __name__ == '__main__':
if __name__ == "__main__":
main()

View file

@ -10,81 +10,148 @@ import urllib3
def get_config():
p = jsonargparse.ArgumentParser(
default_config_files=[
'/etc/netbox_agent.yaml',
'~/.config/netbox_agent.yaml',
'~/.netbox_agent.yaml',
"/etc/netbox_agent.yaml",
"~/.config/netbox_agent.yaml",
"~/.netbox_agent.yaml",
],
prog='netbox_agent',
prog="netbox_agent",
description="Netbox agent to run on your infrastructure's servers",
env_prefix='NETBOX_AGENT_',
default_env=True
env_prefix="NETBOX_AGENT_",
default_env=True,
)
p.add_argument('-c', '--config', action=jsonargparse.ActionConfigFile)
p.add_argument("-c", "--config", action=jsonargparse.ActionConfigFile)
p.add_argument('-r', '--register', action='store_true', help='Register server to Netbox')
p.add_argument('-u', '--update-all', action='store_true', help='Update all infos in Netbox')
p.add_argument('-d', '--debug', action='store_true', help='Print debug infos')
p.add_argument('--update-network', action='store_true', help='Update network')
p.add_argument('--update-inventory', action='store_true', help='Update inventory')
p.add_argument('--update-location', action='store_true', help='Update location')
p.add_argument('--update-psu', action='store_true', help='Update PSU')
p.add_argument('--purge-old-devices', action='store_true',
help='Purge existing (old ?) devices having same name but different serial')
p.add_argument('--expansion-as-device', action='store_true',
help='Manage blade expansions as external devices')
p.add_argument(
"-r", "--register", action="store_true", help="Register server to Netbox"
)
p.add_argument(
"-u", "--update-all", action="store_true", help="Update all infos in Netbox"
)
p.add_argument("-d", "--debug", action="store_true", help="Print debug infos")
p.add_argument("--update-network", action="store_true", help="Update network")
p.add_argument("--update-inventory", action="store_true", help="Update inventory")
p.add_argument("--update-location", action="store_true", help="Update location")
p.add_argument("--update-psu", action="store_true", help="Update PSU")
p.add_argument(
"--purge-old-devices",
action="store_true",
help="Purge existing (old ?) devices having same name but different serial",
)
p.add_argument(
"--expansion-as-device",
action="store_true",
help="Manage blade expansions as external devices",
)
p.add_argument('--log_level', default='debug')
p.add_argument('--netbox.ssl_ca_certs_file', help='SSL CA certificates file')
p.add_argument('--netbox.url', help='Netbox URL')
p.add_argument('--netbox.token', help='Netbox API Token')
p.add_argument('--netbox.ssl_verify', default=True, action='store_true',
help='Disable SSL verification')
p.add_argument('--virtual.enabled', action='store_true', help='Is a virtual machine or not')
p.add_argument('--virtual.cluster_name', help='Cluster name of VM')
p.add_argument('--hostname_cmd', default=None,
help="Command to output hostname, used as Device's name in netbox")
p.add_argument('--device.tags', default=r'',
help='tags to use for a host')
p.add_argument('--device.custom_fields', default=r'',
help='custom_fields to use for a host, eg: field1=v1,field2=v2')
p.add_argument('--device.blade_role', default=r'Blade',
help='role to use for a blade server')
p.add_argument('--device.chassis_role', default=r'Server Chassis',
help='role to use for a chassis')
p.add_argument('--device.server_role', default=r'Server',
help='role to use for a server')
p.add_argument('--tenant.driver',
help='tenant driver, ie cmd, file')
p.add_argument('--tenant.driver_file',
help='tenant driver custom driver file path')
p.add_argument('--tenant.regex',
help='tenant regex to extract Netbox tenant slug')
p.add_argument('--datacenter_location.driver',
help='Datacenter location driver, ie: cmd, file')
p.add_argument('--datacenter_location.driver_file',
help='Datacenter location custom driver file path')
p.add_argument('--datacenter_location.regex',
help='Datacenter location regex to extract Netbox DC slug')
p.add_argument('--rack_location.driver', help='Rack location driver, ie: cmd, file')
p.add_argument('--rack_location.driver_file', help='Rack location custom driver file path')
p.add_argument('--rack_location.regex', help='Rack location regex to extract Rack name')
p.add_argument('--slot_location.driver', help='Slot location driver, ie: cmd, file')
p.add_argument('--slot_location.driver_file', help='Slot location custom driver file path')
p.add_argument('--slot_location.regex', help='Slot location regex to extract slot name')
p.add_argument('--network.ignore_interfaces', default=r'(dummy.*|docker.*)',
help='Regex to ignore interfaces')
p.add_argument('--network.ignore_ips', default=r'^(127\.0\.0\..*|fe80.*|::1.*)',
help='Regex to ignore IPs')
p.add_argument('--network.lldp', help='Enable auto-cabling feature through LLDP infos')
p.add_argument('--inventory', action='store_true',
help='Enable HW inventory (CPU, Memory, RAID Cards, Disks) feature')
p.add_argument('--process-virtual-drives', action='store_true',
help='Process virtual drives information from RAID '
'controllers to fill disk custom_fields')
p.add_argument('--force-disk-refresh', action='store_true',
help='Forces disks detection reprocessing')
p.add_argument('--dump-disks-map',
help='File path to dump physical/virtual disks map')
p.add_argument("--log_level", default="debug")
p.add_argument("--netbox.ssl_ca_certs_file", help="SSL CA certificates file")
p.add_argument("--netbox.url", help="Netbox URL")
p.add_argument("--netbox.token", help="Netbox API Token")
p.add_argument(
"--netbox.ssl_verify",
default=True,
action="store_true",
help="Disable SSL verification",
)
p.add_argument(
"--virtual.enabled", action="store_true", help="Is a virtual machine or not"
)
p.add_argument("--virtual.cluster_name", help="Cluster name of VM")
p.add_argument(
"--hostname_cmd",
default=None,
help="Command to output hostname, used as Device's name in netbox",
)
p.add_argument(
"--device.platform",
default=None,
help="Override device platform. Here we use OS distribution.",
)
p.add_argument("--device.tags", default=r"", help="tags to use for a host")
p.add_argument(
"--preserve-tags",
action="store_true",
help="Append new unique tags, preserve those already present",
)
p.add_argument(
"--device.custom_fields",
default=r"",
help="custom_fields to use for a host, eg: field1=v1,field2=v2",
)
p.add_argument(
"--device.blade_role", default=r"Blade", help="role to use for a blade server"
)
p.add_argument(
"--device.chassis_role",
default=r"Server Chassis",
help="role to use for a chassis",
)
p.add_argument(
"--device.server_role", default=r"Server", help="role to use for a server"
)
p.add_argument("--tenant.driver", help="tenant driver, ie cmd, file")
p.add_argument("--tenant.driver_file", help="tenant driver custom driver file path")
p.add_argument("--tenant.regex", help="tenant regex to extract Netbox tenant slug")
p.add_argument(
"--datacenter_location.driver", help="Datacenter location driver, ie: cmd, file"
)
p.add_argument(
"--datacenter_location.driver_file",
help="Datacenter location custom driver file path",
)
p.add_argument(
"--datacenter_location.regex",
help="Datacenter location regex to extract Netbox DC slug",
)
p.add_argument("--rack_location.driver", help="Rack location driver, ie: cmd, file")
p.add_argument(
"--rack_location.driver_file", help="Rack location custom driver file path"
)
p.add_argument(
"--rack_location.regex", help="Rack location regex to extract Rack name"
)
p.add_argument("--slot_location.driver", help="Slot location driver, ie: cmd, file")
p.add_argument(
"--slot_location.driver_file", help="Slot location custom driver file path"
)
p.add_argument(
"--slot_location.regex", help="Slot location regex to extract slot name"
)
p.add_argument(
"--network.ignore_interfaces",
default=r"(dummy.*|docker.*)",
help="Regex to ignore interfaces",
)
p.add_argument(
"--network.ignore_ips",
default=r"^(127\.0\.0\..*|fe80.*|::1.*)",
help="Regex to ignore IPs",
)
p.add_argument(
"--network.ipmi", default=True, help="Enable gathering IPMI information"
)
p.add_argument(
"--network.lldp", help="Enable auto-cabling feature through LLDP infos"
)
p.add_argument(
"--inventory",
action="store_true",
help="Enable HW inventory (CPU, Memory, RAID Cards, Disks) feature",
)
p.add_argument(
"--process-virtual-drives",
action="store_true",
help="Process virtual drives information from RAID "
"controllers to fill disk custom_fields",
)
p.add_argument(
"--force-disk-refresh",
action="store_true",
help="Forces disks detection reprocessing",
)
p.add_argument(
"--dump-disks-map", help="File path to dump physical/virtual disks map"
)
options = p.parse_args()
return options
@ -95,7 +162,7 @@ config = get_config()
def get_netbox_instance():
if config.netbox.url is None or config.netbox.token is None:
logging.error('Netbox URL and token are mandatory')
logging.error("Netbox URL and token are mandatory")
sys.exit(1)
nb = pynetbox.api(

View file

@ -5,55 +5,57 @@ import sys
from netbox_agent.misc import is_tool
_handle_re = _re.compile('^Handle\\s+(.+),\\s+DMI\\s+type\\s+(\\d+),\\s+(\\d+)\\s+bytes$')
_in_block_re = _re.compile('^\\t\\t(.+)$')
_record_re = _re.compile('\\t(.+):\\s+(.+)$')
_record2_re = _re.compile('\\t(.+):$')
_handle_re = _re.compile(
"^Handle\\s+(.+),\\s+DMI\\s+type\\s+(\\d+),\\s+(\\d+)\\s+bytes$"
)
_in_block_re = _re.compile("^\\t\\t(.+)$")
_record_re = _re.compile("\\t(.+):\\s+(.+)$")
_record2_re = _re.compile("\\t(.+):$")
_type2str = {
0: 'BIOS',
1: 'System',
2: 'Baseboard',
3: 'Chassis',
4: 'Processor',
5: 'Memory Controller',
6: 'Memory Module',
7: 'Cache',
8: 'Port Connector',
9: 'System Slots',
10: ' On Board Devices',
11: ' OEM Strings',
12: ' System Configuration Options',
13: ' BIOS Language',
14: ' Group Associations',
15: ' System Event Log',
16: ' Physical Memory Array',
17: ' Memory Device',
18: ' 32-bit Memory Error',
19: ' Memory Array Mapped Address',
20: ' Memory Device Mapped Address',
21: ' Built-in Pointing Device',
22: ' Portable Battery',
23: ' System Reset',
24: ' Hardware Security',
25: ' System Power Controls',
26: ' Voltage Probe',
27: ' Cooling Device',
28: ' Temperature Probe',
29: ' Electrical Current Probe',
30: ' Out-of-band Remote Access',
31: ' Boot Integrity Services',
32: ' System Boot',
33: ' 64-bit Memory Error',
34: ' Management Device',
35: ' Management Device Component',
36: ' Management Device Threshold Data',
37: ' Memory Channel',
38: ' IPMI Device',
39: ' Power Supply',
40: ' Additional Information',
41: ' Onboard Devices Extended Information',
42: ' Management Controller Host Interface'
0: "BIOS",
1: "System",
2: "Baseboard",
3: "Chassis",
4: "Processor",
5: "Memory Controller",
6: "Memory Module",
7: "Cache",
8: "Port Connector",
9: "System Slots",
10: " On Board Devices",
11: " OEM Strings",
12: " System Configuration Options",
13: " BIOS Language",
14: " Group Associations",
15: " System Event Log",
16: " Physical Memory Array",
17: " Memory Device",
18: " 32-bit Memory Error",
19: " Memory Array Mapped Address",
20: " Memory Device Mapped Address",
21: " Built-in Pointing Device",
22: " Portable Battery",
23: " System Reset",
24: " Hardware Security",
25: " System Power Controls",
26: " Voltage Probe",
27: " Cooling Device",
28: " Temperature Probe",
29: " Electrical Current Probe",
30: " Out-of-band Remote Access",
31: " Boot Integrity Services",
32: " System Boot",
33: " 64-bit Memory Error",
34: " Management Device",
35: " Management Device Component",
36: " Management Device Threshold Data",
37: " Memory Channel",
38: " IPMI Device",
39: " Power Supply",
40: " Additional Information",
41: " Onboard Devices Extended Information",
42: " Management Controller Host Interface",
}
_str2type = {}
for type_id, type_str in _type2str.items():
@ -70,7 +72,7 @@ def parse(output=None):
else:
buffer = _execute_cmd()
if isinstance(buffer, bytes):
buffer = buffer.decode('utf-8')
buffer = buffer.decode("utf-8")
_data = _parse(buffer)
return _data
@ -129,24 +131,31 @@ def get_by_type(data, type_id):
result = []
for entry in data.values():
if entry['DMIType'] == type_id:
if entry["DMIType"] == type_id:
result.append(entry)
return result
def _execute_cmd():
if not is_tool('dmidecode'):
logging.error('Dmidecode does not seem to be present on your system. Add it your path or '
'check the compatibility of this project with your distro.')
if not is_tool("dmidecode"):
logging.error(
"Dmidecode does not seem to be present on your system. Add it your path or "
"check the compatibility of this project with your distro."
)
sys.exit(1)
return _subprocess.check_output(['dmidecode', ], stderr=_subprocess.PIPE)
return _subprocess.check_output(
[
"dmidecode",
],
stderr=_subprocess.PIPE,
)
def _parse(buffer):
output_data = {}
# Each record is separated by double newlines
split_output = buffer.split('\n\n')
split_output = buffer.split("\n\n")
for record in split_output:
record_element = record.splitlines()
@ -164,21 +173,21 @@ def _parse(buffer):
dmi_handle = handle_data[0]
output_data[dmi_handle] = {}
output_data[dmi_handle]['DMIType'] = int(handle_data[1])
output_data[dmi_handle]['DMISize'] = int(handle_data[2])
output_data[dmi_handle]["DMIType"] = int(handle_data[1])
output_data[dmi_handle]["DMISize"] = int(handle_data[2])
# Okay, we know 2nd line == name
output_data[dmi_handle]['DMIName'] = record_element[1]
output_data[dmi_handle]["DMIName"] = record_element[1]
in_block_elemet = ''
in_block_list = ''
in_block_elemet = ""
in_block_list = ""
# Loop over the rest of the record, gathering values
for i in range(2, len(record_element), 1):
if i >= len(record_element):
break
# Check whether we are inside a \t\t block
if in_block_elemet != '':
if in_block_elemet != "":
in_block_data = _in_block_re.findall(record_element[i])
if in_block_data:
@ -192,7 +201,7 @@ def _parse(buffer):
else:
# We are out of the \t\t block; reset it again, and let
# the parsing continue
in_block_elemet = ''
in_block_elemet = ""
record_data = _record_re.findall(record_element[i])
@ -208,7 +217,7 @@ def _parse(buffer):
# This is an array of data - let the loop know we are inside
# an array block
in_block_elemet = record_data2[0]
in_block_list = ''
in_block_list = ""
continue

View file

@ -2,7 +2,7 @@ import re
def get(value, regex):
for line in open(value, 'r'):
for line in open(value, "r"):
r = re.search(regex, line)
if r and len(r.groups()) > 0:
return r.groups()[0]

View file

@ -6,16 +6,16 @@ from shutil import which
# mapping fields from ethtool output to simple names
field_map = {
'Supported ports': 'ports',
'Supported link modes': 'sup_link_modes',
'Supports auto-negotiation': 'sup_autoneg',
'Advertised link modes': 'adv_link_modes',
'Advertised auto-negotiation': 'adv_autoneg',
'Speed': 'speed',
'Duplex': 'duplex',
'Port': 'port',
'Auto-negotiation': 'autoneg',
'Link detected': 'link',
"Supported ports": "ports",
"Supported link modes": "sup_link_modes",
"Supports auto-negotiation": "sup_autoneg",
"Advertised link modes": "adv_link_modes",
"Advertised auto-negotiation": "adv_autoneg",
"Speed": "speed",
"Duplex": "duplex",
"Port": "port",
"Auto-negotiation": "autoneg",
"Link detected": "link",
}
@ -25,7 +25,7 @@ def merge_two_dicts(x, y):
return z
class Ethtool():
class Ethtool:
"""
This class aims to parse ethtool output
There is several bindings to have something proper, but it requires
@ -40,39 +40,40 @@ class Ethtool():
parse ethtool output
"""
output = subprocess.getoutput('ethtool {}'.format(self.interface))
output = subprocess.getoutput("ethtool {}".format(self.interface))
fields = {}
field = ''
fields['speed'] = '-'
fields['link'] = '-'
fields['duplex'] = '-'
for line in output.split('\n')[1:]:
field = ""
fields["speed"] = "-"
fields["link"] = "-"
fields["duplex"] = "-"
for line in output.split("\n")[1:]:
line = line.rstrip()
r = line.find(':')
r = line.find(":")
if r > 0:
field = line[:r].strip()
if field not in field_map:
continue
field = field_map[field]
output = line[r + 1:].strip()
output = line[r + 1 :].strip()
fields[field] = output
else:
if len(field) > 0 and \
field in field_map:
fields[field] += ' ' + line.strip()
if len(field) > 0 and field in field_map:
fields[field] += " " + line.strip()
return fields
def _parse_ethtool_module_output(self):
status, output = subprocess.getstatusoutput('ethtool -m {}'.format(self.interface))
status, output = subprocess.getstatusoutput(
"ethtool -m {}".format(self.interface)
)
if status == 0:
r = re.search(r'Identifier.*\((\w+)\)', output)
r = re.search(r"Identifier.*\((\w+)\)", output)
if r and len(r.groups()) > 0:
return {'form_factor': r.groups()[0]}
return {"form_factor": r.groups()[0]}
return {}
def parse(self):
if which('ethtool') is None:
if which("ethtool") is None:
return None
output = self._parse_ethtool_output()
output.update(self._parse_ethtool_module_output())

View file

@ -1,3 +1,11 @@
import json
import logging
import re
import sys
import traceback
import pynetbox
from netbox_agent.config import config
from netbox_agent.config import netbox_instance as nb
from netbox_agent.lshw import LSHW
@ -5,25 +13,19 @@ from netbox_agent.misc import get_vendor, is_tool
from netbox_agent.raid.hp import HPRaid
from netbox_agent.raid.omreport import OmreportRaid
from netbox_agent.raid.storcli import StorcliRaid
import traceback
import pynetbox
import logging
import json
import re
INVENTORY_TAG = {
'cpu': {'name': 'hw:cpu', 'slug': 'hw-cpu'},
'gpu': {'name': 'hw:gpu', 'slug': 'hw-gpu'},
'disk': {'name': 'hw:disk', 'slug': 'hw-disk'},
'interface': {'name': 'hw:interface', 'slug': 'hw-interface'},
'memory': {'name': 'hw:memory', 'slug': 'hw-memory'},
'motherboard': {'name': 'hw:motherboard', 'slug': 'hw-motherboard'},
'raid_card': {'name': 'hw:raid_card', 'slug': 'hw-raid-card'},
"cpu": {"name": "hw:cpu", "slug": "hw-cpu"},
"gpu": {"name": "hw:gpu", "slug": "hw-gpu"},
"disk": {"name": "hw:disk", "slug": "hw-disk"},
"interface": {"name": "hw:interface", "slug": "hw-interface"},
"memory": {"name": "hw:memory", "slug": "hw-memory"},
"motherboard": {"name": "hw:motherboard", "slug": "hw-motherboard"},
"raid_card": {"name": "hw:raid_card", "slug": "hw-raid-card"},
}
class Inventory():
class Inventory:
"""
Better Inventory items coming, see:
- https://github.com/netbox-community/netbox/issues/3087
@ -62,14 +64,12 @@ class Inventory():
def create_netbox_tags(self):
ret = []
for key, tag in INVENTORY_TAG.items():
nb_tag = nb.extras.tags.get(
name=tag['name']
)
nb_tag = nb.extras.tags.get(name=tag["name"])
if not nb_tag:
nb_tag = nb.extras.tags.create(
name=tag['name'],
slug=tag['slug'],
comments=tag['name'],
name=tag["name"],
slug=tag["slug"],
comments=tag["name"],