Commit graph

180 commits

Author SHA1 Message Date
Cyril Levis
528ecc09b0
fix: allow the inventory of several hp controller of the same model 2023-01-13 17:12:25 +01:00
Cyril Levis
4cf054278f
fix: prevent crash if disk fields attrs missing 2022-12-14 09:00:37 +01:00
Cyril Levis
49b269efa6
chore: fetch more data as custom_fields for hp inventory 2022-12-13 17:27:35 +01:00
Cyril Levis
cedb6818a3
fix: handle big list of hp disk
we cant use p.wait() if too much data.
2022-12-13 17:26:45 +01:00
Cyril Levis
a9af96bba2
fix: get disk serial number 2022-12-13 17:26:14 +01:00
Cyril Levis
ca2a69b66f
fix: enable disk inventory for HPE 2022-12-09 14:19:26 +01:00
Cyril Levis
e0685e7167
fix: add space in ignore_patterns
to prevent false positive like `/data/cache/temp` for eg
2022-12-02 12:28:47 +01:00
Cyril Levis
bf65da0c58
fix: deadlock storcli inventory
When there is a lot of data return by storcli due to a lot of disk, the inventory was in deadlock
Its a known problem, see:
https://docs.python.org/3/library/subprocess.html#subprocess.Popen.wait
2022-11-30 09:49:51 +01:00
Cyril Levis
2d8ec831e6
fix: dvd rom inventory crash
dvdrom dont have serial number, we fallback on a 'unknown' value now to prevent crash
2022-11-17 11:00:48 +01:00
Cyril Levis
e96a50379b
chore: refactor hp raid ignore errors handling 2022-11-16 15:46:57 +01:00
Cyril Levis
633b6d3851
fix: ignore more cache errors 2022-11-16 14:43:35 +01:00
Cyril Levis
9b06584fed
fix: ignore some more hp controllers errors while parsing 2022-11-16 10:45:25 +01:00
Cyril Levis
f7cdd92fa3
Merge pull request #232 from KivraChristoffer/fix-#226
This will solve issue #226
2022-11-10 15:53:46 +01:00
Cyril Levis
bce02a5e7c
Merge branch 'master' into fix/various 2022-11-10 15:51:32 +01:00
Cyril Levis
776f951e3b
fix: hp inventory crash is controller return some warning
about the cache to be re-enable for example

```
DEBUG:urllib3.connectionpool:https://netbox.local:443 "GET /api/dcim/inventory-items/?device_id=9&tag=hw-disk&limit=0 HTTP/1.1" 200 52
('A cache backup failure has occurred. Please execute the "reenablecache" '
 'command')
Traceback (most recent call last):
  File "/usr/bin/netbox_agent", line 33, in <module>
    sys.exit(load_entry_point('netbox-agent==0.7.1', 'console_scripts', 'netbox_agent')())
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/cli.py", line 50, in main
    return run(config)
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/cli.py", line 43, in run
    server.netbox_create_or_update(config)
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/server.py", line 417, in netbox_create_or_update
    self.inventory.create_or_update()
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/inventory.py", line 547, in create_or_update
    self.do_netbox_disks()
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/inventory.py", line 417, in do_netbox_disks
    disks = self.get_hw_disks()
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/inventory.py", line 332, in get_hw_disks
    for raid_card in self.get_raid_cards(filter_cards=True):
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/inventory.py", line 242, in get_raid_cards
    self.raid = raid_class()
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/raid/hp.py", line 180, in __init__
    self.convert_to_dict()
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/raid/hp.py", line 185, in convert_to_dict
    controllers = _parse_ctrl_output(lines)
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/raid/hp.py", line 34, in _parse_ctrl_output
    attr, val = line.split(': ', 1)
```
2022-11-10 15:47:16 +01:00
Cyril Levis
8c12fa8e86
fix: crash if product or description None 2022-11-10 15:19:31 +01:00
Robin Schneider
5d0f0bf2fa
chore: Fix typo 2022-11-03 16:51:43 +01:00
Robin Schneider
4bd4b6bb94
fix: disk is also virtual if product is None
Example on QEMU/KVM:

```json
{'logicalname': '/dev/vda', 'product': None, 'serial': None, 'version': None, 'size': 10737418240, 'description': 'Virtual I/O device', 'type': 'Virtual I/O device'}
```
2022-11-03 16:51:43 +01:00
Robin Schneider
d286fde999
chore: Remove unused API parameter: device_platform 2022-11-03 16:51:42 +01:00
Robin Schneider
dfe937d54e
fix: TypeError: Object of type Record is not JSON serializable 2022-11-03 16:51:33 +01:00
Cyril Levis
f4d7796094
fix: dvd-ram should be consider as virtual to be ignore 2022-08-23 09:00:31 +02:00
Cyril Levis
0b9087fa41
fix: inventory disk issue if size None
```
Traceback (most recent call last):
  File "/usr/bin/netbox_agent", line 33, in <module>
    sys.exit(load_entry_point('netbox-agent==0.7.1', 'console_scripts', 'netbox_agent')())
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/cli.py", line 50, in main
    return run(config)
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/cli.py", line 43, in run
    server.netbox_create_or_update(config)
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/server.py", line 417, in netbox_create_or_update
    self.inventory.create_or_update()
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/inventory.py", line 550, in create_or_update
    self.do_netbox_disks()
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/inventory.py", line 420, in do_netbox_disks
    disks = self.get_hw_disks()
  File "/opt/netbox-agent/lib/python3.6/site-packages/netbox_agent/inventory.py", line 344, in get_hw_disks
    size = int(disk.get('size', 0))
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'
```
2022-08-23 08:41:59 +02:00
Sylvain Rabot
79a08359ae
Add Amazon EC2 to virtual machines (#236) 2022-08-08 15:37:13 +02:00
ChrNic
838ffd8e41 using dynamic ref to NIC 2022-07-21 08:32:24 +02:00
DeathRabbit679
ea66becd3d
Use platform id instead of record (#231) 2022-07-18 21:08:03 +01:00
illes
0cff7d3477
Fix module platform has no attribute linux_distribution (#224) 2022-07-18 21:06:33 +01:00
illes
be770a947a
Fix for missing mgmt-ip in LLDP output (#225) 2022-06-20 13:13:31 +02:00
Cyril Levis
2f23844dfd
fix: hp raid, prevent us to miss some errors when matching only returncode 1
Signed-off-by: Cyril Levis <git@levis.name>
2022-04-04 14:39:19 +02:00
ChrNic
6717b43cc9 Solves if raidcard have no discs 2022-04-04 09:05:42 +02:00
Cyril Levis
46354865d3
chore: update Help
Signed-off-by: Cyril Levis <git@levis.name>
2022-03-31 21:21:19 +02:00
Cyril Levis
3dbeb5b9de
chore: cleanup virtual_machines get_device_platform
Signed-off-by: Cyril Levis <git@levis.name>
2022-03-31 21:04:32 +02:00
Cyril Levis
dfb6b234ba
chore: cleanup, add device_platform in summary
Signed-off-by: Cyril Levis <git@levis.name>
2022-03-31 20:58:50 +02:00
Cyril Levis
9e797c376e
fix: always get_device_platform
Signed-off-by: Cyril Levis <git@levis.name>
2022-03-31 19:54:34 +02:00
Cyril Levis
cf2cc54da4
feat: auto-detect device platform
this platform module still works for python3.6 we use.

Signed-off-by: Cyril Levis <git@levis.name>
2022-03-31 17:33:35 +02:00
Fredrik Bergström
2a4f24f00a not failing if platform is not set 2022-03-31 11:14:26 +02:00
Fredrik Bergström
a7b965a8b5 Added platform to the config 2022-03-30 13:27:13 +02:00
Cyril Levis
dd4d935fb2
fix(#212): prevent server tags to be override.
Add a param `--replace-tags` to override all tags, default behavior is to append new tags.
2022-03-28 15:17:27 +02:00
Cyril Levis
75f14fa895
feat: add QEMU support 2022-03-12 10:40:34 +01:00
Christophe Simon
ad951b9288 Fixed HP raid parser when disks are in JBOD
The function parsing the RAID logical volumes in the HP module did not
manage the case where disks were set in JBOD mode, thus having no RAID
array set.

This patch fixes this by checking if arrays are defined on pdisks before
parsing the logical disks.

Also added returncode read when executing the RAID related commands to
raise a more precise error.
2022-03-11 15:55:07 +01:00
Christophe Simon
3b8917aaf3 Fixed virtual network cards creation
This patch fixes a mismatch between the way network interfaces are
looked for from network cards, and the way they are created:

- If a network card has a NAC address, it used in the search criteria to
  find the network interface
- When creating virtual interfaces, the MAC address is not specified in
  the creation parameters

This raises an exception when a virtual card has a MAC address:

- The virtual interface has been created without MAC address
- But the network card has one, so the interface is looked for using it
- The network interface is not found, so it's created
- An error is risen because an interface with the same name already
  exists

This patch sets the MAC address on the interface if it exists, no matter if
it is virtual or not.
2022-03-08 17:27:06 +01:00
Christophe Simon
84a9aca141 Unnamed network interfaces.
Some interfaces do not have device (logical) name (eth0, for instance), such
as not connected network mezzanine cards in blade servers.

In such situations, the card will be named `unknown[0-9]`.
2022-03-07 17:18:35 +01:00
Christophe Simon
e789619b34 Added disks extended attributes
This patch brings some of the physical and virtual drive attributes as
`custom_fields` to the disks inventory.

The goal is to have this information present to ease disks maintenance
when a drive becomes unavailable and its attributes can't be read anymore
from the RAID controller.

It also helps to standardize the extended disk attributes across the
different manufacturers.

As the disk physical identifers were not available under the correct
format (hexadecimal format using the `xml` output as opposed as `X:Y:Z` format
using the default `list` format), the command line parser has been
refactored to read the `list` format, rather than `xml` one in the
`omreport` raid controller parser.

As the custom fields have to be created prior being able to register
the disks extended attributes, this feature is only activated using the
`--process-virtual-drives` command line parameter, or by setting
`process_virtual_drives` to `true` in the configuration file.

The custom fields to create as `DCIM > inventory item` `Text` are described
below.

    NAME            LABEL                      DESCRIPTION
    mount_point     Mount point                Device mount point(s)
    pd_identifier   Physical disk identifier   Physical disk identifier in the RAID controller
    vd_array        Virtual drive array        Virtual drive array the disk is member of
    vd_consistency  Virtual drive consistency  Virtual disk array consistency
    vd_device       Virtual drive device       Virtual drive system device
    vd_raid_type    Virtual drive RAID         Virtual drive array RAID type
    vd_size         Virtual drive size         Virtual drive array size

In the current implementation, the disks attributes ore not updated: if
a disk with the correct serial number is found, it's sufficient to
consider it as up to date.

To force the reprocessing of the disks extended attributes, the
`--force-disk-refresh` command line option can be used: it removes all
existing disks to before populating them with the correct parsing.
Unless this option is specified, the extended attributes won't be
modified unless a disk is replaced.

It is possible to dump the physical/virtual disks map on the filesystem under
the JSON notation to ease or automate disks management. The file path has to
be provided using the `--dump-disks-map` command line parameter.
2022-03-02 15:53:38 +01:00
Christophe Simon
af9df9ab4b Fixed PSU power consumption calculation
When PSUs were found, the voltage value was never set, raising an
exception when trying to multiply current * voltage (float * None).

This patch reads per PSU voltage (with 230V as default) and define power
consumption from it.
2022-02-23 16:06:32 +01:00
Cyril LEVIS
d1ee380ffb
feat: add custom_fields suppport
```bash
$ netbox_agent -du --device.custom_fields="last_run=$(date),last_run_ts=$(date +'%s')"
```

obviously, custom_fields need to be create manually in netbox admin
2022-02-22 17:45:25 +01:00
Christophe Simon
305d4d41ec Various changes
- Added an option to specify an SSL CA certificates file to talk to the
  Netbox API
- Enhanced GPU expansion bays inventory: internal GPU (VGA) goes into
  the blade, external GPU (3D) goes into the expansion bay
- Unified the way expansion bays are managed (GPU and drive exansion
  bays)
- Started to refactor `network` module to make it more readable
- Dependencies in `setup.py` now reads its requirements from
  `requirements.txt` to avoid double maintenance
2022-02-22 14:51:58 +01:00
Christophe Simon
34c1619ce8 Merge remote-tracking branch 'origin/master' into 176-netbox-2.9 2022-02-21 18:09:25 +01:00
Christophe Simon
5d9602402b
Merge branch 'master' into fix/chriss/purge_remaining_devices 2022-02-15 10:34:50 +01:00
Christophe Simon
2f09cf8d42 Manage blade expansions as independent devices
This patch adds the ability to detect and manage GPU and Disk expansion
bays, and either add their internal components into the device
corresponding to the blade server, or into a dedicated device.

It takes advantage of the work made by @cyrinux on GPU bays management, and
applies the same principle to the external disk bays, but harmonize the
inventory management:

- If no argument is specified on the command line, the GPU cards, RAID
  controllers and their attached disks are added in the blade device,
  and the device corresponding to an expansion device is deleted.
- If the `--expansion-as-device` option is specified on the command
  line, a dedicated device corresponding to the expansion bay is
  created, and the GPUs, RAID card and attached disks are removed from
  the blade device and added to the expansion device.
2022-02-14 14:37:02 +01:00
Christophe Simon
58775c0950 Added option to purge remaining devices
This patch adds an option to clear remaining devices in the context of a
new device replacing an old one with the same name but with a different
hardware.

The serial leading to the bug is described below:

- A first server is registered under name `A` with serial `X`
- A second server is registered under name `B` with serial `Y`
- The server with serial `X` is decomissionned, but not removed
- The server with serial `Y` is reinstalled with name `A`

In this case, a device with serial `X` is well found, and netbox agent
tries to update its name. But this raises an error because of the unique
constraint on name, as another device already holds this name.

The proposed solution to handle this situation is to delete any device
with same `name` but different `serial` before handling a device.

As this is not necessarily the expected behavior, it can be enabled by
using the `--purge-old-devices` to avoid breaking existing inventory.
2022-02-08 18:46:30 +01:00
Cyrinux
8a46af19b8
Fix a crash when missing rack id (#172) 2021-07-20 21:55:47 +02:00