netbox-agent/netbox_agent/raid/omreport.py

141 lines
4.4 KiB
Python
Raw Permalink Normal View History

2019-09-10 11:44:23 +02:00
from netbox_agent.raid.base import Raid, RaidController
Added disks extended attributes This patch brings some of the physical and virtual drive attributes as `custom_fields` to the disks inventory. The goal is to have this information present to ease disks maintenance when a drive becomes unavailable and its attributes can't be read anymore from the RAID controller. It also helps to standardize the extended disk attributes across the different manufacturers. As the disk physical identifers were not available under the correct format (hexadecimal format using the `xml` output as opposed as `X:Y:Z` format using the default `list` format), the command line parser has been refactored to read the `list` format, rather than `xml` one in the `omreport` raid controller parser. As the custom fields have to be created prior being able to register the disks extended attributes, this feature is only activated using the `--process-virtual-drives` command line parameter, or by setting `process_virtual_drives` to `true` in the configuration file. The custom fields to create as `DCIM > inventory item` `Text` are described below. NAME LABEL DESCRIPTION mount_point Mount point Device mount point(s) pd_identifier Physical disk identifier Physical disk identifier in the RAID controller vd_array Virtual drive array Virtual drive array the disk is member of vd_consistency Virtual drive consistency Virtual disk array consistency vd_device Virtual drive device Virtual drive system device vd_raid_type Virtual drive RAID Virtual drive array RAID type vd_size Virtual drive size Virtual drive array size In the current implementation, the disks attributes ore not updated: if a disk with the correct serial number is found, it's sufficient to consider it as up to date. To force the reprocessing of the disks extended attributes, the `--force-disk-refresh` command line option can be used: it removes all existing disks to before populating them with the correct parsing. Unless this option is specified, the extended attributes won't be modified unless a disk is replaced. It is possible to dump the physical/virtual disks map on the filesystem under the JSON notation to ease or automate disks management. The file path has to be provided using the `--dump-disks-map` command line parameter.
2022-02-25 18:43:09 +01:00
from netbox_agent.misc import get_vendor, get_mount_points
from netbox_agent.config import config
import subprocess
import logging
import re
2019-09-10 11:44:23 +02:00
class OmreportControllerError(Exception):
pass
Added disks extended attributes This patch brings some of the physical and virtual drive attributes as `custom_fields` to the disks inventory. The goal is to have this information present to ease disks maintenance when a drive becomes unavailable and its attributes can't be read anymore from the RAID controller. It also helps to standardize the extended disk attributes across the different manufacturers. As the disk physical identifers were not available under the correct format (hexadecimal format using the `xml` output as opposed as `X:Y:Z` format using the default `list` format), the command line parser has been refactored to read the `list` format, rather than `xml` one in the `omreport` raid controller parser. As the custom fields have to be created prior being able to register the disks extended attributes, this feature is only activated using the `--process-virtual-drives` command line parameter, or by setting `process_virtual_drives` to `true` in the configuration file. The custom fields to create as `DCIM > inventory item` `Text` are described below. NAME LABEL DESCRIPTION mount_point Mount point Device mount point(s) pd_identifier Physical disk identifier Physical disk identifier in the RAID controller vd_array Virtual drive array Virtual drive array the disk is member of vd_consistency Virtual drive consistency Virtual disk array consistency vd_device Virtual drive device Virtual drive system device vd_raid_type Virtual drive RAID Virtual drive array RAID type vd_size Virtual drive size Virtual drive array size In the current implementation, the disks attributes ore not updated: if a disk with the correct serial number is found, it's sufficient to consider it as up to date. To force the reprocessing of the disks extended attributes, the `--force-disk-refresh` command line option can be used: it removes all existing disks to before populating them with the correct parsing. Unless this option is specified, the extended attributes won't be modified unless a disk is replaced. It is possible to dump the physical/virtual disks map on the filesystem under the JSON notation to ease or automate disks management. The file path has to be provided using the `--dump-disks-map` command line parameter.
2022-02-25 18:43:09 +01:00
def omreport(sub_command):
command = ["omreport"]
command.extend(sub_command.split())
p = subprocess.Popen(
command,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT
)
p.wait()
stdout = p.stdout.read().decode("utf-8")
if p.returncode != 0:
mesg = "Failed to execute command '{}':\n{}".format(
" ".join(command), stdout
)
raise OmreportControllerError(mesg)
Added disks extended attributes This patch brings some of the physical and virtual drive attributes as `custom_fields` to the disks inventory. The goal is to have this information present to ease disks maintenance when a drive becomes unavailable and its attributes can't be read anymore from the RAID controller. It also helps to standardize the extended disk attributes across the different manufacturers. As the disk physical identifers were not available under the correct format (hexadecimal format using the `xml` output as opposed as `X:Y:Z` format using the default `list` format), the command line parser has been refactored to read the `list` format, rather than `xml` one in the `omreport` raid controller parser. As the custom fields have to be created prior being able to register the disks extended attributes, this feature is only activated using the `--process-virtual-drives` command line parameter, or by setting `process_virtual_drives` to `true` in the configuration file. The custom fields to create as `DCIM > inventory item` `Text` are described below. NAME LABEL DESCRIPTION mount_point Mount point Device mount point(s) pd_identifier Physical disk identifier Physical disk identifier in the RAID controller vd_array Virtual drive array Virtual drive array the disk is member of vd_consistency Virtual drive consistency Virtual disk array consistency vd_device Virtual drive device Virtual drive system device vd_raid_type Virtual drive RAID Virtual drive array RAID type vd_size Virtual drive size Virtual drive array size In the current implementation, the disks attributes ore not updated: if a disk with the correct serial number is found, it's sufficient to consider it as up to date. To force the reprocessing of the disks extended attributes, the `--force-disk-refresh` command line option can be used: it removes all existing disks to before populating them with the correct parsing. Unless this option is specified, the extended attributes won't be modified unless a disk is replaced. It is possible to dump the physical/virtual disks map on the filesystem under the JSON notation to ease or automate disks management. The file path has to be provided using the `--dump-disks-map` command line parameter.
2022-02-25 18:43:09 +01:00
res = {}
section_re = re.compile('^[A-Z]')
current_section = None
current_obj = None
2019-09-10 11:44:23 +02:00
for line in stdout.split('\n'):
Added disks extended attributes This patch brings some of the physical and virtual drive attributes as `custom_fields` to the disks inventory. The goal is to have this information present to ease disks maintenance when a drive becomes unavailable and its attributes can't be read anymore from the RAID controller. It also helps to standardize the extended disk attributes across the different manufacturers. As the disk physical identifers were not available under the correct format (hexadecimal format using the `xml` output as opposed as `X:Y:Z` format using the default `list` format), the command line parser has been refactored to read the `list` format, rather than `xml` one in the `omreport` raid controller parser. As the custom fields have to be created prior being able to register the disks extended attributes, this feature is only activated using the `--process-virtual-drives` command line parameter, or by setting `process_virtual_drives` to `true` in the configuration file. The custom fields to create as `DCIM > inventory item` `Text` are described below. NAME LABEL DESCRIPTION mount_point Mount point Device mount point(s) pd_identifier Physical disk identifier Physical disk identifier in the RAID controller vd_array Virtual drive array Virtual drive array the disk is member of vd_consistency Virtual drive consistency Virtual disk array consistency vd_device Virtual drive device Virtual drive system device vd_raid_type Virtual drive RAID Virtual drive array RAID type vd_size Virtual drive size Virtual drive array size In the current implementation, the disks attributes ore not updated: if a disk with the correct serial number is found, it's sufficient to consider it as up to date. To force the reprocessing of the disks extended attributes, the `--force-disk-refresh` command line option can be used: it removes all existing disks to before populating them with the correct parsing. Unless this option is specified, the extended attributes won't be modified unless a disk is replaced. It is possible to dump the physical/virtual disks map on the filesystem under the JSON notation to ease or automate disks management. The file path has to be provided using the `--dump-disks-map` command line parameter.
2022-02-25 18:43:09 +01:00
if ': ' in line:
attr, value = line.split(': ', 1)
attr = attr.strip()
value = value.strip()
if attr == 'ID':
obj = {}
res.setdefault(current_section, []).append(obj)
current_obj = obj
current_obj[attr] = value
elif section_re.search(line) is not None:
current_section = line.strip()
return res
2019-09-10 11:44:23 +02:00
class OmreportController(RaidController):
def __init__(self, controller_index, data):
self.data = data
self.controller_index = controller_index
def get_product_name(self):
Added disks extended attributes This patch brings some of the physical and virtual drive attributes as `custom_fields` to the disks inventory. The goal is to have this information present to ease disks maintenance when a drive becomes unavailable and its attributes can't be read anymore from the RAID controller. It also helps to standardize the extended disk attributes across the different manufacturers. As the disk physical identifers were not available under the correct format (hexadecimal format using the `xml` output as opposed as `X:Y:Z` format using the default `list` format), the command line parser has been refactored to read the `list` format, rather than `xml` one in the `omreport` raid controller parser. As the custom fields have to be created prior being able to register the disks extended attributes, this feature is only activated using the `--process-virtual-drives` command line parameter, or by setting `process_virtual_drives` to `true` in the configuration file. The custom fields to create as `DCIM > inventory item` `Text` are described below. NAME LABEL DESCRIPTION mount_point Mount point Device mount point(s) pd_identifier Physical disk identifier Physical disk identifier in the RAID controller vd_array Virtual drive array Virtual drive array the disk is member of vd_consistency Virtual drive consistency Virtual disk array consistency vd_device Virtual drive device Virtual drive system device vd_raid_type Virtual drive RAID Virtual drive array RAID type vd_size Virtual drive size Virtual drive array size In the current implementation, the disks attributes ore not updated: if a disk with the correct serial number is found, it's sufficient to consider it as up to date. To force the reprocessing of the disks extended attributes, the `--force-disk-refresh` command line option can be used: it removes all existing disks to before populating them with the correct parsing. Unless this option is specified, the extended attributes won't be modified unless a disk is replaced. It is possible to dump the physical/virtual disks map on the filesystem under the JSON notation to ease or automate disks management. The file path has to be provided using the `--dump-disks-map` command line parameter.
2022-02-25 18:43:09 +01:00
return self.data['Name']
2019-09-10 11:44:23 +02:00
def get_manufacturer(self):
return None
def get_serial_number(self):
Added disks extended attributes This patch brings some of the physical and virtual drive attributes as `custom_fields` to the disks inventory. The goal is to have this information present to ease disks maintenance when a drive becomes unavailable and its attributes can't be read anymore from the RAID controller. It also helps to standardize the extended disk attributes across the different manufacturers. As the disk physical identifers were not available under the correct format (hexadecimal format using the `xml` output as opposed as `X:Y:Z` format using the default `list` format), the command line parser has been refactored to read the `list` format, rather than `xml` one in the `omreport` raid controller parser. As the custom fields have to be created prior being able to register the disks extended attributes, this feature is only activated using the `--process-virtual-drives` command line parameter, or by setting `process_virtual_drives` to `true` in the configuration file. The custom fields to create as `DCIM > inventory item` `Text` are described below. NAME LABEL DESCRIPTION mount_point Mount point Device mount point(s) pd_identifier Physical disk identifier Physical disk identifier in the RAID controller vd_array Virtual drive array Virtual drive array the disk is member of vd_consistency Virtual drive consistency Virtual disk array consistency vd_device Virtual drive device Virtual drive system device vd_raid_type Virtual drive RAID Virtual drive array RAID type vd_size Virtual drive size Virtual drive array size In the current implementation, the disks attributes ore not updated: if a disk with the correct serial number is found, it's sufficient to consider it as up to date. To force the reprocessing of the disks extended attributes, the `--force-disk-refresh` command line option can be used: it removes all existing disks to before populating them with the correct parsing. Unless this option is specified, the extended attributes won't be modified unless a disk is replaced. It is possible to dump the physical/virtual disks map on the filesystem under the JSON notation to ease or automate disks management. The file path has to be provided using the `--dump-disks-map` command line parameter.
2022-02-25 18:43:09 +01:00
return self.data.get('DeviceSerialNumber')
2019-09-10 11:44:23 +02:00
def get_firmware_version(self):
Added disks extended attributes This patch brings some of the physical and virtual drive attributes as `custom_fields` to the disks inventory. The goal is to have this information present to ease disks maintenance when a drive becomes unavailable and its attributes can't be read anymore from the RAID controller. It also helps to standardize the extended disk attributes across the different manufacturers. As the disk physical identifers were not available under the correct format (hexadecimal format using the `xml` output as opposed as `X:Y:Z` format using the default `list` format), the command line parser has been refactored to read the `list` format, rather than `xml` one in the `omreport` raid controller parser. As the custom fields have to be created prior being able to register the disks extended attributes, this feature is only activated using the `--process-virtual-drives` command line parameter, or by setting `process_virtual_drives` to `true` in the configuration file. The custom fields to create as `DCIM > inventory item` `Text` are described below. NAME LABEL DESCRIPTION mount_point Mount point Device mount point(s) pd_identifier Physical disk identifier Physical disk identifier in the RAID controller vd_array Virtual drive array Virtual drive array the disk is member of vd_consistency Virtual drive consistency Virtual disk array consistency vd_device Virtual drive device Virtual drive system device vd_raid_type Virtual drive RAID Virtual drive array RAID type vd_size Virtual drive size Virtual drive array size In the current implementation, the disks attributes ore not updated: if a disk with the correct serial number is found, it's sufficient to consider it as up to date. To force the reprocessing of the disks extended attributes, the `--force-disk-refresh` command line option can be used: it removes all existing disks to before populating them with the correct parsing. Unless this option is specified, the extended attributes won't be modified unless a disk is replaced. It is possible to dump the physical/virtual disks map on the filesystem under the JSON notation to ease or automate disks management. The file path has to be provided using the `--dump-disks-map` command line parameter.
2022-02-25 18:43:09 +01:00
return self.data.get('Firmware Version')
def _get_physical_disks(self):
pds = {}
res = omreport('storage pdisk controller={}'.format(
self.controller_index
))
for pdisk in [d for d in list(res.values())[0]]:
disk_id = pdisk['ID']
size = re.sub('B .*$', 'B', pdisk['Capacity'])
pds[disk_id] = {
'Vendor': get_vendor(pdisk['Vendor ID']),
'Model': pdisk['Product ID'],
'SN': pdisk['Serial No.'],
'Size': size,
'Type': pdisk['Media'],
'_src': self.__class__.__name__,
}
return pds
def _get_virtual_drives_map(self):
pds = {}
res = omreport('storage vdisk controller={}'.format(
self.controller_index
))
for vdisk in [d for d in list(res.values())[0]]:
vdisk_id = vdisk['ID']
device = vdisk['Device Name']
mount_points = get_mount_points()
mp = mount_points.get(device, 'n/a')
size = re.sub('B .*$', 'B', vdisk['Size'])
vd = {
'vd_array': vdisk_id,
'vd_size': size,
'vd_consistency': vdisk['State'],
'vd_raid_type': vdisk['Layout'],
'vd_device': vdisk['Device Name'],
'mount_point': ', '.join(sorted(mp)),
}
drives_res = omreport(
'storage pdisk controller={} vdisk={}'.format(
self.controller_index, vdisk_id
))
for pdisk in [d for d in list(drives_res.values())[0]]:
pds[pdisk['ID']] = vd
return pds
2019-09-10 11:44:23 +02:00
def get_physical_disks(self):
Added disks extended attributes This patch brings some of the physical and virtual drive attributes as `custom_fields` to the disks inventory. The goal is to have this information present to ease disks maintenance when a drive becomes unavailable and its attributes can't be read anymore from the RAID controller. It also helps to standardize the extended disk attributes across the different manufacturers. As the disk physical identifers were not available under the correct format (hexadecimal format using the `xml` output as opposed as `X:Y:Z` format using the default `list` format), the command line parser has been refactored to read the `list` format, rather than `xml` one in the `omreport` raid controller parser. As the custom fields have to be created prior being able to register the disks extended attributes, this feature is only activated using the `--process-virtual-drives` command line parameter, or by setting `process_virtual_drives` to `true` in the configuration file. The custom fields to create as `DCIM > inventory item` `Text` are described below. NAME LABEL DESCRIPTION mount_point Mount point Device mount point(s) pd_identifier Physical disk identifier Physical disk identifier in the RAID controller vd_array Virtual drive array Virtual drive array the disk is member of vd_consistency Virtual drive consistency Virtual disk array consistency vd_device Virtual drive device Virtual drive system device vd_raid_type Virtual drive RAID Virtual drive array RAID type vd_size Virtual drive size Virtual drive array size In the current implementation, the disks attributes ore not updated: if a disk with the correct serial number is found, it's sufficient to consider it as up to date. To force the reprocessing of the disks extended attributes, the `--force-disk-refresh` command line option can be used: it removes all existing disks to before populating them with the correct parsing. Unless this option is specified, the extended attributes won't be modified unless a disk is replaced. It is possible to dump the physical/virtual disks map on the filesystem under the JSON notation to ease or automate disks management. The file path has to be provided using the `--dump-disks-map` command line parameter.
2022-02-25 18:43:09 +01:00
pds = self._get_physical_disks()
vds = self._get_virtual_drives_map()
for pd_identifier, vd in vds.items():
if pd_identifier not in pds:
logging.error(
'Physical drive {} listed in virtual drive {} not '
'found in drives list'.format(
pd_identifier, vd['vd_array']
)
)
continue
pds[pd_identifier].setdefault('custom_fields', {}).update(vd)
pds[pd_identifier]['custom_fields']['pd_identifier'] = pd_identifier
return list(pds.values())
2019-09-10 11:44:23 +02:00
class OmreportRaid(Raid):
def __init__(self):
self.controllers = []
Added disks extended attributes This patch brings some of the physical and virtual drive attributes as `custom_fields` to the disks inventory. The goal is to have this information present to ease disks maintenance when a drive becomes unavailable and its attributes can't be read anymore from the RAID controller. It also helps to standardize the extended disk attributes across the different manufacturers. As the disk physical identifers were not available under the correct format (hexadecimal format using the `xml` output as opposed as `X:Y:Z` format using the default `list` format), the command line parser has been refactored to read the `list` format, rather than `xml` one in the `omreport` raid controller parser. As the custom fields have to be created prior being able to register the disks extended attributes, this feature is only activated using the `--process-virtual-drives` command line parameter, or by setting `process_virtual_drives` to `true` in the configuration file. The custom fields to create as `DCIM > inventory item` `Text` are described below. NAME LABEL DESCRIPTION mount_point Mount point Device mount point(s) pd_identifier Physical disk identifier Physical disk identifier in the RAID controller vd_array Virtual drive array Virtual drive array the disk is member of vd_consistency Virtual drive consistency Virtual disk array consistency vd_device Virtual drive device Virtual drive system device vd_raid_type Virtual drive RAID Virtual drive array RAID type vd_size Virtual drive size Virtual drive array size In the current implementation, the disks attributes ore not updated: if a disk with the correct serial number is found, it's sufficient to consider it as up to date. To force the reprocessing of the disks extended attributes, the `--force-disk-refresh` command line option can be used: it removes all existing disks to before populating them with the correct parsing. Unless this option is specified, the extended attributes won't be modified unless a disk is replaced. It is possible to dump the physical/virtual disks map on the filesystem under the JSON notation to ease or automate disks management. The file path has to be provided using the `--dump-disks-map` command line parameter.
2022-02-25 18:43:09 +01:00
res = omreport('storage controller')
2019-09-10 11:44:23 +02:00
Added disks extended attributes This patch brings some of the physical and virtual drive attributes as `custom_fields` to the disks inventory. The goal is to have this information present to ease disks maintenance when a drive becomes unavailable and its attributes can't be read anymore from the RAID controller. It also helps to standardize the extended disk attributes across the different manufacturers. As the disk physical identifers were not available under the correct format (hexadecimal format using the `xml` output as opposed as `X:Y:Z` format using the default `list` format), the command line parser has been refactored to read the `list` format, rather than `xml` one in the `omreport` raid controller parser. As the custom fields have to be created prior being able to register the disks extended attributes, this feature is only activated using the `--process-virtual-drives` command line parameter, or by setting `process_virtual_drives` to `true` in the configuration file. The custom fields to create as `DCIM > inventory item` `Text` are described below. NAME LABEL DESCRIPTION mount_point Mount point Device mount point(s) pd_identifier Physical disk identifier Physical disk identifier in the RAID controller vd_array Virtual drive array Virtual drive array the disk is member of vd_consistency Virtual drive consistency Virtual disk array consistency vd_device Virtual drive device Virtual drive system device vd_raid_type Virtual drive RAID Virtual drive array RAID type vd_size Virtual drive size Virtual drive array size In the current implementation, the disks attributes ore not updated: if a disk with the correct serial number is found, it's sufficient to consider it as up to date. To force the reprocessing of the disks extended attributes, the `--force-disk-refresh` command line option can be used: it removes all existing disks to before populating them with the correct parsing. Unless this option is specified, the extended attributes won't be modified unless a disk is replaced. It is possible to dump the physical/virtual disks map on the filesystem under the JSON notation to ease or automate disks management. The file path has to be provided using the `--dump-disks-map` command line parameter.
2022-02-25 18:43:09 +01:00
for controller in res['Controller']:
ctrl_index = controller['ID']
2019-09-10 11:44:23 +02:00
self.controllers.append(
Added disks extended attributes This patch brings some of the physical and virtual drive attributes as `custom_fields` to the disks inventory. The goal is to have this information present to ease disks maintenance when a drive becomes unavailable and its attributes can't be read anymore from the RAID controller. It also helps to standardize the extended disk attributes across the different manufacturers. As the disk physical identifers were not available under the correct format (hexadecimal format using the `xml` output as opposed as `X:Y:Z` format using the default `list` format), the command line parser has been refactored to read the `list` format, rather than `xml` one in the `omreport` raid controller parser. As the custom fields have to be created prior being able to register the disks extended attributes, this feature is only activated using the `--process-virtual-drives` command line parameter, or by setting `process_virtual_drives` to `true` in the configuration file. The custom fields to create as `DCIM > inventory item` `Text` are described below. NAME LABEL DESCRIPTION mount_point Mount point Device mount point(s) pd_identifier Physical disk identifier Physical disk identifier in the RAID controller vd_array Virtual drive array Virtual drive array the disk is member of vd_consistency Virtual drive consistency Virtual disk array consistency vd_device Virtual drive device Virtual drive system device vd_raid_type Virtual drive RAID Virtual drive array RAID type vd_size Virtual drive size Virtual drive array size In the current implementation, the disks attributes ore not updated: if a disk with the correct serial number is found, it's sufficient to consider it as up to date. To force the reprocessing of the disks extended attributes, the `--force-disk-refresh` command line option can be used: it removes all existing disks to before populating them with the correct parsing. Unless this option is specified, the extended attributes won't be modified unless a disk is replaced. It is possible to dump the physical/virtual disks map on the filesystem under the JSON notation to ease or automate disks management. The file path has to be provided using the `--dump-disks-map` command line parameter.
2022-02-25 18:43:09 +01:00
OmreportController(ctrl_index, controller)
2019-09-10 11:44:23 +02:00
)
def get_controllers(self):
return self.controllers