Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 31 Next »

This article details how to generate report using the RestAPI which will mirror as closely as possible the "OutputDevices" report which was available in V3 of iQSonar.
You can download the current working version of this script here, and simply customise the $user, $pass and $sonar values and it should run in your environment


Fields in the original report

This information is taken from the iQSonar Developer API guide for iQSonar V3

FieldV3 CommentRestAPI Comment
HostnameHostname of the deviceGet this from the /devices RestAPI endpoint 
FQDNFully QUalified Domain Name of the deviceGet this from the /devices/{Device_ID} RestAPI endpoint
OSOS description and service packGet this from the /devices/{Device_ID} RestAPI endpoint
OS Install DateOS description and service packGet this from the /devices/{Device_ID} RestAPI endpoint
May not always be available
LocationLocation of the deviceIn V4 this represents the target configuration set used to define the device.
Get this from the /devices RestAPI endpoint
Serial NumberThe serial number of the deviceGet this from the /devices RestAPI endpoint
PhysicalCPUCountNumber of physical CPUs or sockets in the Physical Device (which may differ from the Virtual or Logical host when a device is a virtual machine or partition)

IF the device is a physical device, this information is derived from the /devices/{Device_ID} RestAPI endpoint
IF the device is a virtual device AND we have scanned the virtualisation host, the information is also available.

Where the virtualisation host is not scanned, then this field and all the other information about the physical host will not be able to be populated.

PhysicalCoreCountSum of cores across all CPUs in the Physical Device (which may differ from the Virtual or Logical host when a device is a virtual machine or partition)Get this from the /devices/{Device_ID} RestAPI endpoint; May not always be available
PhysicalCoresPerCPUPhysical core count divided by the physical CPU countGet this from the /devices/{Device_ID} RestAPI endpoint; May not always be available
PhysicalCPUManufacturerManufacturer of the CPU on the Physical DeviceGet this from the /devices/{Device_ID} RestAPI endpoint; May not always be available
PhysicalCPUModelModel of the CPU on the Physical DeviceGet this from the /devices/{Device_ID} RestAPI endpoint; May not always be available
PhysicalCPUSpeedSpeed of the CPU on the Physical DeviceGet this from the /devices/{Device_ID} RestAPI endpoint; May not always be available
PhysicalRAMTotal amount of RAM on the Physical DeviceGet this from the /devices/{Device_ID} RestAPI endpoint; May not always be available
VirtualCPUCountNumber of Virtual CPUs perceived by the device (only populated when the device is Virtual or Logical)Get this from the /devices/{Device_ID} RestAPI endpoint; Only available if device is virtual or logical
VirtualCoreCountSum of Cores perceived by the device (only populated when the device is Virtual or Logical)Get this from the /devices/{Device_ID} RestAPI endpoint; Only available if device is virtual or logical
VirtualRAMAmount of perceived RAM by the device (only populated when the device is Virtual or Logical)Get this from the /devices/{Device_ID} RestAPI endpoint; Only available if device is virtual or logical
DeviceModelModel of the deviceGet this from the /devices RestAPI endpoint for a physical device.
PhysicalModelSocketCountNumber of sockets that may be populated with physical CPUs on the Physical DeviceGet this from the /devices/{Device_ID} RestAPI endpoint; May not always be available
PhysicalModelCoreCountMaximum number of cores per CPU according to the model documentationGet this from the /devices/{Device_ID} RestAPI endpoint; May not always be available
PhysicalDeviceManufacturerManufacturer of the Physical DeviceGet this from the /devices/{Device_ID} RestAPI endpoint; May not always be available
PhysicalHostnameHostname of the Physical DeviceGet this from the /devices/{Device_ID} RestAPI endpoint for virtual devices; May not always be available
PhysicalFQDNFully Qualified Domain Name of the Physical DeviceGet this from the /devices/{Device_ID} RestAPI endpoint; May not always be available
IP Address
  • If more than 1, will be semi-colon separated. 
  • Info such as DHCPServer from IPAddress table will not be in OutputDevices. 
  • Will only include IPv4.
V4 also exposes IP6 addresses.
PhysicalMACAddressIf more than 1, will be semi-colon separated.
VirtualMACAddressIf more than 1, will be semi-colon separated.
ClusterInformationThe virtualization cluster that the physical machine is part ofOnly available if it is part of a cluster, and the cluster has been scanned
ClusterNameName of clusterOnly available if it is part of a cluster, and the cluster has been scanned
PartitioningMethodVirtualization Method (VMware, HyperV, LPAR etc.). If this field is null it indicates the device is physical.
DerivedCPU
Not directly available via RestAPI. Must be coded in the script.
DerivedCoresPerCPU
Not directly available via RestAPI. Must be coded in the script.
BiosConcatenation of Device.BIOSName, DeviceBIOSManufacturer, Device.BIOSVersion separated by semi-colons.Get this from the /devices/{Device_ID} RestAPI endpoint.
LastScanDateLast date the device was scannedGet this from the /devices/{Device_ID} RestAPI endpoint.
DeviceIDThe unique identifier for this deviceThe Rest API gives unique identifiers in GUID format
PhysicalDeviceIDThe unique identifier of the Physical Device (where relevant) – used to map logical devices or Virtual machines back to the Physical Device which hosts them.Get this from the /devices/{Device_ID} RestAPI endpoint; May not always be available
PhysicalCPUNotesNotes that indicate CPU vs socket mismatches or CPU core values that don’t match the CPU modelThis V3 data is not directly available via RestAPI. This functionality is part of DataHub for V4, and if required must be coded in the script.
NotesNotes related to the device modelThis V3 data is not directly available via RestAPI. This functionality is part of DataHub for V4, and if required must be coded in the script.
ExternalLinkLink to model documentation from the vendorThis V3 data is not available via RestAPI.
DNSHostnameThe hostname of the device as reported from DNSGet this from the /devices/{Device_ID} RestAPI endpoint; May not always be available
DNSFQDNThe fully qualified hostname of the device as reported from DNSGet this from the /devices/{Device_ID} RestAPI endpoint; May not always be available
PhysicalDNSHostnameThe hostname of the physical device as reported from DNSGet this from the /devices/{Device_ID} RestAPI endpoint; May not always be available
PhysicalDNSFQDNThe fully qualified hostname of the physical device as reported from DNSGet this from the /devices/{Device_ID} RestAPI endpoint; May not always be available
MeasurementCommentWill contain additional info such as “Believed to be: Oracle Linux Server release 5.7” where appropriateThis V3 data is not available via RestAPI.

Connect to the RestAPI

The RestAPI uses HTTP basic authentication rather than domain credentials. To write a script that extracts data from the RestAPI you will need to know the login name and password for an iQSonar user who has the "access Rest API" permission enabled. By default the admin user always has this permission. These variables will need to be configured for your site:

Initial connection
$user    = "admin"
$pass    = "password"
$sonar	 = "iQSonar Host"
$secpass = ConvertTo-SecureString $pass -AsPlainText -Force
$credential = New-Object System.Management.Automation.PSCredential($user,$secpass)

The first step is to determine exactly how many devices we're going to be collecting data for. This information is returned in the header of the RestAPI return value. By default a call to the devices endpoint will return a batch of 200 devices, and the user is expected to page through the list of available devices by making successive calls to this endpoint using an offset parameter to specify how many devices have already been seen. The fetch_size parameter is used to determine how many devices are returned per batch. The fastest way to determine the total number of available devices is to explicitly request the first device only. Since we want data from the headers of the result, we use the Invoke-WebRequest PowerShell cmdlet to get this information.

Get total number of devices
$uri     = -join ("http://", $sonar, "/api/v1/devices/?offset=1&fetch_size=1")
$r = Invoke-WebRequest $uri -Credential $credential
 
# $r.headers has HTML headers, $r.content has text content
$deviceCount = $r.headers.'X-fetch-count'

Build the CSV file header row

PowerShell has a number of built in libraries for handling output to various file formats. For this example we will be saving the results in a CSV file that can be viewed directly in EXCEL or imported into other databases.

Note that one limitation of using the PowerShell System.Object to build the CSV file is that the order of the colums is not fixed. 


Define columns for the CSV file
# Build the CSV File header row
$csv = @()
$row = New-Object System.Object
$row | Add-Member -MemberType NoteProperty -Name "Hostname" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "FQDN" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "OS" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "OS-Install-Date" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "Location" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "Serial-Number" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUCount" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalCoreCount" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalCoresPerCPU" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUManufacturer" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUModel" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUSpeed" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalRAM" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "VirtualCPUCount" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "VirtualCoreCount" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "VirtualRAM" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "DeviceModel" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalModelSocketCount" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalModelCoreCount" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalDeviceManufacturer" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalHostname" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalFQDN" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "IP-Address" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalMACAddress" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "VirtualMACAddress" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "ClusterInformation" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "ClusterName" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PartitioningMethod" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "DerivedCPU" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "DerivedCoresPerCPU" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "Bios" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "LastScanDate" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "DeviceID" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalDeviceID" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUNotes" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "Notes" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "ExternalLink" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "DNSHostname" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "DNSFQDN" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalDNSHostname" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalDNSFQDN" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "MeasurementComment" -Value $null

Process the list of devices

In the RestAPI the "devices" endpoint returns some summary information about each device and a link to the rest of the information about the device.

In an estate with fewer than about 500 devices scanned, you may well find that the most efficient way to process the list of devices is to pull all of the the summary information in one go. You can specify this by setting "fetch_size=all", and only use one outer loop. However in larger estates we would expect that a fetch_size of between 100 and 500 would produce best performance, and the default of 200 devices at a time is a good place to start. To demonstrate, rather than populating the CSV file, let us just print out the host name of each device seen. We use the Invoke-RestMethod powershell command to get the results in JSON format and convert them automatically to a PowerShell object.

Trivial processing - get the host names

Nested loops to process devices
$fs     = 100			# fetch_size
$offset = 1				# offset
$seen = 1;				# first offset is 1, not 0
while ( $seen -lt $devicecount)
{
	# Outer loop, grab a batch (defined by fetch_size) of devices
	$uri = -join ("http://", $sonar, "/api/v1/devices/?offset=", $offset,  "&fetch_size=", $fs)
	$devices = Invoke-RestMethod $url -Credential $credential
	$i = 1
	while ($i -lt $devices.count)
	{
		# inner loop - process individual devices in this batch

		# Example Only
		$str = -join( "Hostname is: ", $devices[$i].host_name)
		write-host $str
		
		$i = $i + 1;		# keep track for inner loop
		$seen = $seen + 1;  # keep track for outer loop
	}
	# Finished this batch
	$offset = $seen
}

Slightly more advanced, get all information available from the devices endpoint

Of the required fields, we can populate a small number of them directly from the devices endpoint without delving any deeper into the RestAPI. This includes the Hostname, the Serial Number, the DeviceID, and the Location. In many cases we can also get the DNS FQDN.

This code fragment also demonstrates dealing with missing values. In the JSON returned by the RestAPI, if the device is missing certain information the key-value pairs are simply not returned in most cases. Your code needs to handle this gracefully.

In other cases (for example qualified_name) there can be an array of values returned in no particular order. You need to loop over the array looking for the value you're interested in (e.g. FQDN or DNSFQDN) and ignoring the values you are not interested in (e.g. UnixName)

Process info from the Devices endpoint
$fs     = 100			# fetch_size
$offset = 1				# offset
$seen   = 1;			# first offset is 1, not 0
while ( $seen -lt $devicecount)
{
	# Outer loop, grab a batch (defined by fetch_size) of devices
	$url = -join ("http://", $sonar, "/api/v1/devices/?offset=", $offset,  "&fetch_size=", $fs)
	$devices = Invoke-RestMethod $url -Credential $credential
	$i = 1
	while ($i -lt $devices.count)
	{
		# inner loop - process individual devices in this batch
		# We want to populate some of the data fields in the CSV file - what can we fill in just from the DEVICES end point
		$row = New-Object System.Object		
		$row | Add-Member -MemberType NoteProperty -Name "DeviceID" -Value $devices[$i].device_id
		
		# If the path was only partly successful we might not have a serial number. However we can safely store the empty string.
		# if you are processing RestAPI using PHP or a different scripting language that is stricter about handling undefined variables,
		# you need error handling code around here too.
		$row | Add-Member -MemberType NoteProperty -Name "Serial-Number" -Value $devices[$i].serial_number
		
		# For certain devices, for example some SNMP devices, or for storage devices we might not get a hostname! Explicitly call out a device with no hostname. 
		# Also if hostname is not defined, there will be no qualified name(s)
		if (!$devices[$i].host_name)
		{
			# Hostname is not defined
			$row | Add-Member -MemberType NoteProperty -Name "Hostname" -Value "(undefined)"
			$row | Add-Member -MemberType NoteProperty -Name "FQDN" -Value "(undefined)"
			$row | Add-Member -MemberType NoteProperty -Name "DNSFQDN" -Value "(undefined)"
		}
		else {
			# Hostname is defined
			$row | Add-Member -MemberType NoteProperty -Name "Hostname" -Value $devices[$i].host_name
			# If the array is not present, $num gets value 0, and so the inner loop will not fire
			# we have already set FQDN and DNSFQDN to "(undefined)" where there is no hostname, but we can sometimes have a hostname but no FQDN
			$num = $devices[$i].qualified_name.count
			if ($num -eq 0)
			{
				$row | Add-Member -MemberType NoteProperty -Name "FQDN" -Value "(undefined)"
				$row | Add-Member -MemberType NoteProperty -Name "DNSFQDN" -Value "(undefined)"
			}
			else
			{
				$j   = 0
				$fqdn = "(No FQDN)"
				while ($j -lt $num)
				{
					if ($devices[$i].qualified_name[$j].name_type -eq "DNSFQDN")
					{
						# DNSFQDN can also be used for FQDN.
						$fqdn = $devices[$i].qualified_name[$j].name
						$row | Add-Member -MemberType NoteProperty -Name "DNSFQDN" -Value $devices[$i].qualified_name[$j].name
						$row | Add-Member -MemberType NoteProperty -Name "FQDN" -Value $devices[$i].qualified_name[$j].name
					}
					elseif ($devices[$i].qualified_name[$j].name_type -eq "FQDN")
					{
						$row | Add-Member -MemberType NoteProperty -Name "FQDN" -Value $devices[$i].qualified_name[$j].name
					}
					$j = $j + 1
				}
			}		
		}
		# Locations is an array containing one or more locations. We can store locations[0].name as the location
		$row | Add-Member -MemberType NoteProperty -Name "Location" -Value $devices[$i].locations[0].name

		# The Last Scan Date should always be there, but if absent PowerShell will happily record a NULL value for us.
		$row | Add-Member -MemberType NoteProperty -Name "LastScanDate" -Value $devices[$i].last_scan
				
		$csv += $row
		$i = $i + 1;		# keep track for inner loop
		$seen = $seen + 1;  # keep track for outer loop
	}
	# Finished this batch
	$offset = $seen
}

Providing feedback to the user

Once we start iterating down into subsequent API calls for individual devices, the script will run for much longer than when we limit ourselves to the top level data. Therefore we need to provide some feedback so that the user does not assume the script has crashed. When we have finally processed all of the data, we can report to the user that we are finished, and save the CSV file.

User Feedback
#
# To find out how many devices there are, ask for the first device and look at the header for the device count
# offset=1 means start with first device. fetch_size=1 means only return one record.
$uri     = -join ("http://", $sonar, "/api/v1/devices/?offset=1&fetch_size=1")
$r = Invoke-WebRequest $uri -Credential $credential
# $r.headers has HTML headers, $r.content has text content
$deviceCount = $r.headers.'X-fetch-count'

# Let the user know how many devices we can see
$output = -join ( "There are: ", $deviceCount, " devices in total")
write-host $output
#
# ... set up CSV
#

$fs     = 100			# fetch_size
$offset = 1				# offset
$seen   = 1;			# first offset is 1, not 0
while ( $seen -lt $devicecount)
{
	# Outer loop, grab a batch (defined by fetch_size) of devices
	$url = -join ("http://", $sonar, "/api/v1/devices/?offset=", $offset,  "&fetch_size=", $fs)
	$devices = Invoke-RestMethod $url -Credential $credential
	$i = 1
	while ($i -lt $devices.count)
	{
		# inner loop - process individual devices in this batch

		# do something with the data

		$csv += $row
		$i = $i + 1;		# keep track for inner loop
		$seen = $seen + 1;  # keep track for outer loop
		if ( $seen % 10 -eq 0) {
			# progress indicator - display a "." every 10 devices
			write-host "." -nonewline
		}
	}
	# Finished this batch
	$offset = $seen
}
write-host " Done. Saving output to OutputDevices.csv now." 
# Save the output to a file
$csv | Export-csv OutputDevices.csv -NoTypeInformation



Getting the rest of the info

There is a lot more information saved about each device which is not returned in the summary provided by the devices endpoint.

The way to access this information is by using the additional endpoints defined by devices/device-id

This endpoint in turn contain links to more endpoints. As of Gwynn R3, these endpoints are:

  • devices/device-id/applications
  • devices/device-id/components
  • devices/device-id/installed_software

For the purposes of the OutputDevices report, all the information we are going to use is in the devices/device-id endpoint. A link to this endpoint us returned in the summary data in the "self" variable.

Get additional data
# Get extra data about the host
$currDevice = Invoke-RestMethod $devices[$i].self -Credential $credential

Some of the data returned by this second invocation can be stored directly into the CSV fields, for example BIOS information or OS Install date, as long as it is present:

Store simple data
$row | Add-Member -MemberType NoteProperty -Name "OS" -Value $currDevice.operating_system.name
if (!$currDevice.os_install_date)
{
	$row | Add-Member -MemberType NoteProperty -Name "OS-Install-Date" -Value "(not available)"
}
else
{
	$row | Add-Member -MemberType NoteProperty -Name "OS-Install-Date" -Value $currDevice.os_install_date
}
if (!$currDevice.bios.name)
{
	$row | Add-Member -MemberType NoteProperty -Name "Bios" -Value $null
}
else 
{
	$row | Add-Member -MemberType NoteProperty -Name "Bios" -Value $currDevice.bios.name
} 

Some of the data requires additional logic. For example we might have no IP address (for a SAN datastore attached to a VMWare vCenter), we might have exactly one IP address, or we might have multiple addresses (for example a node in an Oracle RAC)

Handle IP Address
# IP Address is an array of 0 or more IP Addresses. We might not have any, or we might have LOTS.
$num = $currDevice.ip_address.count
if ($num -eq 0)
{
	$row | Add-Member -MemberType NoteProperty -Name "IP-Address" -Value "(no IP address)"
}
else 
{
	if ($num -eq 1)
	{
		$row | Add-Member -MemberType NoteProperty -Name "IP-Address" -Value $currDevice.ip_address[0].ip_address
	}
	else {
		# list of addresses, needs to be separated by semi-colons
		$addresslist = ""
		$j = 0
		while ($j -lt $num)
		{
			$addresslist = -join($addresslist, $currDevice.ip_address[$j].ip_address)
			if ($j -lt $num)
			{
				$addresslist = -join($addresslist, ";")
			}
			$j = $j + 1
		}
		$row | Add-Member -MemberType NoteProperty -Name "IP-Address" -Value $addresslist
	}
}

Additional details for physical hosts

Details for Physical hosts
# Is this device physical or virtual
if ($currDevice.is_virtual -eq "false")
{
	# This is a physical target
	# We can fill in physical hardware details directly, and leave the entries for virtual hardware blank
	
	# Physical device RAM
	$row | Add-Member -MemberType NoteProperty -Name "PhysicalRAM" -Value = total_memory_mb
			
	# Physical Device CPU Info
	# Sometimes we don't get CPU Information
	if ( ($currDevice.cpu_count -eq 0) -or (!$currDevice.cpu_count) )
	{
		# Physical CPU info not avialble
		$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUCount" -Value "n/a"
		$row | Add-Member -MemberType NoteProperty -Name "PhysicalCoreCount" -Value "n/a"
		$row | Add-Member -MemberType NoteProperty -Name "PhysicalCoresPerCPU" -Value "n/a"
		$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUManufacturer" -Value "n/a"
		$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUModel" -Value "n/a"
		$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUSpeed" -Value "n/a"
	}
	else {
		# we have at least some CPU info
		$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUCount" -Value $currDevice.cpu_count
		$row | Add-Member -MemberType NoteProperty -Name "PhysicalCoreCount" -Value $currDevice.core_count
		# Further details will be derived from cpu[0] if present
		if (!$currDevice.cpu[0].manufacturer)
		{
			# additional info missing
			$row | Add-Member -MemberType NoteProperty -Name "PhysicalCoresPerCPU" -Value "n/a"
			$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUManufacturer" -Value "n/a"
			$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUModel" -Value "n/a"
			$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUSpeed" -Value "n/a"
		}
		else
		{
			$row | Add-Member -MemberType NoteProperty -Name "PhysicalCoresPerCPU" -Value $currDevice.cpu[0].core_count
			$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUManufacturer" -Value $currDevice.cpu[0].manufacturer
			$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUModel" -Value $currDevice.cpu[0].cpu_model
			$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUSpeed" -Value $currDevice.cpu[0].cpu_speed
		}
	}
	$row | Add-Member -MemberType NoteProperty -Name "PhysicalModelSocketCount" -Value "n/a" # not exposed in V4 RestAPI
	$row | Add-Member -MemberType NoteProperty -Name "PhysicalModelCoreCount" -Value "n/a" # not exposed in V4 RestAPI
	$row | Add-Member -MemberType NoteProperty -Name "PhysicalDeviceManufacturer" -Value $currDevice.manufacturer

	$row | Add-Member -MemberType NoteProperty -Name "PhysicalHostname" -Value $devices[$i].host_name			
	# 
	# Add this later
	# $row | Add-Member -MemberType NoteProperty -Name "PhysicalMACAddress" -Value $null			
}

Additional details for virtual hosts

For virtual hosts, we will have virtual device info corresponding to the physical device info stored above.

Additionally if iQSonar has scanned the virtualization host and identified the host that the VM is running on, we will also have the details for the host server and a pointer to any cluster that the host might be part of.

Physical and Virtual info for VMs
else 
		{
			# This is a virtual target
			if ( ($currDevice.cpu_count -eq 0) -or (!$currDevice.cpu_count) )
			{
				# Physical CPU info not avialble
				$row | Add-Member -MemberType NoteProperty -Name "VirtualCPUCount" -Value "n/a"
				$row | Add-Member -MemberType NoteProperty -Name "VirtualCoreCount" -Value "n/a"
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCoresPerCPU" -Value "n/a"
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUManufacturer" -Value "n/a"
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUModel" -Value "n/a"
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUSpeed" -Value "n/a"
			}
			$row | Add-Member -MemberType NoteProperty -Name "VirtualCPUCount" -Value $currDevice.cpu_count
			$row | Add-Member -MemberType NoteProperty -Name "VirtualCoreCount" -Value $currDevice.core_count
			$row | Add-Member -MemberType NoteProperty -Name "VirtualRAM" -Value = $currDevice.total_memory_mb
			$row | Add-Member -MemberType NoteProperty -Name "DeviceModel" -Value $currDevice.model
			# Do something with MAC addresses for VMs
			
			# Now, if the VM host we're running on was scanned, we can do something with the following:
			if (!$currDevice.virtual_host.device_id)
			{
				# host not scanned, record an error by putting "n/a" in physical hostname, leave everything else blank
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalHostname" -Value "n/a"
			}
			else 
			{
				# host was scanned, so let's populate as many details as possible about the physical host
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUCount" -Value $currDevice.virtual_host.cpu_count
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCoreCount" -Value $currDevice.virtual_host.core_count
				if ( ($currDevice.virtual_host.cpu_count -ne 0) -and ($currDevice.virtual_host.core_count -ne 0))
				{
					$cpcpu = $currDevice.virtual_host.core_count / $currDevice.virtual_host.cpu_count
					$row | Add-Member -MemberType NoteProperty -Name "PhysicalCoresPerCPU" -Value $cpcpu
				}
				# In order to get the CPU Manufacturer information, you need to follow the link to 
				# $currDevice.virtual_host.self and parse the CPU info there. This is left as an exercise to the user
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUModel" -Value "n/a"
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUSpeed" -Value "n/a"
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalRAM" -Value $currDevice.virtual_host.total_memory_mb
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalDeviceManufacturer" -Value $currDevice.virtual_host.manufacturer
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalHostname" -Value $currDevice.virtual_host.host_name
				
				# In order to get the MAC Address for the physical host, you need to follow the link to 
				# $currDevice.virtual_host.self and parse the MAC info there. This is left as an exercise to the user
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalMACAddress" -Value "n/a"
				
			}
		}

Putting it all together

This is the script so far. It gathers all the available data from the devices and devices/device_id endpoints, and saves it as a CSV file. The order of the columns in the CSV file is non-deterministic. (It is set by the order of insertion of the columns of the last item in the dataset.)


Completed Script
#
# PowerShell Worked Examples
#
# Generate OutputDevices equivalent report via RestAPI
#

# The RestAPI uses HTTP authentication. Set $user and $pass to the username and password needed for your iQSonar instance
# set the $host to the IPAddress, name or FQDN of the iQSonar server
$user    = "admin"
$pass    = "password"
$sonar	 = "iQSonar Host"

$secpass = ConvertTo-SecureString $pass -AsPlainText -Force
$credential = New-Object System.Management.Automation.PSCredential($user,$secpass)

 
#
# To find out how many devices there are, ask for the first device and look at the header for the device count
# offset=1 means start with first device. fetch_size=1 means only return one record.
$uri     = -join ("http://", $sonar, "/api/v1/devices/?offset=1&fetch_size=1")
$r = Invoke-WebRequest $uri -Credential $credential
 
# $r.headers has HTML headers, $r.content has text content
$deviceCount = $r.headers.'X-fetch-count'

# Let the user know how many devices we can see
$output = -join ( "There are: ", $deviceCount, " devices in total")
write-host $output

# Build the CSV File header row
$csv = @()
$row = New-Object System.Object
$row | Add-Member -MemberType NoteProperty -Name "Hostname" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "FQDN" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "OS" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "OS-Install-Date" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "Location" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "Serial-Number" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUCount" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalCoreCount" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalCoresPerCPU" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUManufacturer" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUModel" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUSpeed" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalRAM" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "VirtualCPUCount" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "VirtualCoreCount" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "VirtualRAM" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "DeviceModel" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalModelSocketCount" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalModelCoreCount" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalDeviceManufacturer" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalHostname" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalFQDN" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "IP-Address" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalMACAddress" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "VirtualMACAddress" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "ClusterInformation" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "ClusterName" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PartitioningMethod" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "DerivedCPU" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "DerivedCoresPerCPU" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "Bios" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "LastScanDate" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "DeviceID" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalDeviceID" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUNotes" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "Notes" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "ExternalLink" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "DNSHostname" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "DNSFQDN" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalDNSHostname" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "PhysicalDNSFQDN" -Value $null
$row | Add-Member -MemberType NoteProperty -Name "MeasurementComment" -Value $null


$fs     = 100			# fetch_size
$offset = 1				# offset
$seen   = 1;			# first offset is 1, not 0
while ( $seen -lt $devicecount)
{
	# Outer loop, grab a batch (defined by fetch_size) of devices
	$url = -join ("http://", $sonar, "/api/v1/devices/?offset=", $offset,  "&fetch_size=", $fs)
	$devices = Invoke-RestMethod $url -Credential $credential
	$i = 1
	while ($i -lt $devices.count)
	{
		# inner loop - process individual devices in this batch
		# We want to populate some of the data fields in the CSV file - what can we fill in just from the DEVICES end point
		$row = New-Object System.Object		
		$row | Add-Member -MemberType NoteProperty -Name "DeviceID" -Value $devices[$i].device_id
		
		# If the path was only partly successful we might not have a serial number. However we can safely store the empty string.
		# if you are processing RestAPI using PHP or a different scripting language that is stricter about handling undefined variables,
		# you need error handling code around here too.
		$row | Add-Member -MemberType NoteProperty -Name "Serial-Number" -Value $devices[$i].serial_number
		
		# For certain devices, for example some SNMP devices, or for storage devices we might not get a hostname! Explicitly call out a device with no hostname. 
		# Also if hostname is not defined, there will be no qualified name(s)
		if (!$devices[$i].host_name)
		{
			# Hostname is not defined
			$row | Add-Member -MemberType NoteProperty -Name "Hostname" -Value "(undefined)"
			$row | Add-Member -MemberType NoteProperty -Name "FQDN" -Value "(undefined)"
			$row | Add-Member -MemberType NoteProperty -Name "DNSFQDN" -Value "(undefined)"
		}
		else {
			# Hostname is defined
			$row | Add-Member -MemberType NoteProperty -Name "Hostname" -Value $devices[$i].host_name
			# If the array is not present, $num gets value 0, and so the inner loop will not fire
			# we have already set FQDN and DNSFQDN to "(undefined)" where there is no hostname, but we can sometimes have a hostname but no FQDN
			$num = $devices[$i].qualified_name.count
			if ($num -eq 0)
			{
				$row | Add-Member -MemberType NoteProperty -Name "FQDN" -Value "(undefined)"
				$row | Add-Member -MemberType NoteProperty -Name "DNSFQDN" -Value "(undefined)"
			}
			else
			{
				$j   = 0
				$fqdn = "(No FQDN)"
				while ($j -lt $num)
				{
					if ($devices[$i].qualified_name[$j].name_type -eq "DNSFQDN")
					{
						# DNSFQDN can also be used for FQDN.
						$fqdn = $devices[$i].qualified_name[$j].name
						$row | Add-Member -MemberType NoteProperty -Name "DNSFQDN" -Value $devices[$i].qualified_name[$j].name
						$row | Add-Member -MemberType NoteProperty -Name "FQDN" -Value $devices[$i].qualified_name[$j].name
						if ($currDevice.is_virtual -eq "false")
						{
							$row | Add-Member -MemberType NoteProperty -Name "PhysicalDNSHostname" -Value $null
							$row | Add-Member -MemberType NoteProperty -Name "PhysicalDNSFQDN" -Value $devices[$i].qualified_name[$j].name
						}
					}
					elseif ($devices[$i].qualified_name[$j].name_type -eq "FQDN")
					{
						$row | Add-Member -MemberType NoteProperty -Name "PhysicalDNSHostname" -Value $null
						$row | Add-Member -MemberType NoteProperty -Name "FQDN" -Value $devices[$i].qualified_name[$j].name
					}
					$j = $j + 1
				}
			}		
		}
		# Locations is an array containing one or more locations. We can store locations[0].name as the location
		$row | Add-Member -MemberType NoteProperty -Name "Location" -Value $devices[$i].locations[0].name
		
		# The Last Scan Date should always be there, but if absent PowerShell will happily record a NULL value for us.
		$row | Add-Member -MemberType NoteProperty -Name "LastScanDate" -Value $devices[$i].last_scan
		
		# Get extra data about the host
		$currDevice = Invoke-RestMethod $devices[$i].self -Credential $credential
		$row | Add-Member -MemberType NoteProperty -Name "OS" -Value $currDevice.operating_system.name
		if (!$currDevice.os_install_date)
		{
			$row | Add-Member -MemberType NoteProperty -Name "OS-Install-Date" -Value "(not available)"
		}
		else
		{
			$row | Add-Member -MemberType NoteProperty -Name "OS-Install-Date" -Value $currDevice.os_install_date
		}
		if (!$currDevice.bios.name)
		{
			$row | Add-Member -MemberType NoteProperty -Name "Bios" -Value $null
		}
		else 
		{
			$row | Add-Member -MemberType NoteProperty -Name "Bios" -Value $currDevice.bios.name
		}		
		# IP Address is an array of 0 or more IP Addresses. We might not have any, or we might have LOTS.
		$num = $currDevice.ip_address.count
		if ($num -eq 0)
		{
			$row | Add-Member -MemberType NoteProperty -Name "IP-Address" -Value "(no IP address)"
		}
		else 
		{
			if ($num -eq 1)
			{
				$row | Add-Member -MemberType NoteProperty -Name "IP-Address" -Value $currDevice.ip_address[0].ip_address
			}
			else {
				# list of addresses, needs to be separated by semi-colons
				$addresslist = ""
				$j = 0
				while ($j -lt $num)
				{
					$addresslist = -join($addresslist, $currDevice.ip_address[$j].ip_address)
					if ($j -lt $num)
					{
						$addresslist = -join($addresslist, ";")
					}
					$j = $j + 1
				}
				$row | Add-Member -MemberType NoteProperty -Name "IP-Address" -Value $addresslist
			}
		}
		
		# Is this device physical or virtual
		if ($currDevice.is_virtual -eq "false")
		{
			# This is a physical target
			# We can fill in physical hardware details directly, and leave the entries for virtual hardware blank
			
			# Physical device RAM
			$row | Add-Member -MemberType NoteProperty -Name "PhysicalRAM" -Value $currDevice.total_memory_mb
			
			# Physical Device CPU Info
			# Sometimes we don't get CPU Information
			if ( (!$currDevice.cpu_count) -or ($currDevice.cpu_count -eq 0) )
			{
				# Physical CPU info not avialble
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUCount" -Value "n/a"
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCoreCount" -Value "n/a"
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCoresPerCPU" -Value "n/a"
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUManufacturer" -Value "n/a"
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUModel" -Value "n/a"
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUSpeed" -Value "n/a"
			}
			else
			{
				# we have at least some CPU info
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUCount" -Value $currDevice.cpu_count
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCoreCount" -Value $currDevice.core_count
				# Further details will be derived from cpu[0] if present
				if (!$currDevice.cpu[0].manufacturer)
				{
					# additional info missing
					$row | Add-Member -MemberType NoteProperty -Name "PhysicalCoresPerCPU" -Value "n/a"
					$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUManufacturer" -Value "n/a"
					$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUModel" -Value "n/a"
					$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUSpeed" -Value "n/a"
				}
				else
				{
					$row | Add-Member -MemberType NoteProperty -Name "PhysicalCoresPerCPU" -Value $currDevice.cpu[0].core_count
					$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUManufacturer" -Value $currDevice.cpu[0].manufacturer
					$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUModel" -Value $currDevice.cpu[0].cpu_model
					$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUSpeed" -Value $currDevice.cpu[0].cpu_speed
				}
			}
			$row | Add-Member -MemberType NoteProperty -Name "PhysicalModelSocketCount" -Value "n/a" # not exposed in V4 RestAPI
			$row | Add-Member -MemberType NoteProperty -Name "PhysicalModelCoreCount" -Value "n/a" # not exposed in V4 RestAPI
			$row | Add-Member -MemberType NoteProperty -Name "PhysicalDeviceManufacturer" -Value $currDevice.manufacturer

			$row | Add-Member -MemberType NoteProperty -Name "PhysicalHostname" -Value $devices[$i].host_name			
			# 
			# Add this later
			# $row | Add-Member -MemberType NoteProperty -Name "PhysicalMACAddress" -Value $null
			
		}
		else 
		{
			# This is a virtual target
			if (($currDevice.cpu_count -eq 0) -or (!$currDevice.cpu_count))
			{
				# Physical CPU info not avialble
				$row | Add-Member -MemberType NoteProperty -Name "VirtualCPUCount" -Value "n/a"
				$row | Add-Member -MemberType NoteProperty -Name "VirtualCoreCount" -Value "n/a"
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCoresPerCPU" -Value "n/a"
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUManufacturer" -Value "n/a"
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUModel" -Value "n/a"
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUSpeed" -Value "n/a"
			}
			else {			
				$row | Add-Member -MemberType NoteProperty -Name "VirtualCPUCount" -Value $currDevice.cpu_count
				$row | Add-Member -MemberType NoteProperty -Name "VirtualCoreCount" -Value $currDevice.core_count
			}
			$row | Add-Member -MemberType NoteProperty -Name "VirtualRAM" -Value $currDevice.total_memory_mb
			$row | Add-Member -MemberType NoteProperty -Name "DeviceModel" -Value $currDevice.model
			
			# Do something with MAC addresses for VMs
			
			# Now, if the VM host we're running on was scanned, we can do something with the following:
			if (!$currDevice.virtual_host.device_id)
			{
				# host not scanned, record an error by putting "n/a" in physical hostname, leave everything else blank
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalHostname" -Value "n/a"
			}
			else 
			{
				# host was scanned, so let's populate as many details as possible about the physical host
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUCount" -Value $currDevice.virtual_host.cpu_count
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCoreCount" -Value $currDevice.virtual_host.core_count
				if ( ($currDevice.virtual_host.cpu_count -ne 0) -and ($currDevice.virtual_host.core_count -ne 0) )
				{
					$cpcpu = $currDevice.virtual_host.core_count / $currDevice.virtual_host.cpu_count
					$row | Add-Member -MemberType NoteProperty -Name "PhysicalCoresPerCPU" -Value $cpcpu
				}
				# In order to get the CPU Manufacturer information, you need to follow the link to 
				# $currDevice.virtual_host.self and parse the CPU info there. This is left as an exercise to the user
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUModel" -Value "n/a"
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalCPUSpeed" -Value "n/a"
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalRAM" -Value $currDevice.virtual_host.total_memory_mb
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalDeviceManufacturer" -Value $currDevice.virtual_host.manufacturer
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalHostname" -Value $currDevice.virtual_host.host_name
				
				# In order to get the MAC Address for the physical host, you need to follow the link to 
				# $currDevice.virtual_host.self and parse the MAC info there. This is left as an exercise to the user
				$row | Add-Member -MemberType NoteProperty -Name "PhysicalMACAddress" -Value "n/a"
			}
		}
		$csv += $row
		$i = $i + 1;		# keep track for inner loop
		$seen = $seen + 1;  # keep track for outer loop
		if ( $seen % 10 -eq 0) {
			# progress indicator - display a "." every 10 devices
			write-host "." -nonewline
		}
	}
	# Finished this batch
	$offset = $seen
}
# Save the output to a file
write-host " Done. Saving output to OutputDevices.csv now."
$csv | Export-csv OutputDevices.csv -NoTypeInformation

But... we want all the colums in the same order as the original report

The script above works to extract the data, but if you want to feed the CSV file to a data processing application which was written for a V3 datasource, you probably want to have the columns in the "correct" order (including empty columns) so that your third party application does not need to be reconfigured or re-written. In that case you would need to re-write the code as follows:

  • At the start of the inner device loop, set up empty variables for each column,
  • As we discover the "correct" data for each variable, update them.
  • At the end of the inner loop, write all the variables into the row in the correct order, then add the row to the CSV file variable.

Related articles



  • No labels